Skip to content

ZooKeeper multi-server cluster setup

A ZooKeeper multi-server cluster setup (also known as a replicated mode) ensures high availability and reliability for the coordination service. This setup requires running multiple ZooKeeper instances (servers) that communicate with each other to maintain a synchronized state.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

To enable replication, specific configuration parameters must be defined in the zoo.cfg file on each server.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

Configuration Parameters

The core zoo.cfg properties for a cluster involve timing settings for the initial connection and synchronization limits, as well as the explicit listing of cluster participants.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

Key parameters include:

  • tickTime: The basic time unit (in milliseconds) used by ZooKeeper for heartbeats.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]
  • initLimit: The time limit (in ticks) for followers to connect to and sync with the leader during the initial startup phase.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]
  • syncLimit: The time limit (in ticks) for a follower to sync with the leader after the connection is established.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]
  • dataDir: The directory where the snapshot and database files are stored.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]
  • clientPort: The port listening for client connections.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

Defining Cluster Members

Servers are defined using the server.x format, where x is the unique server ID.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md] The value follows the syntax: hostname:peerPort:leaderElectionPort.

  • peerPort: The port used for peer-to-peer communication between servers.
  • leaderElectionPort: The port used for leader election.

Example configuration entries:^[600-developer-big-data-zookpeeper-zookpeeeper-01.md] 

server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

Server Identity (myid)

For the configuration to work, each server must be assigned a unique ID corresponding to the server.x entry in zoo.cfg.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md] This is done by creating a file named myid in the directory specified by the dataDir parameter (e.g., /var/lib/zookeeper/myid or /opt/zookeeper/data/zkData/myid).^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

The file should contain only the numeric ID of the server (e.g., 1, 2, or 3) and nothing else.^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]

  • [[Configuration Management]]
  • [[High Availability]]
  • [[Leader Election]]

Sources

^[600-developer-big-data-zookpeeper-zookpeeeper-01.md]