Storm Cluster Configuration¶

Storm Cluster Configuration primarily involves editing the storm.yaml file to set up the distributed environment.^[storm-01.md] This file defines the coordination services, master nodes, local storage directories, and communication ports required for the cluster to operate.^[storm-01.md]

Core Configuration Parameters¶

Cluster Coordination and Management¶

The cluster requires a list of ZooKeeper servers to coordinate state management.^[storm-01.md] Additionally, the nimbus.seeds parameter must be configured to define the master nodes (Nimbus) of the cluster.^[storm-01.md]

Local Storage and Ports¶

Configuration parameters also specify the local directory used for storing Storm data (storm.local.dir) and the list of ports available for workers (supervisor.slots.ports).^[storm-01.md]

Example Configuration¶

A typical storm.yaml configuration includes the following key settings^[storm-01.md]:

storm.zookeeper.servers:
  - "hadoop101"
  - "hadoop102"
  - "hadoop103"

nimbus.seeds: ["hadoop101", "hadoop102", "hadoop103"]

storm.local.dir: "/opt/storm/date"

supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703

Cluster Startup Commands¶

Once configured, the cluster components are started using specific commands on the respective nodes^[storm-01.md]:

Nimbus: bin/storm nimbus &
Supervisor: bin/storm supervisor &
UI: bin/storm ui & (typically run on one node, accessible via port 8080)
Log Viewer: bin/storm logviewer & (required to view logs in the UI)

[[Nimbus]]
[[Supervisor]]
Zookeeper
[[Worker process]]
[[Topology]]

Sources¶

^[storm-01.md]