Skip to content

Multi-node shell script utilities

Multi-node shell script utilities are custom Bash scripts designed to synchronize files and execute commands across multiple Linux server nodes simultaneously.^[600-developer-linux-centos7-command.md] They are commonly used in cluster management (e.g., Hadoop environments) to streamline administrative tasks.

Core Utilities

xsync.sh

The xsync.sh script is a file synchronization utility that copies a specific file or directory from the local node to all defined remote nodes.^[600-developer-linux-centos7-command.md] It uses rsync to ensure that only differences are transferred, making the process efficient^[600-developer-linux-centos7-command.md].

Logic flow: 1. Parameter validation: Checks if arguments are provided; exits if none are found^[600-developer-linux-centos7-command.md]. 2. Path resolution: Retrieves the filename and the absolute path of the parent directory^[600-developer-linux-centos7-command.md]. 3. User identification: Determines the current user to construct the remote destination path^[600-developer-linux-centos7-command.md]. 4. Synchronization loop: Iterates through a range of host numbers (e.g., 101 to 103) and executes rsync to copy the file to the corresponding path on each remote host^[600-developer-linux-centos7-command.md].

xcall.sh

The xcall.sh script is a command execution utility that runs a specified command on the local machine and all remote nodes^[600-developer-linux-centos7-command.md].

Logic flow: 1. Validation: Ensures at least one argument (the command) is provided^[600-developer-linux-centos7-command.md]. 2. Local execution: Executes the command on the local machine first ($@ expands to all arguments)^[600-developer-linux-centos7-command.md]. 3. Remote execution: Iterates through the defined host range and uses ssh to execute the command on each remote node^[600-developer-linux-centos7-command.md].

Configuration Requirements

To function correctly, these scripts typically require the following setup:

  • SSH Password-less Login: The scripts rely on SSH (e.g., ssh hadoop$host $@), necessitating that SSH keys be copied to the authorized_keys of all target nodes to allow automated access^[600-developer-linux-centos7-command.md].
  • Hostnames/IPs: The scripts assume a standardized naming convention for the nodes (e.g., hadoop101, hadoop102) which are resolved via the /etc/hosts file or DNS^[600-developer-linux-centos7-command.md].
  • PATH Environment Variable: It is recommended to add these scripts to the system's PATH (e.g., in ~/.bashrc) so they can be executed from any directory^[600-developer-linux-centos7-command.md].

Sources

^[600-developer-linux-centos7-command.md]

  • [[SSH]]
  • [[Rsync]]
  • [[Bash Scripting]]
  • [[Cluster Management]]