Skip to content

xsync distributed file synchronization script

xsync is a shell script designed to synchronize files or directories across multiple nodes in a distributed cluster environment^[600-developer__linux__centos7-command.md].

Functionality

The script automates the distribution of a specific file to all configured cluster nodes using the rsync utility^[600-developer__linux__centos7-command.md]. Unlike commands that might simply copy everything, rsync is used here to replicate data intelligently: it skips existing files with identical content and only replaces files that have changed^[600-developer__linux__centos7-command.md].

Workflow

When executed, the script performs the following steps^[600-developer__linux__centos7-command.md]:

  1. Parameter Validation: It checks the number of input arguments. If no arguments are provided, the script prints "no args" and exits^[600-developer__linux__centos7-command.md].
  2. Path Resolution: It parses the input path to extract the filename and resolves the absolute path of the parent directory^[600-developer__linux__centos7-command.md].
  3. User Identification: It retrieves the current username using whoami^[600-developer__linux__centos7-command.md].
  4. Distribution Loop: The script iterates through a defined range of hostnames (e.g., hadoop101 to hadoop103)^[600-developer__linux__centos7-command.md].
  5. Synchronization: For each host, it executes rsync to send the file from the local absolute path to the corresponding path on the remote host^[600-developer__linux__centos7-command.md].

Usage

Typically, the script is stored in a system path such as /usr/local/bin/xsync.sh to allow execution from any location^[600-developer__linux__centos7-command.md]. It assumes that passwordless [[SSH]] login has been configured between the controlling node and the target nodes, as rsync operates over SSH^[600-developer__linux__centos7-command.md].

Sources

^[600-developer__linux__centos7-command.md]