Install

macOS

$ brew install rsync

Ubuntu

$ sudo apt-get install rsync

Configuration of the rsync Daemon

The rsync daemon is an alternative to SSH for remote backups. Although more difficult to configure, it does provide some benefits. For example, using SSH to make a remote backup of an entire system requires that the SSH daemon allow root login, which is considered a security risk. Using the rsync daemon allows for root login via SSH to be disabled.

Edit the file /etc/default/rsync to start rsync as daemon using xinetd. The entry listed below, should be changed from false to inetd.

RSYNC_ENABLE=inetd

Install xinetd because it’s not installed by default.

$ sudo apt-get -y install xinetd

Create the file /etc/xinetd.d/rsync to launch rsync via xinetd. It should contain the following lines of text.

service rsync
{
    disable = no
    socket_type = stream
    wait = no
    user = root
    server = /usr/bin/rsync
    server_args = --daemon
    log_on_failure += USERID
    flags = IPv6
}

Create the file /etc/rsyncd.conf configuration for rsync in daemon mode. The file should contain the following. In the file, user should be replaced with the name of user on the remote machine being logged into.

max connections = 2
log file = /var/log/rsync.log
timeout = 300

[share]
comment = Public Share
path = /home/share
read only = no
list = yes
uid = nobody
gid = nogroup
auth users = user

Introduction

From the man page:

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

In other words, rsync is a tool for efficiently copying and backing up data from one location (the source) to another (the destination). It is efficient because it only transfers files which are different between the source and destination directories.

Usage

`-a` - 递归+同步元信息

-a参数可以替代-r，除了可以递归同步以外，还可以同步元信息（比如修改时间、权限等）。由于 rsync 默认使用文件大小和修改时间决定文件是否需要更新，所以-a比-r更有用。下面的用法才是常见的写法。

$ rsync -a source destination

目标目录destination如果不存在，rsync 会自动创建。执行上面的命令后，源目录source被完整地复制到了目标目录destination下面，即形成了destination/source的目录结构。

如果只想同步源目录source里面的内容到目标目录destination，则需要在源目录后面加上斜杠。

$ rsync -a source/ destination

`-n` - dry run

`--delete`

默认情况下，rsync 只确保源目录的所有内容（明确排除的文件除外）都复制到目标目录。它不会使两个目录保持相同，并且不会删除文件。如果要使得目标目录成为源目录的镜像副本，则必须使用--delete参数，这将删除只存在于目标目录、不存在于源目录的文件。

$ rsync -av --delete source/ destination

上面命令中，--delete参数会使得destination成为source的一个镜像。

`-z` - compress

If you are transferring files that have not already been compressed, like text files, you can reduce the network transfer by adding compression with the -z option:

$ rsync -az source destination

`-P` - progress

The -P flag is very helpful. It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:

$ rsync -azP source destination
Outputsending incremental file list
./
file1
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=99/101)
file10
           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=98/101)
file100
           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=97/101)
file11
           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=96/101)
. . .

`-X` - preserve extended attributes

This option causes rsync to update the remote extended attributes to be the same as the local ones.

`-v` - verbose

Demo

The simplest method for backing up over a network is to use rsync via SSH (using the -e ssh option). Alternatively, you can use the rsync daemon (see Rsync Daemon which requires much more configuration. Local backup only requires rsync and read/write access to the folders being synchronized. Below you will find examples of commands that can be used to backup in either case. It should be noted, that a network sync can be performed locally so long as the folder is shared (say by Samba) and then mounted to the machine with folder1. This process gets around having to use ssh but is less secure and should only be used in secure private networks, like at your home.

Local Backup

$ sudo rsync -azvv /home/path/folder1/ /home/path/folder2

Backup Over Network

$ sudo rsync --dry-run --delete -azvv -e ssh /home/path/folder1/ remoteuser@remotehost.remotedomain:/home/path/folder2

An explanation of above options to commands:

--dry-run This tells rsync to not actually do anything. It will just write a log of what it would do to the screen. Once you’ve made sure everything will work as you expect, you have to remove this option, and run the command again to perform the actual backup.
--delete deletes files that don’t exist on the system being backed up.(Optional)
-a preserves the date and times, and permissions of the files (same as -rlptgoD).
With this option rsync will:
- Descend recursively into all directories (-r),
  - copy symlinks as symlinks (-l),
  - preserve file permissions (-p),
  - preserve modification times (-t),
  - preserve groups (-g),
  - preserve file ownership (-o), and
  - preserve devices as devices (-D).
-z compresses the data
-vv increases the verbosity of the reporting process
-e specifies remote shell to use
/folder1 and folder2 In the examples above, folder1 and 2 are placeholders for the directories to be synchronized. Folder1 is the original folder, and 2 is the new folder, or existing one to be brought in sync with the first. Replace them with the folders you’d like. A / was added after folder1 so that only the contents, rather than whole folder, would be moved into the second.

SSH

Access via remote shell:
  Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
  Push: rsync [OPTION...] SRC... [USER@]HOST:DEST

Run the following command to check if everything is ok. The output listed is just a sample, should be what is on your shared remote machine. Hostname can be replaced by the IP address of the machine.

$ sudo rsync user@hostname::share
  Password: 
  drwxr-xr-x        4096 2006/12/13 09:41:59 .
  drwxr-xr-x        4096 2006/11/23 18:00:03 folders

Syncing to a remote system is trivial if you have SSH access to the remote machine and rsync installed on both sides. Once you have SSH access verified between the two machines, you can sync the dir1 folder from earlier to a remote computer by using this syntax (note that we want to transfer the actual directory in this case, so we omit the trailing slash):

$ rsync -a ~/dir1 username@remote_host:destination_directory

This is called a “push” operation because it pushes a directory from the local system to a remote system. The opposite operation is “pull”. It is used to sync a remote directory to the local system. If the dir1 were on the remote system instead of our local system, the syntax would be:

$ rsync -a username@remote_host:/home/username/dir1 place_to_sync_on_local_machine

指定端口

For instance, one of our customers executed the command:

$ rsync -v -v -e 'ssh -p <port>' ./testfile user@XXXXX:/home

Here, <port> denotes the SSH port number.

This ended up in the error code 12.

Initially, our Dedicated Engineers made sure that ssh -p YY user@XXXXX works fine. Since this was working fine, we checked the Rsync in the remote end.

rsync 协议

Access via rsync daemon:
  Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
        rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
  Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
        rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST

如果想知道 rsync 守护程序分配的所有 module 列表，可以执行下面命令。

$ rsync rsync://192.168.18.134

rsync 协议可以直接用rsync://来指定地址。

$ rsync -av source/ rsync://192.168.122.32/module/destination

My Case

# via SMB
$ time rsync -aPXcv --delete test-rsync /Volumes/SSDPool2/rsync
# via ssh
$ time rsync -avPX --delete -e 'ssh -p 34270' ../myBlogV2 root@192.168.18.93:/mnt/SSDPool2/DS/rsync

# via rsync protocol
$ rsync -aPXz --delete myBlogV2 rsync://192.168.18.93/sw_rsync/
# sync from ubuntu to truenas
$ rsync -az -vvvv node_exporter-1.0.1.linux-amd64.tar.gz rsync://192.168.18.93/sw_rsync/

Troubleshoot

`about to call exit(12)`

该 module 没有设置写权限

rsync -vvvv a.puml 192.168.18.93::sw_rsync/sd
opening tcp connection to 192.168.18.93 port 873
Connected to 192.168.18.93 (192.168.18.93)
msg checking charset: UTF-8
sending daemon args: --server -vvvve.LsfxCIvu . sw_rsync/sd  (4 args)
(Client) Protocol versions: remote=31, negotiated=31
[sender] make_file(a.puml,*,0)
[sender] flist start=0, used=1, low=0, high=0
[sender] i=0 <NULL> a.puml mode=0100644 len=1 flags=1000
send_file_list done
[sender] flist_eof=1
file list sent
send_files starting
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
[sender] _exit_cleanup(code=12, file=io.c, line=228): entered
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]
[sender] _exit_cleanup(code=12, file=io.c, line=228): about to call exit(12)

Reference

Sync from macOS to Linux

TrueNAS

Ubuntu

https://help.ubuntu.com/community/rsync

【Linux】命令 - rsync

Install

macOS

Ubuntu

Configuration of the rsync Daemon

Introduction

Usage

`-a` - 递归+同步元信息

`-n` - dry run

`--delete`

`-z` - compress

`-P` - progress

`-X` - preserve extended attributes

`-v` - verbose

Demo

Local Backup

Backup Over Network

SSH

指定端口

rsync 协议

My Case

Troubleshoot

`about to call exit(12)`

Reference

FEATURED TAGS

FRIENDS

TOC

Install

macOS

Ubuntu

Configuration of the rsync Daemon

Introduction

Usage

-a - 递归+同步元信息

-n - dry run

--delete

-z - compress

-P - progress

-X - preserve extended attributes

-v - verbose

Demo

Local Backup

Backup Over Network

SSH

指定端口

rsync 协议

My Case

Troubleshoot

about to call exit(12)

Reference

FEATURED TAGS

FRIENDS

`-a` - 递归+同步元信息

`-n` - dry run

`--delete`

`-z` - compress

`-P` - progress

`-X` - preserve extended attributes

`-v` - verbose

`about to call exit(12)`