Linux

Rsync command can be launched in one of two ways – this gives the option of 3 different ways to launch rsync transfer, here is how. rsync command can be used as a client with “rsync” (sans daemon) or as an rsync daemon with “rsync –daemon“. The rsync client can launch 3 different transfer types: regular file or folder transfers like cp, or launch a transfer session over ssh, or launch an rsync session over rsync port 873 — rsync daemon can start an rsync server which waits for connections and can receive or send rsync transfer. Any rsync transfer can send 1 or many files or folders. Any rsync transfer can source or target a local file or a remote file.

Good articles:

* https://download.samba.org/pub/rsync/rsync.html (applies to both)
* https://download.samba.org/pub/rsync/rsyncd.conf.html (applies to daemon)

 

(1) RSYNC FILE TRANSFER

Requirement: apt-get install rsync  on the server

To do an RSYNC file transfer its as simple as having rsync simply installed on the server

Commands:

rsync <options> <local source file> <local destination file>

# Crude Examples:
rsync test1.txt test1234.txt
rsync -avhP /folder1 /folder2
rsync -avhP --stats /folder1 /folder2
rsync -avhP --stats --log-file=/root/rsynccopy.log /folder1 /folder2

 

(2) RSYNC OVER SSH

Good Articles:

* http://www.cyberciti.biz/tips/linux-use-rsync-transfer-mirror-files-directories.html
* https://www.digitalocean.com/community/tutorials/how-to-copy-files-with-rsync-over-ssh

Requirement: apt-get install rsync openssh-client openssh-server  on source and destination

To do an RSYNC over SSH files transfer you must have RSYNC and SSH installed on both destination and source box and you must also have IP connectivity between source and destination (so that they can ping each other and more importantly talk to each other on the SSH port which is tcp 22, but you can change that number):

Commands:

rsync <options> <local source file> user@destination:/full/path/

rsync <options> <local source file> destination:/full/path/

Note the above command format shows copying a local file to a remote destination, but you can flip it and copy a remote file over to a local location.

If you dont put in a /full/path then the file would copy to the home directory of the user your using (if you dont specify a user it will just assume to use the same username as the one on the local machine). Everything is relative to the home path of the ssh user.

# Crude Examples:
rsync test1.txt root@10.100.1.236:~/test1234.txt
rsync -a /folder1 root@1:~/folder2 

rsync -ravP test1.txt root@10.100.1.236:~/test123.txt 
rsync -av test1.txt 10.100.1.250:Backups/test1.txt
rsync -av -e "ssh -l user1" test1.txt 10.100.1.250:Backups/test1.txt

 

(3) RSYNC DAEMON also known as RSYNC SERVER

Good Articles:

* http://transamrit.net/docs/rsync/
* http://www.jveweb.net/en/archives/2011/01/running-rsync-as-a-daemon.html
* http://www.togaware.com/linux/survivor/Rsync_Server.html

Requirement: apt-get install rsync  on both servers

The other way to use RSYNC is with an RSYNC DAEMON. It requires having an RSYNC SERVER setup on the destination or source side (it doesnt matter which side). The RSYNC SERVER or RSYNC DAEMON is configured with a “samba” like configuration file. For more information on this check this out: https://download.samba.org/pub/rsync/rsyncd.conf.html 

Here is an example of an rsync configuration on a server, this works if your pointing transfer away and to it (so it works both as destination and source):

## -- example 1 server config -- #

# cat /etc/rsyncd.conf
charset = utf-8
[Backups]
  path = /data/Backups
  comment = Rsync Backup's offsite destination
  use chroot = true
  uid = root
  gid = root
  read only = false
#[Media]
  #path = /data/Media
  #comment =
  #use chroot = true
  #uid = root
  #gid = root

Note: that you can comment things with # and that the config looks similar to SAMBA configuration (after all they are made by the same people)

This server can be launched like this:

/usr/bin/rsync --no-detach --daemon --config /etc/rsyncd.conf

Here is another example of a config (very simple one from a ReadyNAS 4.x):

## -- example 2 server config -- ##

# cat /etc/rsyncd.conf
charset = utf-8
[Backups]
  path = /c/Backups
  comment = "Backup Share"
  use chroot = true
  uid = root
  gid = root
  read only = false

Here is how the rsync server looks like from netstat view:

# netstat -nlp | grep rsync
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN     2916/rsync
tcp6       0      0 :::873                  :::*                    LISTEN     2916/rsync

Commands:

rsync <options> <local source file> rsync://user@destination/share/file

rsync <options> <local source file> user@destination::share/file

Note:  In the above command format the copy is from local source file to a remote rsync server location. However the source file can be from a remote location and you could be copying it locally

Note: With RSYNC OVER SSH the format is user@destination:/full/path, but with RSYNC SERVER the format is user@destination::sharename

Everything copies to or from the specified shares in this mode. So in the above example 1 server config we need to specify “Backups” in the share field of the command and copy only to or from Backups like so.

# Crude Examples:
rsync -v test1.txt rsync://root@10.100.1.236/Backups/testing123.txt 
rsync -v test1.txt 10.100.1.236::Backups/testing123.txt

Bonus Topic Copying into a folder or over a folder

### TRICK WHICH APPLIES TO ALL METHODS ###

# below copy tricks with the slash "/" work for rsync file transfers, rsync over ssh, and rsync daemon transfers. But I will only show & explain this trick on this method

# you get different behaviour if the destination folder exists & whether or not you use an ending slash on the source

# IF FOLDER EXISTS: one command copied folder1 into folder2, the other copies the contents of folder1 into folder2
rsync -avvv /folder1/ 10.100.1.250:/folder2   # copies folder1 to folder2, to get folder2/folder1
rsync -avvv /folder1/ 10.100.1.250:/folder2/  # copies everything in folder1 to folder2 (similar to folder1/* to folder2 - you will not have folder2/folder1)

# IF FOLDER DOESNT EXIST: same behavior for both commands
rsync -avvv /folder1/ 10.100.1.250:/folder2  # copies everything in folder1 to folder2 (similar to folder1/* to folder2 - you will not have folder2/folder1)

rsync -avvv /folder1/ 10.100.1.250:/folder2/  # copies everything in folder1 to folder2 (similar to folder1/* to folder2 - you will not have folder2/folder1)

Summary

## Local Transfers Example ##
rsync test1.txt test1234.txt
rsync -avhP /folder1 /folder2
rsync -avhP --stats /folder1 /folder2
rsync -avhP --stats --log-file=/root/rsynccopy.log /folder1 /folder2

## Rsync over SSH - must have rsync and ssh ##
rsync test1.txt root@10.100.1.236:~/test1234.txt
rsync -a /folder1 root@1:~/folder2 
rsync -ravP test1.txt root@10.100.1.236:~/test123.txt 
rsync -av test1.txt 10.100.1.250:Backups/test1.txt
rsync -av -e "ssh -l user1" test1.txt 10.100.1.250:Backups/test1.txt

## Rsync Server - must have rsync server on remote side ##
rsync -v test1.txt rsync://root@10.100.1.236/Backups/testing123.txt 
rsync -v test1.txt 10.100.1.236::Backups/testing123.txt

The end

Leave a Reply

Your email address will not be published. Required fields are marked *