Rsync Examples (Linux Command)

HowTo Use rsync For Transferring Files Under Linux or UNIX

How do you install and use rsync to synchronize files and directories from one location (or one server) to another location? – A common question asked by new sys admin.

rsync is a free software computer program for Unix and Linux like systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.

So what is unique about the rsync command?

It can perform differential uploads and downloads (synchronization) of files across the network, transferring only data that has changed. The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection.

How do I install rsync?

Use any one of the following commands to install rsync. If you are using Debian or Ubuntu Linux, type the following command:

OR

If you are using Red Hat Enterprise Linux (RHEL) / CentOS 4.x or older version, type the following command:

RHEL / CentOS 5.x or newer (or Fedora Linux) user type the following command:

Always use rsync over ssh

Since rsync does not provide any security while transferring data it is recommended that you use rsync over ssh session. This allows a secure remote connection. Now let us see some examples of rsync command.

Comman rsync command options

  • --delete : delete files that don’t exist on sender (system)
  • -v : Verbose (try -vv for more detailed information)
  • -e "ssh options" : specify the ssh as remote shell
  • -a : archive mode
  • -r : recurse into directories
  • -z : compress file data

Task : Copy file from a local computer to a remote server

Copy file from /www/backup.tar.gz to a remote server called openbsd.nixcraft.in

Output:

Please note that symbol ~ indicate the users home directory (/home/jerry).

Task : Copy file from a remote server to a local computer

Copy file /home/jerry/webroot.txt from a remote server openbsd.nixcraft.in to a local computer’s /tmp directory:

Task: Synchronize a local directory with a remote directory

Task: Synchronize a remote directory with a local directory

Task: Synchronize a local directory with a remote rsync server or vise-versa

OR

Task: Mirror a directory between my “old” and “new” web server/ftp

You can mirror a directory between my “old” (my.old.server.com) and “new” web server with the command (assuming that ssh keys are set for password less authentication)

Read related previous articles

Other options – rdiff and rdiff-backup

The rdiff command uses the rsync algorithm. A utility called rdiff-backup has been created which is capable of maintaining a backup mirror of a file or directory over the network, on another server. rdiff-backup stores incremental rdiff deltas with the backup, with which it is possible to recreate any backup point. Next time I will write about these utilities.

rsync for Windows Server/XP/7/8

Please note if you are using MS-Windows, try any one of the program:

  1. DeltaCopy
  2. NasBackup
Further readings

=> Read rsync man page
=> Official rsync documentation

 

How to Backup Linux? 15 rsync Command Examples

rsync stands for remote sync.

rsync is used to perform the backup operation in UNIX / Linux.

rsync utility is used to synchronize the files and directories from one location to another in an effective way. Backup location could be on local server or on remote server.

Important features of rsync

  • Speed: First time, rsync replicates the whole content between the source and destination directories. Next time, rsync transfers only the changed blocks or bytes to the destination location, which makes the transfer really fast.
  • Security: rsync allows encryption of data using ssh protocol during transfer.
  • Less Bandwidth: rsync uses compression and decompression of data block by block at the sending and receiving end respectively. So the bandwidth used by rsync will be always less compared to other file transfer protocols.
  • Privileges: No special privileges are required to install and execute rsync

Syntax

Source and destination could be either local or remote. In case of remote, specify the login name, remote server name and location.

Example 1. Synchronize Two Directories in a Local Server

To sync two directories in a local computer, use the following rsync -zvr command.

In the above rsync example:

  • -z is to enable compression
  • -v verbose
  • -r indicates recursive

Now let us see the timestamp on one of the files that was copied from source to destination. As you see below, rsync didn’t preserve timestamps during sync.

Example 2. Preserve timestamps during Sync using rsync -a

rsync option -a indicates archive mode. -a option does the following,

  • Recursive mode
  • Preserves symbolic links
  • Preserves permissions
  • Preserves timestamp
  • Preserves owner and group

Now, executing the same command provided in example 1 (But with the rsync option -a) as shown below:

As you see below, rsync preserved timestamps during sync.

Example 3. Synchronize Only One File

To copy only one file, specify the file name to rsync command, as shown below.

Example 4. Synchronize Files From Local to Remote

rsync allows you to synchronize files/directories between the local and remote system.

While doing synchronization with the remote server, you need to specify username and ip-address of the remote server. You should also specify the destination directory on the remote server. The format is [email protected]:path

As you see above, it asks for password while doing rsync from local to remote server.

Sometimes you don’t want to enter the password while backing up files from local to remote server. For example, If you have a backup shell script, that copies files from local to remote server using rsync, you need the ability to rsync without having to enter the password.

To do that, setup ssh password less login as we explained earlier.

Example 5. Synchronize Files From Remote to Local

When you want to synchronize files from remote to local, specify remote path in source and local path in target as shown below.

Example 6. Remote shell for Synchronization

rsync allows you to specify the remote shell which you want to use. You can use rsync ssh to enable the secured remote connection.

Use rsync -e ssh to specify which remote shell to use. In this case, rsync will use ssh.

Example 7. Do Not Overwrite the Modified Files at the Destination

In a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source.

Use rsync -u option to do exactly that. (i.e do not overwrite a file at the destination, if it is modified). In the following example, the file called Basenames is already modified at the destination. So, it will not be overwritten with rsync -u.

Example 8. Synchronize only the Directory Tree Structure (not the files)

Use rsync -d option to synchronize only directory tree from source to the destination. The below example, synchronize only directory tree in recursive manner, not the files in the directories.

Example 9. View the rsync Progress during Transfer

When you use rsync for backup, you might want to know the progress of the backup. i.e how many files are copies, at what rate it is copying the file, etc.

rsync –progress option displays detailed progress of rsync execution as shown below.

You can also use rsnapshot utility (that uses rsync) to backup local linux server, or backup remote linux server.

Example 10. Delete the Files Created at the Target

If a file is not present at the source, but present at the target, you might want to delete the file at the target during rsync.

In that case, use –delete option as shown below. rsync delete option deletes files that are not there in source directory.

Target has the new file called new-file.txt, when synchronize with the source with –delete option, it removed the file new-file.txt

Example 11. Do not Create New File at the Target

If you like, you can update (Sync) only the existing files at the target. In case source has new files, which is not there at the target, you can avoid creating these new files at the target. If you want this feature, use –existing option with rsync command.

First, add a new-file.txt at the source.

Next, execute the rsync from the target.

If you see the above output, it didn’t receive the new file new-file.txt

Example 12. View the Changes Between Source and Destination

This option is useful to view the difference in the files or directories between source and destination.

At the source:

At the destination:

In the above example, between the source and destination, there are two differences. First, owner and group of the file Dirname differs. Next, size differs for the file Basenames.

Now let us see how rsync displays this difference. -i option displays the item changes.

In the output it displays some 9 letters in front of the file name or directory name indicating the changes.

In our example, the letters in front of the Basenames (and Dirnames) says the following:

Example 13. Include and Exclude Pattern during File Transfer

rsync allows you to give the pattern you want to include and exclude files or directories while doing synchronization.

In the above example, it includes only the files or directories starting with ‘P’ (using rsync include) and excludes all other files. (using rsync exclude ‘*’ )

Example 14. Do Not Transfer Large Files

You can tell rsync not to transfer files that are greater than a specific size using rsync –max-size option.

max-size=100K makes rsync to transfer only the files that are less than or equal to 100K. You can indicate M for megabytes and G for gigabytes.

Example 15. Transfer the Whole File

One of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.

If network bandwidth is not an issue for you (but CPU is), you can transfer the whole file, using rsync -W option. This will speed-up the rsync process, as it doesn’t have to perform the checksum at the source and destination.

Additional rsync Tutorials



Comments

comments

Leave a Reply