The Archives

  • 15.Sep.11
    Resume rsync file transfers network | rafacas | (1)
    I did not know that rsync had the resume capability till last week when I had to transfer almost 200GB between two servers with no good connection. I think some context is needed here: My company has two servers in two different cities where the backups are stored. There is a daily syncronization between the backup tree between the servers. That is, a cron task that calls a script that basically runs rsync. Due to the bad connection, I have had to add the -P option to the rsync command so that the command run by the cron task is: $ rsync ...
  • 25.Nov.09
    Comparing directories in local or remote hosts shell | rafacas | (0)
    I frequently need to compare two directories (with similar structures) to find different or new files. When I have to compare them locally, I use the diff command. $ ls test* test: file1 file2 file3 file4 file5 test.new: file1 file2 file3 file4 file5 file6 $ diff -qr test test.new Files test/file2 and test.new/file2 differ Files test/file4 and test.new/file4 differ Only in test.new: file6 The -q option tells diff to tell only whether the files differ and the -r option is for recursively comparing inside subdirectories. A more complicated problem arises when the comparation has to be done remotely, that is one of the directories is in another host (Samba, NFS, etc ...
  • 30.Dec.08
    rsync with ssh-rsa+authorized_keys security, shell | pfortuny | (0)
    Much like explained in a previous post, one can use a passwordless RSA key to set up a cron job doing an rsync of one's computer on a remote server, via ssh. The relevant part of the authorized_keys file is (everything in the same line): command="rsync --server -vlogDtpr . /home/pfortuny/backup/",no-port-forwarding, no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa ..... Taking into account that the -vlogDtpr depends on the specific options of the rsync command you issue. In my case, the script is simply (right now) $ cat ./bin/backup.sh #!/bin/sh cd /home/pfortuny /usr/bin/rsync -av -e "ssh -i /home/pfortuny/.ssh/backup_key" --filter ": .rsync.dirs" \ ~/ pfortuny@remote.server.mine:"~/backup/" Comments: The option -av is expanded to -vlogDtpr on the server, ...