Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
backup [2015/07/14 00:06]
slackermedia
backup [2018/06/13 05:57]
slackermedia [Local Backup]
Line 1: Line 1:
-[[{arrowp.png|border:​0;​background:​none;​width:​0;​display:​inline-block;​position:​absolute;​top:​0;​left:​0;​}sox|]] 
  
-[[{arrown.png|border:​0;​background:​none;​width:​0;​display:​inline-block;​position:​absolute;​top:​0;​margin-left:​2.5em;​}rescue|]] 
  
-/*+/* if they are here, we dont need to convince them to backup
 ======System Backups====== ======System Backups======
 +
  
 The reason so many appreciate "the cloud" today is because of the assurance that their data is safe. It's not safety from the NSA or computer attacks that they appreciate, it's the safety from dying hard drives, because everyone knows that the cloud is redundant, and the cloud is automated. It just backs stuff up, all the time. The reason so many appreciate "the cloud" today is because of the assurance that their data is safe. It's not safety from the NSA or computer attacks that they appreciate, it's the safety from dying hard drives, because everyone knows that the cloud is redundant, and the cloud is automated. It just backs stuff up, all the time.
Line 17: Line 16:
 */ */
  
-======rdiff-backup======+=====Backing Up with Rdiff-Backup===== 
 + 
 +<WRAP info> 
 +**TL;DR** \\ 
 +1. [[drives|Format a big drive]] as UDF. \\ 
 +2. Install ''​rdiff-backup''​ \\ 
 +3. ''​rdiff-backup --exclude-globbing-filename $HOME/​.excludes $HOME /​path/​to/​drive''​ \\ 
 +4. Automate that command with ''​cron''​ \\ 
 +</​WRAP>​
  
 There are several good backup applications aimed at local and network backups. Some are geared toward sys admins managing entire VFX houses, others are geared toward the advanced home user, and still others are aimed at regular, humble, everyday artists like us. No matter the audience, the best solutions can be //​automated//​ so that you don't have to remember to do it, or wait for huge data dumps when you finally do remember to backup. A steady, automated, "​always on" backup scheme tends to be the smartest backup plan. There are several good backup applications aimed at local and network backups. Some are geared toward sys admins managing entire VFX houses, others are geared toward the advanced home user, and still others are aimed at regular, humble, everyday artists like us. No matter the audience, the best solutions can be //​automated//​ so that you don't have to remember to do it, or wait for huge data dumps when you finally do remember to backup. A steady, automated, "​always on" backup scheme tends to be the smartest backup plan.
  
 The most straight-forward and flexible backup application available is [[http://​rdiff-backup.nongnu.org|rdiff-backup]],​ available from [[http://​slackbuilds.org]]. It has one dependency, ''​librsync'',​ which is also available on slackbuilds.org. The most straight-forward and flexible backup application available is [[http://​rdiff-backup.nongnu.org|rdiff-backup]],​ available from [[http://​slackbuilds.org]]. It has one dependency, ''​librsync'',​ which is also available on slackbuilds.org.
 +
 +<WRAP important>​
 +This section assumes that you are interested in backing up your personal data, not the entire system, since the OS is free and can be re-constructed. \\
 +\\
 +There is a case for backing up the entire system (quicker recovery in the event of a full system disaster being the main one), and this //can// be done; just replace ''/​home/​$USER''​ in this chapter with ''/''​ and run your backups as root. See [[http://​rdiff-backup.nongnu.org/​docs.html]] for more information on what directories you should exclude when doing a full system backup.
 +</​WRAP>​
  
 Rdiff-backup is a terminal command only; it has no graphical user interface. If you are not comfortable using the terminal, it's still ok to use rdiff-backup because the point of it is that you don't have to use it much; you automate it. That's what we'll do together here, and once you get through this chapter, you will never have to use rdiff-backup directly again, except to recover data but under those circumstances,​ you probably won't mind learning a new command or two. Rdiff-backup is a terminal command only; it has no graphical user interface. If you are not comfortable using the terminal, it's still ok to use rdiff-backup because the point of it is that you don't have to use it much; you automate it. That's what we'll do together here, and once you get through this chapter, you will never have to use rdiff-backup directly again, except to recover data but under those circumstances,​ you probably won't mind learning a new command or two.
Line 28: Line 41:
  
  
-======Rdiff-backup ​Recipes======+======Rdiff-backup ​Plans======
  
 There are a few likely scenarios for backing up: There are a few likely scenarios for backing up:
Line 38: Line 51:
   * [[#​bkpcherrypick|You only want to backup a few important files as you generate them.]]   * [[#​bkpcherrypick|You only want to backup a few important files as you generate them.]]
  
-Here are recipes on how to create and use each of those backup schemes in such a way that it is as automated as possible. 
  
  
Line 81: Line 93:
  
 <​code>​ <​code>​
-LABEL=safehouse ​    /​safehouse ​   udf   auto,user 0 0+LABEL=safehouse ​    /​safehouse ​   udf   user,​rw  ​0 0
 </​code>​ </​code>​
  
Line 87: Line 99:
 </​WRAP>​ </​WRAP>​
  
-**3.** ​Create a directory for your backups on the drive.+**3.** ​Perform an initial backup.
  
 <WRAP indent> <WRAP indent>
-In addition ​to backup ​directorycreate an empty file that you can use as a test to make sure that the drive is present (this will be used laterduring ​the [[#automate|automation]] phase).+The first backup is the largest and slowest backup because it is copying everything over to your backup ​drive. Subsequent backups will be much much smallerbecause it will only copy changes to that base layer. 
 + 
 +It is vital that this initial backup is done the exact same way it's going to be done on all future backups, for many reasons, but essentially because ​you need to get confirmation ​that everything has been set up correctly //before// you try to automate it. If you do a backup with a drive connected to your computer, and then try to switch to a networked backup, you might do user permissions wrong, or file paths might be inconsistentor any number of problems, and you won't ever realise it because you'll have automated ​the process, never knowing that all you have automated is failure. 
 + 
 +Do not do that. 
 + 
 +Make your first backup the initial test of your backup process. If it fails, refine your process, troubleshoot,​ and try again. When you get it right, you'll know it, and you'll know exactly what to automate. 
 + 
 +The basic command to backup your home directory (where all of your personal data lives): 
 + 
 +For a locally attached drive:
  
 <​code>​ <​code>​
-mkdir /safehouse/$USER +rdiff-backup --terminal-verbosity 8 --print-statistics \ 
-$ touch /safehouse/go+--exclude-globbing-filelist ​/home/klaatu/​.excludes /​home/​klaatu/​ \ 
 +/safehouse/klaatuWork/
 </​code>​ </​code>​
  
-At this point, ​you have created:+<WRAP tip> 
 +You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options. 
 +</​WRAP>​
  
-  * A standard location ​for your backup destination''/​safehouse''​) +Or for a server:
-  * A test file to help detect if the drive is accessible: ''/​safehouse/​go''​ +
-  * A directory to hold your backups: ''/​safehouse/​$USER''​ (where $USER has been converted by the computer to your username)+
  
-You have completed ​the setup phase.+<​code>​ 
 +$ rdiff-backup --terminal-verbosity 8 --print-statistics \ 
 +--exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ \ 
 +klaatu@10.1.1.11::/​safehouse/​klaatuWork/​ 
 +</​code>​ 
 + 
 +<WRAP alert> 
 +Notice that the server path contains //two// ''::''​ between the hostname and the destination path.
 </​WRAP>​ </​WRAP>​
 +
 +This command backs up your entire home directory to the ''/​safehouse''​ location. Do not bother creating the destination file (in this example'​s case, the ''/​safehouse/​klaatuWork''​ folder); ''​rdiff-backup''​ will [attempt] to create that for you and warn you if it cannot write to the drive.
 +
 +No matter what, your first backup will take a long time, so just let it work.
 +</​WRAP>​
 +
 +You have completed the setup phase.
  
  
Line 127: Line 164:
 </​code>​ </​code>​
  
-If nothing is listed, then the drive has not mounted successfully. Verify that you have labelled it properly with the ''​vid''​ flag (if using UDF as recommended;​ if you are using ext{2-4}, try using ''​e2label''​, and also question yourself about why you are not heeding Slackermedia'​s advice about UDF).+If nothing is listed, then the drive has not mounted successfully. Verify that you have labelled it properly with the ''​vid''​ flag (if using UDF as recommended;​ if you are using ext{2-4}, try using ''​e2label''​).
  
  
-**2.** ​Verify that you have access to the drive:+**2.** ​Change a file or add a new test file, and then run the your backup command again.
  
 <​code>​ <​code>​
-touch /safehouse/go || echo "​Nope"​+rdiff-backup -v 8 /​home/​klaatu/ ​/safehouse/klaatuWork/
 </​code>​ </​code>​
  
-Silence indicates success. If you get "​Nope"​ in responsethen your ''/​etc/​fstab''​ entry is incorrector you are not using UDF as recommended.+This timethe process ​is relatively fastsince it has only copied over the changes.
  
 +☛ You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options.
  
-**3.** Perform an initial backup.+☛ Assuming everything is working as expected, [[#​automate|automate]] the process and never think about it again.
  
-The first backup is the largest and slowest backup because it is copying everything over to your backup drive. Subsequent backups will be much much smaller, because it will only copy changes to that base layer. +☛ For information on recovering data //from// a backup, read the section on [[#​recovery|restoring files]].
- +
-The basic command to backup your home directory (where all of your personal data lives): +
- +
-<​code>​ +
-$ rdiff-backup -v 9 $HOME /​safehouse/​$USER +
-</​code>​ +
- +
-<WRAP tip> +
-You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options. +
-</​WRAP>​ +
- +
-This command backs up your entire home directory to the ''/​safehouse''​ location. It will take a long time, so just let it work. +
- +
- +
-**4.** Once the backup is complete, try changing a file or adding a new test file, and then run the command again. This time, the process is quite fast, since it has only copied over the changes. +
- +
-**5.** Assuming everything is working as expected, [[#​automate|automate]] the process and never think about it again. +
- +
-<WRAP tip> +
-For information on recovering data //from// a backup, read the section on [[#​recovery|file recovery]]. +
-</​WRAP>​+
  
  
Line 183: Line 200:
 ====Easy (the "​It'​s better than nothing"​ solution)==== ====Easy (the "​It'​s better than nothing"​ solution)====
  
-Identify ​your backup drive to your system, so that your computer knows to backup to it when it gets plugged in. +You need to identify ​your backup drive to your system, so that your computer knows to backup to it when it gets plugged in. The easiest way to do this is to lie to your computer and tell it that your backup drive is always attached; any backup attempts made when the drive is //not// attached will fail, while any backup attempt when the drive happens to be attached will succeed. To implement this method, just use the [[#​bkplocal|backup to a local drive]] method, and then skip down to the [[#​automate|automation]] section.
- +
-The easiest way to do this is to lie to your computer and tell it that your backup drive is always attached; any backup attempts made when the drive is //not// attached will fail, while any backup attempt when the drive happens to be attached will succeed. To implement this method, just use the [[#​bkplocal|backup to a local drive]] method, and then skip down to the [[#​automate|automation]] section.+
  
 The danger of using this easy method is that if you have your computer set to backup every 4 hours, for example, and you attach your drive but only leave it attached for 3.75 hours, then you miss a backup cycle. Keep doing that, and you may end up with far fewer backups than expected. It practise, it's better than nothing, and as long as you remember to leave your laptop on and awake overnight with your backup drive attached, you are sure to hit one or two backup cycles (depending on how often you sleep). The danger of using this easy method is that if you have your computer set to backup every 4 hours, for example, and you attach your drive but only leave it attached for 3.75 hours, then you miss a backup cycle. Keep doing that, and you may end up with far fewer backups than expected. It practise, it's better than nothing, and as long as you remember to leave your laptop on and awake overnight with your backup drive attached, you are sure to hit one or two backup cycles (depending on how often you sleep).
  
 +{{anchor:​udev}}
 ====udev (the "I spent my weekend with udev" solution)==== ====udev (the "I spent my weekend with udev" solution)====
  
 The more difficult solution is to use ''​udev'',​ the sub-system in Linux that supplies your computer with "​device events"​. It's the system that labels things like DVD-ROM drives, network cards, external hard drives, mousen, keyboards, joysticks and gamepads, and so on. That makes it a very handy utility, and like everything else in Linux, it's very well exposed to the user so you can manually script it to, for instance, perform certain tasks when a certain hard drive is plugged in. The more difficult solution is to use ''​udev'',​ the sub-system in Linux that supplies your computer with "​device events"​. It's the system that labels things like DVD-ROM drives, network cards, external hard drives, mousen, keyboards, joysticks and gamepads, and so on. That makes it a very handy utility, and like everything else in Linux, it's very well exposed to the user so you can manually script it to, for instance, perform certain tasks when a certain hard drive is plugged in.
 +
  
 **The script** **The script**
Line 201: Line 218:
  
 <​code>​ <​code>​
-#!/usr/bin/bash+#!/bin/sh
  
-echo $date > /​tmp/​udev.log+date > /​tmp/​udev.log
 </​code>​ </​code>​
  
Line 228: Line 245:
 With ''​udevadm''​ monitor running, plug in your backup drive and watch as all kinds of information is spewed out onto your screen. Notice, particularly,​ that the type of event is an ''​ADD''​ event. That's a good way of identifying what type of event we want. With ''​udevadm''​ monitor running, plug in your backup drive and watch as all kinds of information is spewed out onto your screen. Notice, particularly,​ that the type of event is an ''​ADD''​ event. That's a good way of identifying what type of event we want.
  
-The ''​udevadm''​ monitor provides a lot of good info, but we can see it with prettier formatting with the command ''​udevadm info'',​ assuming you know where your drive is currently located in your ''​dev''​ tree. If not, unplug and then plug your thumbdrive back in and then immediately issue this command:+The ''​udevadm''​ monitor provides a lot of good info, but we can see it with prettier formatting with the command ''​udevadm info -a -n'',​ assuming you know where your drive is currently located in your ''​dev''​ tree. If not, unplug and then plug your thumbdrive back in and then immediately issue this command:
  
 <​code>​ <​code>​
Line 242: Line 259:
 This will return a wealth of information. Focus on the first block of info, first. This will return a wealth of information. Focus on the first block of info, first.
  
-The feedback from ''​udevadm info''​ starts with the device specified by the ''​devpath'',​ and then walks up the chain of parent devices. It prints for every device found, all possible attributes in the ''​udev''​ rules key format. ​+The feedback from ''​udevadm info -a -n''​ starts with the device specified by the ''​devpath'',​ and then walks up the chain of parent devices. It prints for every device found, all possible attributes in the ''​udev''​ rules key format. ​
  
 A ''​udev''​ rule must contain one attribute from one single "​parent"​ device; parent attributes are things that describe a device from the most basic level, such as "​it'​s something that has been plugged into a physical port" or "it is something with a size of XX" or "this is a removable device"​. A ''​udev''​ rule must contain one attribute from one single "​parent"​ device; parent attributes are things that describe a device from the most basic level, such as "​it'​s something that has been plugged into a physical port" or "it is something with a size of XX" or "this is a removable device"​.
  
-That info, just under the ''​udevadm info''​ introductory paragraph, looks a little something like this:+That info, just under the ''​udevadm info -a -n''​ introductory paragraph, looks a little something like this:
  
 <​code>​ <​code>​
Line 289: Line 306:
 If you see the date and time returned from ''/​tmp/​udev.log'',​ then the script has worked. If you see the date and time returned from ''/​tmp/​udev.log'',​ then the script has worked.
 </​WRAP>​ </​WRAP>​
 +
  
 **Refining the Rule into Something Useful** **Refining the Rule into Something Useful**
Line 325: Line 343:
  
 </​WRAP>​ </​WRAP>​
 +
  
 **Refining the Script** **Refining the Script**
Line 331: Line 350:
 Once you can comfortably confirm that your script is triggered by ''​udev''​ when you attach your backup drive, turn your attention to the functionality of the script. Right now it is useless, doing nothing more than logging the fact that it has been executed. Once you can comfortably confirm that your script is triggered by ''​udev''​ when you attach your backup drive, turn your attention to the functionality of the script. Right now it is useless, doing nothing more than logging the fact that it has been executed.
  
-Here is an example backup-on-attach script:+Here is an example backup-on-attach script. You can use it exactly as is EXCEPT for the last command; you need to change the path of the directory you want backed up (unless your username is also ''​klaatu''​) and the destination (unless you also named your destination folder ''​klaatu''​):
  
 <​code>​ <​code>​
Line 339: Line 358:
  
 COUNT=0 COUNT=0
 +# wait for drive to mount and be writable
 while [ "​$COUNT"​ -lt "​6"​ ]; do while [ "​$COUNT"​ -lt "​6"​ ]; do
     sleep 5     sleep 5
     if [ -b /safehouse ]; then     if [ -b /safehouse ]; then
-        if [ -w /safehouse/go ]; then+        if [ -w /safehouse/klaatu ​]; then
             # everything is ok             # everything is ok
             COUNT=9             COUNT=9
Line 357: Line 376:
 done done
  
 +# this is hopeless, get out
 if [ "​${COUNT}"​ == "​6"​ ]; then if [ "​${COUNT}"​ == "​6"​ ]; then
     echo "Could not locate SAFEHOUSE. Backup NOT complete. THIS IS BAD. `date`"​ >> $HOME/​safehouse.log     echo "Could not locate SAFEHOUSE. Backup NOT complete. THIS IS BAD. `date`"​ >> $HOME/​safehouse.log
Line 362: Line 382:
 fi fi
  
 +# if we are here, everything is good to go
 if [ "​${COUNT}"​ -gt 6 ]; then if [ "​${COUNT}"​ -gt 6 ]; then
-    ​echo "wot it work"+    ​##BACKUP COMMAND 
 +    rdiff-backup /​home/​klaatu /​safehouse/​klaatu
 fi fi
 </​code>​ </​code>​
  
-For my KDE machinesI then unmount ​the drive so that I can then manually mount it in my graphical environment. On Fluxbox and E17, or when not running a desktop at all, I habitually use /mnt/zip as my mount point anyway, so on those machines I leave off the umount command.+<WRAP tip> 
 +You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For thisread the section on [[#​excludes|exclude]] options, and then add those options to the ''##​BACKUP COMMAND''​ section. 
 +</​WRAP>​
  
-That's pretty much it. Using this method, I reliably backup my thumbdrives every time they get plugged into my computers. You could use the same trick in reverse by designating an external harddrive as the drive that will, upon being plugged in, backup your entire computer. ​+</​WRAP>​
  
 +☛ Try changing a file or adding a new test file, and then attach your hard drive. A backup process should be started by ''​udev'',​ this time copying over only the changes you have made to your home directory. Check for errors in ''/​tmp''​ and in your home directory.
  
 +☛ Assuming everything is working as expected, [[#​automate|automate]] the process and never think about it again.
  
 +☛ For information on recovering data //from// a backup, read the section on [[#​recovery|restoring files]].
  
  
-<WRAP tip> +{{anchor:​bkpserver}} 
-You might have exclusions ​that you don't want to have backed up (for instance, do you ever want your trash bin backed up?)For this, read the section ​on [[#​excludes|exclude]] options.+=====Backup to a Personal Server===== 
 + 
 +If you have a home server, or a server away from home that you can physically access, you might prefer ​to do your backups remotely. The advantage to remote backups is that you neither get tied down by having a hard drive attached to your computernor do you have to remember to attach it if you like a mobile lifestyle. If your server is off-site, then you have some insurance against disaster, since the likelihood of both your personal computer and your off-site server facing destruction is less likely when they are not physically located in the same building. 
 + 
 +You should have gone through the setup steps before starting ​this processso you should have a hard drive in the UDF format with the initial backup of your data on it. Your server should have a ''/​safehouse''​ mount point on it, and the drive should be set as mountable in ''​fstab''​. Both your client and your server must have ''​rdiff-backup''​ installed. 
 + 
 +<WRAP alert> 
 +For rdiff-backup to work remotely, //both// your client (the machine whose data is being backed up) and the server (the computer receiving the backup data for storage) must have ''​rdiff-backup''​ installed. That's **two** installs of ''​rdiff-backup''​.
 </​WRAP>​ </​WRAP>​
  
-This command backs up your entire home directory to the ''/​safehouse''​ location. It will take a long time, so just let it work.+**1.** Mount the drive on your server:
  
 +<​code>​
 +$ mount /safehouse
 +</​code>​
  
-**4.** Once the backup is complete, try changing a file or adding a new test fileand then run the command again. This timethe process is quite fast, since it has only copied over the changes.+**2.** Make the server accessible by ''​ssh'' ​or ''​rsync''​ (for ''​rsync''​you'll also need to run the rsync daemonwhich requires additional setup.
  
-**5.** Assuming everything is working as expected, [[#​automate|automate]] the process and never think about it again.+For security'​s sake, and because ''​rdiff-backup''​ will not prompt you for a password, you must use an ssh key to log into the remote serverFor more information on ssh keyssee the [[#​automate|automation]] section.
  
-For information on recovering data //from// a backupread the section on [[#recovery|file recovery]].+On the server, you should disable ssh-password sign-in (this option is found in ''​/etc/ssh/sshd_config''​)and consider using [[http://​fail2ban.org|Fail2Ban]].
  
 +**3.** Attempt a remote backup:
  
 +<​code>​
 +$ rdiff-backup -v 8 $HOME username@your-server.info::/​safehouse/​username
 +</​code>​
  
 +This time, the process is quite fast, since it has only copied over the changes.
  
-{{anchor:​bkpserver}} +☛ You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options.
-====Backup ​to a Server====+
  
-If your computer ​is a workstation (or "​desktop"​ or "​tower"​) then it makes sense to backup your data to a drive that you just leave plugged in to the computer. The advantage is that the backups occur in the backgroundwithout any special action required of you. The dis-advantage is that the drive is physically connected to your computer, so if accidentally erase your entire computer ​and everything connected to it, then you have blown away both your data and the backup copies of your backup data (the likelihood of you devising such a command by accident is extremely rare).+☛ Assuming everything ​is working as expected[[#​automate|automate]] ​the process ​and never think about it again.
  
-As with //any// local backup, ​this method also is not fire-, flood-, or Velociraptor-proof. That is, if something catastrophic happens to your house or apartment, then your data is gone and so is your backup.+☛ For information on recovering data //from// backup, ​read the section on [[#​recovery|restoring files]].
  
-On the other hand, if you accidentally delete a file an hour before your deadline, you can pull the most recent version off of a drive sitting under your desk and move on with your life. 
  
-As always with devising ​backup plan, the cost-to-benefit factor ​is your own to ponder.+{{anchor:​bkpcloud}} 
 +=====Backup to Someone Else's Server===== 
 + 
 +Backing up to server that you do not own or do not have physical access to can be tricky, because you may not have access to it in such a way that remote backups are allowed. Besides, the remote server must also have ''​rdiff-backup''​ installed on it, and it may not have that.  
 + 
 +On the other hand, you might; it depends on what you are paying for, or what the sys admin is providing you. 
 + 
 +If you have ''​ssh''​ access to the server, then you can treat it the same as [[#​bkpserver|your own private server]] in terms of setup.  
 + 
 +Other methods of accessing the server will likely not work, since ''​rdiff-backup''​ must be on the remote server, and needs ssh access. 
 + 
 +You might look into other backup solutions, like [[http://​luckybackup.sourceforge.net]]. 
  
 {{anchor:​bkpcherrypick}} {{anchor:​bkpcherrypick}}
-====Backup ​to Local Drive====+=====Backup a la Carte=====
  
-If your computer is workstation (or "​desktop"​ or "​tower"​) then it makes sense to backup your data to a drive that you just leave plugged in to the computer. The advantage is that the backups occur in the background, without ​any special action required ​of you. The dis-advantage is that the drive is physically connected to your computerso if accidentally erase your entire computer ​and everything connected ​to it, then you have blown away both your data and the backup ​copies of your backup data (the likelihood of you devising such a command by accident is extremely rare).+Some people generate more data than others. ​If you are, for example, ​writer ​or any kind of artist dealing in small file formats, and you find it's easier ​to just backup files as you create them, then that's a valid backup ​option...as long as you remember to do it.
  
-As with //any// local backup, this method also is not fire-, flood-, ​or Velociraptor-proof. That isif something catastrophic happens to your house or apartmentthen your data is gone and so is your backup.+Consider purchasing a large-capacity thumbdrive and using it as either your master work directory ​or as your [[#​bkpconnect|plug-and-backup]] destination. That wayevery time you plug your drive into your computerit gets backed up; you no longer have to remember to do it manually!
  
-On the other hand, if you accidentally delete a file an hour before your deadline, you can pull the most recent version off of drive sitting under your desk and move on with your life.+If you are using rdiff-backup, you can backup ​single file between backup script occurrences:​
  
-As always with devising a backup ​plan, the cost-to-benefit factor is your own to ponder.+<​code>​ 
 +$ rdiff-backup --include $HOME/foo.txt --exclude '​**'​ $HOME /​safehouse 
 +</​code>​
  
 +Another option is to wield the power of the Internet, and use a good git service. There are several simple clients for git, including the very appealing [[http://​sparkleshare.org|SparkeShare]] or, if you have access to a server and the time to configure it, [[http://​owncloud.org|ownCloud]]. These both have Dropbox-style sync folders.
  
  
Line 418: Line 473:
 =====Excludes===== =====Excludes=====
  
 +No matter what your backup plan, space is limited. There are bound to be things hanging around in your home directory that just aren't important to you. You can and should exclude these files from being backed up.
  
 +You could specify the directories and files you want excluded in your ''​rdiff-backup''​ command, but if you are automating the backups (and you should be), then what not to backup also should be automated.
  
-[[{arrown.png|border:0;background:none;width:0;display:inline-block;float:right;​}rescue|]][[{arrowp.png|border:0;background:none;width:0;float:right;}sox|]]+The solution is to have a local ''​~/​.excludes''​ file, listing each directory or file you want excluded from the ''​rdiff-backup''​ process. Such a file is easy to createjust echo the path of a directory or file into ''​~/​.excludes''​if it does not exist, the file will be created for you, and the entry added. 
 + 
 +<​code>​ 
 +$ echo '/​home/​klaatu/​.local/​share/​Trash'​ >> ~/​.excludes 
 +</​code>​ 
 + 
 +As you find more files that you want to exclude, use the same command. 
 + 
 +<​code>​ 
 +$ echo '/​home/​klaatu/​stockFootage'​ >> ~/​.excludes 
 +$ echo '/​home/​klaatu/​downloads'​ >> ~/​.excludes 
 +</​code>​ 
 + 
 +The only thing you must do is change the command you use to start the backups such that it knows where to look for a list of files to ignore: 
 + 
 +<​code>​ 
 +rdiff-backup --exclude-globbing-filelist \ 
 + /​home/​klaatu/​.excludes \ 
 + /​home/​klaatu/​ \ 
 + ​klaatu@homelinux.local:/​safehouse 
 +</​code>​ 
 + 
 + 
 +{{anchor:​automate}} 
 +=====Automate===== 
 + 
 +Surprisingly,​ the hardest part about ''​rdiff-backup''​ or any backup plan is just determining //how// it should happenonce you have a hard drive dedicated to backing up your data, and have devised a way for that hard drive to be available, the automation part of the plan is simple. 
 + 
 +====Cron==== 
 + 
 +Processes on Linux can be scheduled with ''​cron''​. Each user has a cron table or ''​crontab'',​ which can be edited to run tasks at certain times. 
 + 
 +<​code>​ 
 +$ EDITOR=emacs crontab -e 
 +</​code>​ 
 + 
 +or if you prefer kate: 
 + 
 +<​code>​ 
 +$ EDITOR=kate crontab -e 
 +</​code>​ 
 + 
 +Your ''​crontab''​ will most likely be empty. Add a line like this (do not copy and paste this, modify it for your setup): 
 + 
 +<​code>​ 
 +*/6 * * * rdiff-backup --exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ klaatu@example.local::/​safehouse/​klaatu/​  
 +</​code>​ 
 + 
 +This sets up an automatic backup every 6th hour (on the hour) of every day of every month, regardless of the day of the week. 
 + 
 +The syntax for scheduling events in ''​cron''​ is: 
 + 
 +  * Minute (0-59)This is almost never a ''​*''​ (which would mean to perform an action //every minute//It is combined with the **hour** field, such that the 0th minute of the 4th hour (4:00), some action would occur. 
 +  * Hour (0-23)24 hour clock4 is 4am, 16 is 4pm. Use the ''/''​ (slash) to do things at increments. ''​*/​4''​ means any hour that is cleanly divisible by 4 (so, every 4 hours). ''​*/​3''​ is every three hours, and so on. ''​*''​ means //every// hour at whatever minute is set in the **minute** field. 
 +  * Day of Month (1-31)''​*''​ means every day. ''​*/​2''​ would mean every other day, ''​*/​3''​ every three days, and so on. 
 +  * Month (1-12): Same principle as the day field. 
 +  * Day of Week (0-6)0 is Sunday and 6 is Saturday.  
 + 
 +For example, to run a backup once a day at 23:00: 
 + 
 +<​code>​ 
 +0 23 * * * rdiff-backup blah blah 
 +</​code>​ 
 + 
 +To run backups every other hour only during 9:00 to 17:00 (business hours): 
 + 
 +<​code>​ 
 +0 9,​11,​13,​15,​17 * * * rdiff-backup blah blah 
 +</​code>​ 
 + 
 +The same, excluding weekends: 
 + 
 +<​code>​ 
 +0 9,​11,​13,​15,​17 * * 1,2,3,4,5 rdiff-backup blah blah 
 +</​code>​ 
 + 
 +And so on. If you are unclear, there are several good examples online, probably for the exact schedule you are planning. 
 + 
 +<WRAP alert> 
 +''​Cron''​ only works if your computer is //on//. 
 +</​WRAP>​ 
 + 
 +====SSH Keys==== 
 + 
 +If you are backing up to a local drive (that is, a drive attached physically to your computer), then you can skip this section. 
 + 
 +If you are backing up to a server over SSH, then you presumably need to sign into that server. You are not prompted by ''​rdiff-backup''​ for a password, and even if you were, you wouldn'​t see it because you're automating the backup process. 
 + 
 +Therefore, you must set up ssh key sign-in for your server. 
 + 
 +Slackware makes this very easy. If you do not already have an ssh key on your Slackermedia machine, create one (it's free): 
 + 
 +<​code>​ 
 +$ ssh-keygen 
 +</​code>​ 
 + 
 +When prompted to create a passphrase, **leave the passphrase empty**. If you give you ssh key a passphrase, then you are defeating the purpose of using an ssh key for this use case. 
 + 
 +If you already have an ssh key on your source machine, then you can use that. If, however, that key requires a password, then it's useless for automated backups. Create a new key, exclusively for the backup server. For example: 
 + 
 +<​code>​ 
 +$ ssh-keygen 
 +Generating public/​private rsa key pair. 
 +Enter file in which to save the key (/​home/​klaatu/​.ssh/​id_rsa):​ /​home/​klaatu/​.ssh/​safehouse_rsa 
 +Enter passphrase (empty for no passphrase):​ LEAVE THIS EMPTY 
 +Enter same passphrase again: LEAVE THIS EMPTY 
 +Your identification has been saved in /​home/​klaatu/​.ssh/​safehouse_rsa. 
 +</​code>​ 
 + 
 +Now configure ''​ssh''​ to use your new ''​safehouse_rsa''​ key whenever you attempt to login to your backup host. 
 + 
 +Open ''​~/​.ssh/​config''​ in your favourite text editor. If that file does not exist (it probably won't) then create it. Here are two examples of valid configurations,​ one for a server with a fully-qualified domain name, and one for an internal home server: 
 + 
 +<​code>​ 
 +host slackermedia.info 
 +        hostname slackermedia.info 
 +        identityfile /​home/​klaatu/​.ssh/​safehouse_rsa 
 +        protocol 2 
 + 
 +host 10.1.1.11 
 +     ​hostname 10.1.1.11 
 +     ​identityfile /​home/​klaatu/​.ssh/​safehouse_rsa 
 +     ​protocol 2 
 +</​code>​ 
 + 
 +Once your ssh key has been generated, copy it to your server: 
 + 
 +<​code>​ 
 +$ ssh-copy-id -i /​home/​klaatu/​.ssh/​safehouse_rsa klaatu@example.com 
 +</​code>​ 
 + 
 +Test it out by ssh'​ing into the server: You should not be prompted for a password. 
 + 
 +<​code>​ 
 +$ ssh klaatu@10.1.1.11"​ 
 +</​code>​ 
 + 
 +Assuming success, you now have password-less entry into your backup server (and so does rdiff-backup,​ since it will be running under your username). 
 + 
 + 
 +{{anchor:​recovery}
 +=====Restoring (Recovering) Files===== 
 + 
 +Once you have a backup, you obviously are able to restore files that you have either lost or changed beyond repair. 
 + 
 +<WRAP important>​ 
 +The command to get a file back off of a drive differs depending on how you are accessing the drive. For clarity, examples of both local and remote access are given. 
 +</​WRAP>​ 
 + 
 +Restore the most-recently backed-up version of a file: 
 + 
 +<​code>​ 
 +$ rdiff-backup --restore-as-of now /​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw.kra 
 +$ rdiff-backup --restore-as-of now klaatu@example.local::/​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw.kra 
 +</​code>​ 
 + 
 +The ''​--restore-as-of''​ option (''​-r''​) can go back is time farther than just the most recent back-up. For instance, maybe you think the best good version of a file existed two days ago: 
 + 
 +<​code>​ 
 +$ rdiff-backup --restore-as-of 2D /​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw.kra 
 +$ rdiff-backup --restore-as-of 2D klaatu@example.local::/​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw.kra 
 +</​code>​ 
 + 
 +Or maybe you aren't sure, and would rather view a few different versions. It's all backed up, so you can resurrect as many versions as you like: 
 + 
 +<​code>​ 
 +$ rdiff-backup --restore-as-of 2D /​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw-2days.kra 
 +$ rdiff-backup --restore-as-of 5D /​safehouse/​klaatu/​paint/​jlw.kra ~/​paint/​jlw-5days.kra 
 +</​code>​ 
 + 
 +Other acceptable time formats include 33m33s (33 minutes and 33 seconds ago) and full dates like 2015-01-01 for that New Years photo that you wanted to forget but are now having second thoughts about. For more options, see the ''​TIME FORMATS''​ section of the ''​rdiff-backup''​ man page. 
 + 
 +If you know even more detail about your file, you can use ''​rdiff-backup''​ to restore directly from an incremental backup file. Increment files are stored in your backup destination (''/​safehouse'',​ what ever drive or server that may exist on) in ''​rdiff-backup-data/​increments''​. 
 + 
 +They hold the previous versions of changed files. If you specify one directly: 
 + 
 +<​code>​ 
 +rdiff-backup /​safehouse/​klaatu/​rdiff-backup-data/​increments/​novel.2014-11-24T02:​21:​41-07:​00.diff.gz ~ 
 +</​code>​ 
 + 
 + 
 +=====Dummy Checks===== 
 + 
 +It's not a bad idea to check in on your backups every now and again, just to verify that what you think is happening is really happening.  
 + 
 +To check your backups, log into your backup server or attach your backup drive, and have a look at the latest ten (or so) backup files created: 
 + 
 +<​code>​ 
 +$ find /​path/​to/​backup/​dir -type f -printf "%C+ %p\n" | sort -rn | head -n 10 
 +</​code>​ 
 + 
 +Adjust your backup scheme as necessary, depending on what you discover. 
 + 
 + 
 + 
 +=====Documentation===== 
 + 
 +There are several good article on ''​rdiff-backup''​. [[http://​www.kirya.net/​articles/​backups-using-rdiff-backup]] is quite good. 
 + 
 +<WRAP centeralign>​ 
 +<wrap fa>[[sox|R]]</​wrap>​ <wrap fa>​[[start|S]]</​wrap>​ <wrap fa>​[[rescue|Q]]</​wrap>​ 
 +</​WRAP>​