Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
backup [2015/07/16 01:50]
slackermedia
backup [2021/06/03 19:48] (current)
Line 1: Line 1:
-[[{arrowp.png|border:​0;​background:​none;​width:​0;​display:​inline-block;​position:​absolute;​top:​0;​left:​0;​}sox|]] 
  
-[[{arrown.png|border:​0;​background:​none;​width:​0;​display:​inline-block;​position:​absolute;​top:​0;​margin-left:​2.5em;​}rescue|]] 
  
 /* if they are here, we dont need to convince them to backup /* if they are here, we dont need to convince them to backup
Line 95: Line 93:
  
 <​code>​ <​code>​
-LABEL=safehouse ​    /​safehouse ​   udf   auto,user 0 0+LABEL=safehouse ​    /​safehouse ​   udf   user,​rw  ​0 0
 </​code>​ </​code>​
  
Line 101: Line 99:
 </​WRAP>​ </​WRAP>​
  
-**3.** ​Create a directory for your backups on the drive.+**3.** ​Perform an initial backup.
  
 <WRAP indent> <WRAP indent>
-In addition to a backup ​directory, create an empty file that you can use as a test to make sure that the drive is present (this will be used laterduring the [[#​automate|automation]] phase).+The first backup ​is the largest and slowest backup because it is copying everything over to your backup drive. Subsequent backups ​will be much much smallerbecause it will only copy changes to that base layer.
  
-<​code>​ +It is vital that this initial backup is done the exact same way it's going to be done on all future backups, for many reasons, but essentially because you need to get confirmation that everything has been set up correctly ​//before// you try to automate it. If you do a backup with a drive connected to your computer, and then try to switch to a networked backup, you might do user permissions wrong, or file paths might be inconsistent,​ or any number of problems, and you won't ever realise it because you'll have automated the process, never knowing that all you have automated is failure.
-$ mkdir /safehouse/$USER +
-$ touch /safehouse/go +
-</​code>​+
  
-At this point, you have created:+Do not do that.
  
-  * A standard location for your backup ​destination:​ ''/​safehouse''​) +Make your first backup ​the initial ​test of your backup process. If it fails, refine your process, troubleshoot,​ and try again. When you get it right, you'll know it, and you'll know exactly what to automate.
-  * A test file to help detect if the drive is accessible: ''/​safehouse/​go''​ +
-  * A directory to hold your backups: ​''​/​safehouse/​$USER''​ (where $USER has been converted by the computer ​to your username)+
  
-</​WRAP>​+The basic command to backup your home directory (where all of your personal data lives):
  
-**4.** Perform an initial backup. +For a locally attached ​drive:
- +
-The first backup is the largest and slowest backup because it is copying everything over to your backup ​drive. Subsequent backups will be much much smaller, because it will only copy changes to that base layer. +
- +
-The basic command to backup your home directory (where all of your personal data lives):+
  
 <​code>​ <​code>​
-$ rdiff-backup -v 9 $HOME /safehouse/$USER+$ rdiff-backup --terminal-verbosity 8 --print-statistics \ 
 +--exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ \ 
 +/safehouse/klaatuWork/
 </​code>​ </​code>​
  
Line 133: Line 124:
 </​WRAP>​ </​WRAP>​
  
-This command backs up your entire home directory to the ''/​safehouse''​ location. ​It will take a long time, so just let it work.+Or for a server: 
 + 
 +<​code>​ 
 +$ rdiff-backup --terminal-verbosity 8 --print-statistics \ 
 +--exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ \ 
 +klaatu@10.1.1.11::/​safehouse/​klaatuWork/​ 
 +</​code>​ 
 + 
 +<WRAP alert> 
 +Notice that the server path contains //two// ''::''​ between the hostname and the destination path. 
 +</​WRAP>​ 
 + 
 +This command backs up your entire home directory to the ''/​safehouse''​ location. ​Do not bother creating the destination file (in this example'​s case, the ''/​safehouse/​klaatuWork''​ folder); ''​rdiff-backup''​ will [attempt] to create that for you and warn you if it cannot write to the drive. 
 + 
 +No matter what, your first backup ​will take a long time, so just let it work.
 </​WRAP>​ </​WRAP>​
  
Line 159: Line 164:
 </​code>​ </​code>​
  
-If nothing is listed, then the drive has not mounted successfully. Verify that you have labelled it properly with the ''​vid''​ flag (if using UDF as recommended;​ if you are using ext{2-4}, try using ''​e2label''​, and also question yourself about why you are not heeding Slackermedia'​s advice about UDF).+If nothing is listed, then the drive has not mounted successfully. Verify that you have labelled it properly with the ''​vid''​ flag (if using UDF as recommended;​ if you are using ext{2-4}, try using ''​e2label''​).
  
  
-**2.** ​Verify that you have access to the drive:+**2.** ​Change a file or add a new test file, and then run the your backup command again.
  
 <​code>​ <​code>​
-touch /safehouse/go || echo "​Nope"​+rdiff-backup -v 8 /​home/​klaatu/ ​/safehouse/klaatuWork/
 </​code>​ </​code>​
  
-Silence indicates success. If you get "​Nope"​ in response, then your ''/​etc/​fstab''​ entry is incorrect, or you are not using UDF as recommended. +This time, the process is relatively ​fast, since it has only copied over the changes.
- +
- +
-**3.** Now that everything is set, try changing a file or adding a new test file, and then run the your backup command. +
- +
-<​code>​ +
-$ rdiff-backup -v 9 $HOME /​safehouse/​$USER +
-</​code>​ +
- +
-This time, the process is quite fast, since it has only copied over the changes.+
  
 ☛ You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options. ☛ You might have exclusions that you don't want to have backed up (for instance, do you ever want your trash bin backed up?). For this, read the section on [[#​excludes|exclude]] options.
Line 208: Line 204:
 The danger of using this easy method is that if you have your computer set to backup every 4 hours, for example, and you attach your drive but only leave it attached for 3.75 hours, then you miss a backup cycle. Keep doing that, and you may end up with far fewer backups than expected. It practise, it's better than nothing, and as long as you remember to leave your laptop on and awake overnight with your backup drive attached, you are sure to hit one or two backup cycles (depending on how often you sleep). The danger of using this easy method is that if you have your computer set to backup every 4 hours, for example, and you attach your drive but only leave it attached for 3.75 hours, then you miss a backup cycle. Keep doing that, and you may end up with far fewer backups than expected. It practise, it's better than nothing, and as long as you remember to leave your laptop on and awake overnight with your backup drive attached, you are sure to hit one or two backup cycles (depending on how often you sleep).
  
 +{{anchor:​udev}}
 ====udev (the "I spent my weekend with udev" solution)==== ====udev (the "I spent my weekend with udev" solution)====
  
Line 221: Line 218:
  
 <​code>​ <​code>​
-#!/usr/bin/bash+#!/bin/sh
  
-echo $date > /​tmp/​udev.log+date > /​tmp/​udev.log
 </​code>​ </​code>​
  
Line 248: Line 245:
 With ''​udevadm''​ monitor running, plug in your backup drive and watch as all kinds of information is spewed out onto your screen. Notice, particularly,​ that the type of event is an ''​ADD''​ event. That's a good way of identifying what type of event we want. With ''​udevadm''​ monitor running, plug in your backup drive and watch as all kinds of information is spewed out onto your screen. Notice, particularly,​ that the type of event is an ''​ADD''​ event. That's a good way of identifying what type of event we want.
  
-The ''​udevadm''​ monitor provides a lot of good info, but we can see it with prettier formatting with the command ''​udevadm info'',​ assuming you know where your drive is currently located in your ''​dev''​ tree. If not, unplug and then plug your thumbdrive back in and then immediately issue this command:+The ''​udevadm''​ monitor provides a lot of good info, but we can see it with prettier formatting with the command ''​udevadm info -a -n'',​ assuming you know where your drive is currently located in your ''​dev''​ tree. If not, unplug and then plug your thumbdrive back in and then immediately issue this command:
  
 <​code>​ <​code>​
Line 262: Line 259:
 This will return a wealth of information. Focus on the first block of info, first. This will return a wealth of information. Focus on the first block of info, first.
  
-The feedback from ''​udevadm info''​ starts with the device specified by the ''​devpath'',​ and then walks up the chain of parent devices. It prints for every device found, all possible attributes in the ''​udev''​ rules key format. ​+The feedback from ''​udevadm info -a -n''​ starts with the device specified by the ''​devpath'',​ and then walks up the chain of parent devices. It prints for every device found, all possible attributes in the ''​udev''​ rules key format. ​
  
 A ''​udev''​ rule must contain one attribute from one single "​parent"​ device; parent attributes are things that describe a device from the most basic level, such as "​it'​s something that has been plugged into a physical port" or "it is something with a size of XX" or "this is a removable device"​. A ''​udev''​ rule must contain one attribute from one single "​parent"​ device; parent attributes are things that describe a device from the most basic level, such as "​it'​s something that has been plugged into a physical port" or "it is something with a size of XX" or "this is a removable device"​.
  
-That info, just under the ''​udevadm info''​ introductory paragraph, looks a little something like this:+That info, just under the ''​udevadm info -a -n''​ introductory paragraph, looks a little something like this:
  
 <​code>​ <​code>​
Line 309: Line 306:
 If you see the date and time returned from ''/​tmp/​udev.log'',​ then the script has worked. If you see the date and time returned from ''/​tmp/​udev.log'',​ then the script has worked.
 </​WRAP>​ </​WRAP>​
 +
  
 **Refining the Rule into Something Useful** **Refining the Rule into Something Useful**
Line 352: Line 350:
 Once you can comfortably confirm that your script is triggered by ''​udev''​ when you attach your backup drive, turn your attention to the functionality of the script. Right now it is useless, doing nothing more than logging the fact that it has been executed. Once you can comfortably confirm that your script is triggered by ''​udev''​ when you attach your backup drive, turn your attention to the functionality of the script. Right now it is useless, doing nothing more than logging the fact that it has been executed.
  
-Here is an example backup-on-attach script. You can use it exactly as is EXCEPT for the last command; you need to change the path of the directory you want backed up (unless your username is also ''​klaatu''​) and the destination:​+Here is an example backup-on-attach script. You can use it exactly as is EXCEPT for the last command; you need to change the path of the directory you want backed up (unless your username is also ''​klaatu''​) and the destination ​(unless you also named your destination folder ''​klaatu''​):
  
 <​code>​ <​code>​
Line 364: Line 362:
     sleep 5     sleep 5
     if [ -b /safehouse ]; then     if [ -b /safehouse ]; then
-        if [ -w /safehouse/go ]; then+        if [ -w /safehouse/klaatu ​]; then
             # everything is ok             # everything is ok
             COUNT=9             COUNT=9
Line 430: Line 428:
  
 <​code>​ <​code>​
-$ rdiff-backup -v $HOME username@your-server.info::/​safehouse/​username+$ rdiff-backup -v $HOME username@your-server.info::/​safehouse/​username
 </​code>​ </​code>​
  
Line 462: Line 460:
  
 Consider purchasing a large-capacity thumbdrive and using it as either your master work directory or as your [[#​bkpconnect|plug-and-backup]] destination. That way, every time you plug your drive into your computer, it gets backed up; you no longer have to remember to do it manually! Consider purchasing a large-capacity thumbdrive and using it as either your master work directory or as your [[#​bkpconnect|plug-and-backup]] destination. That way, every time you plug your drive into your computer, it gets backed up; you no longer have to remember to do it manually!
 +
 +If you are using rdiff-backup,​ you can backup a single file between backup script occurrences:​
 +
 +<​code>​
 +$ rdiff-backup --include $HOME/​foo.txt --exclude '​**'​ $HOME /safehouse
 +</​code>​
  
 Another option is to wield the power of the Internet, and use a good git service. There are several simple clients for git, including the very appealing [[http://​sparkleshare.org|SparkeShare]] or, if you have access to a server and the time to configure it, [[http://​owncloud.org|ownCloud]]. These both have Dropbox-style sync folders. Another option is to wield the power of the Internet, and use a good git service. There are several simple clients for git, including the very appealing [[http://​sparkleshare.org|SparkeShare]] or, if you have access to a server and the time to configure it, [[http://​owncloud.org|ownCloud]]. These both have Dropbox-style sync folders.
Line 518: Line 522:
  
 <​code>​ <​code>​
-0 */6 * * * rdiff-backup --exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ klaatu@example.local:/​safehouse/​klaatu/ ​+0 */6 * * * rdiff-backup --exclude-globbing-filelist /​home/​klaatu/​.excludes /​home/​klaatu/​ klaatu@example.local::/​safehouse/​klaatu/ ​
 </​code>​ </​code>​
  
Line 601: Line 605:
  
 <​code>​ <​code>​
-$ ssh-copy-id klaatu@example.com+$ ssh-copy-id ​-i /​home/​klaatu/​.ssh/​safehouse_rsa ​klaatu@example.com
 </​code>​ </​code>​
  
Line 652: Line 656:
 rdiff-backup /​safehouse/​klaatu/​rdiff-backup-data/​increments/​novel.2014-11-24T02:​21:​41-07:​00.diff.gz ~ rdiff-backup /​safehouse/​klaatu/​rdiff-backup-data/​increments/​novel.2014-11-24T02:​21:​41-07:​00.diff.gz ~
 </​code>​ </​code>​
 +
 +
 +=====Dummy Checks=====
 +
 +It's not a bad idea to check in on your backups every now and again, just to verify that what you think is happening is really happening. ​
 +
 +To check your backups, log into your backup server or attach your backup drive, and have a look at the latest ten (or so) backup files created:
 +
 +<​code>​
 +$ find /​path/​to/​backup/​dir -type f -printf "%C+ %p\n" | sort -rn | head -n 10
 +</​code>​
 +
 +Adjust your backup scheme as necessary, depending on what you discover.
  
  
Line 659: Line 676:
 There are several good article on ''​rdiff-backup''​. [[http://​www.kirya.net/​articles/​backups-using-rdiff-backup]] is quite good. There are several good article on ''​rdiff-backup''​. [[http://​www.kirya.net/​articles/​backups-using-rdiff-backup]] is quite good.
  
-[[{arrown.png|border:​0;​background:​none;​width:​0;​display:​inline-block;​float:​right;​}rescue|]][[{arrowp.png|border:​0;​background:​none;​width:​0;​float:​right;​}sox|]]+<WRAP centeralign>​ 
 +<wrap fa>[[sox|R]]</​wrap>​ <wrap fa>[[start|S]]</​wrap>​ <wrap fa>​[[rescue|Q]]</​wrap>​ 
 +</​WRAP>​