System Rescue

One of the many reasons for using Linux is to protect yourself against loss of data and system data. Obviously nothing can insure you 100% against either of these, but on Linux there is the advantage of having control over the system to the point that if it is possible to recover a downed system, you are able to recover at least your data.

This wouldn't be a chapter about system recovery without the obligatory admonishment that the best prevention against data disaster is backing-up your data in the first place. It may be obligatory, but that doesn't mean it's any less true. If you want to guarantee that your data is safe, back it up. The sooner you establish a reliable backup regimen, the better.

With that said, these are the different steps you can take to recover a system, in the order of the least serious to the most serious.

Forgot your password

If you have forgotten your password, you should be able to re-set it with the root password.

If you have forgotten the root password, re-set the root password first.

To re-set your user password, get to a terminal. This might mean opening Konsole from the K Menu or switching to a text-only console with alt ctrl F2.

Then become root and change the password:

$ su 
# passwd klaatu

If you had to flip to a text-only console, use alt ctrl F7 to return to your graphical desktop.

Forgot the root password

Here is how to reset a forgotten root password:

  • Boot your computer from the install media.
  • When the initial Slackware-install boot screen appears, hit Return as you normally would.
  • Accept the keymap (or change it if you do not use the default North American keyboard)
  • Login, as usual, as root with no password.

At this point, you're sitting in the normal shell for the Slackware installer. Here is where the terminology can get weird; we are here to reset the root password, but remember that the beginning point of a UNIX filesystem is also called root and is a completely different concept. So remember that there is the root location and root user, and we're talking about both of those things here. I will be very explicit as to which “root” concept I am talking about from this point on.

Right now, your computer thinks the root [the location] point of your harddrive is the DVD drive. Think about it; you booted from the DVD so it makes sense. What you need to do is re-assign what the computer sees as its root [the location] partition so that any commands you give it applies to a new root [the location]; ie, the actual physical harddrive upon which you have installed Slackware-with-a-forgotten-password. That's precisely what chroot was written to do.

Mount the physical harddrive to some place. I just create a new directory and mount it there:

# mkdir /rescue
# mount /dev/sda1 /rescue 

Now change directory into the mounted harddrive and make it the new root [location] for the system.

# cd /rescue
# chroot .

Now, as far as your computer knows, you are running your normal everyday Slackware system just as if you'd booted off of it in the first place. Yes, the computer has no idea you booted off of a DVD. This means that if you run the passwd command to change your password, it will be changing the password for your harddrive's root [the person] user.

Do that now:

# passwd

Enter a new passphrase for your root identity, and then reboot.

When you reboot, be sure to eject the DVD so that your computer doesn't boot off of it again.

What to do if your OS dies

If, for whatever reason, the operating system on your computer stops working, and you feel that it is beyond further troubleshooting, then presumably it's time to re-install. This should not happen; “normally”, this does not happen. The typical lifespan of Slackware is that it runs until a major update is released, and then you upgrade, and continue running until the next one, and so on.

But computers being computers, anything is possible.

If you find yourself at the point that you have no choice but to re-install (or you just want a fresh start for any reason), then you should back up these directories:

  • /home - this is where all of the user-generated data lives. If you have done anything creative on your computer that you want to keep, it is in /home somewhere.
  • /usr/local/share - you may or may not have placed assets in a shared location such as /usr/local/share. In theory, anything you placed there should also exist elsewhere (ie, a master backup or resource drive) but if not, grab it now.
  • /etc - if you, as root, customised any settings that you do not believe are trivial to re-create, grab the /etc directory.

Everything outside of these directories are, theoretically, replaceable in one way or another (usually by re-installing from the Internet or an install disc, depending on what it is).

In any case, rsync is generally the safest and easiest tool to use when syncing directories. For example, to grab a copy of the /home directory:

$ su
# rsync -av /home/ /path/to/backup/location/

To sync it the other way after the re-install:

$ su
# rsync -av /path/to/backup/location/ /home/

The important thing to realise is that the end slashes (/) should be the same, every time. If you omit a trailing slash at the end of one path, directories get nested inside of one another, and the sync is inaccutare. You won't lose data, but your data will become messy and the results are not what you expect.

The appropriate workflow for a re-install:

  • Backup the important directories with rsync.
  • Do a fresh install of Slackware.
  • Re-create your user(s).
  • Sync the backup user directories to the fresh, empty user directories. Do this as root if you are dealing with more than just your own data.
  • Also as root, sync any other backup directories.
  • Re-build Slackermedia.
  • Take a coffee break.

Dying hard drive

First: you should be backing up your data on a regular basis.

If your hard drive shows signs of dying (files are becoming corrupt, bootup stalls frequently for emergency disk checks, the drive sounds louder or noisier than usual), then you must stop what you are doing. Stop using the drive; shutdown the computer. The more you use the drive, the more corruption and damage is likely to occur.

The sooner you stop using a dying hard drive, the better your chances of rescuing your data.

  • Do not attempt to rescue your data yet, since that in itself would require you to use the hard drive. Before you do anything else, go and purchase a new drive.
    1. Install Slackware onto your new drive.
    2. Create your users.

    Ideally, you have been backing up all of your data, so the old drive can die at any moment with minimal or zero data loss to you. Verify that your backups have been running by logging into your backup server, or just connecting your backup drive, and find the latest backup:

$ find /path/to/backup/dir -type f -printf "%C+ %p\n" | sort -rn | head -n 10

Assuming a recent backup exists, restore from your backups, and optionally grab whatever additional data you want to save off of your dying drive, and move on to re-building Slackermedia.

If you have not been backing up, then it's time to calculate risks. To be clear: you should not be at this point. You must establish an automated backup system in the future.

Your choices are:

  • If you feel confident that the drive has some life in it, then you can try to rescue the data yourself. If you were forced to stop working because your drive stopped doing what it normally does, then you should not attempt this.
  • Take your old, dying drive to a professional data rescue company. This will cost a thousand or more dollars.

To get data from your dying drive, just plug it in and mount it like any other drive. If it will not mount, then assume it is beyond recovery, and take it to a professional.

Once you have all the data you want to save, replace your hard drive.