Hard Drives and Linux

Hard drives can broadly be classified as either internal or external. Internal drives tend to be easier to manage, because they are not portable; they are integrated in some way with your operating system, even if it is just extra storage space that you use on an as-needed basis.

External hard drives are more complex, but only because they tend to move around. You may not move it often, or you might literally have it on your keychain. The bottom line is that external hard drives tend to visit not just one computer, and not even just one operating system, and as a multimedia artist, you are likely to have to deal with that frequently.

The good news is that drives coming to your studio can, more often than not, be read by Slackware. There are only a few caveats:

Macs mostly use HFS+ formatted drives. HFS+ is a relatively volatile file system; it is rather prone to failure and corruption, but luckily tools to repair them tend to be pretty good.

If you have a drive that claims to be “Mac compatible” and you want to use it on both Linux and Mac OS, then it is probably formatted as HFS+. HFS+, in addition to being one of the least stable file systems on the market, is crafted intentionally to be incompatible with other systems; in order to write to HFS+ from Linux, you must disable journaling on the drive.

You can, however, read from it without doing anything to the drive.

Disabling the journal on an HFS+ drive must be done from within Mac OS. If you do not have access to Mac OS, then you cannot write to the HFS+ drive, do not attempt to write to the drive; you could do damage to the files.

If you are stuck with a “Mac compatible” drive and want to use it as an active “normal” drive in your studio, with full read and write capabilities, then your best bet is to copy all of the data off of the drive, re-format it, and then copy the data back onto it.

If you require the drive to remain compatible with a Mac as well as Linux (and, as a side benefit, Windows), then use the UDF format.

If these are not valid options for you, then use Mac OS to disable the journal on the drive. Mac OS may later re-activate the journal without notice, so you may have to do this often.

If you have a drive that claims to be “Windows compatible” and you want to use it on both Linux and Windows, then it is probably formatted as NTFS or ExFAT, with a small chance of it being FAT32.

None of these filesystems are particularly good but they are all well-supported by Linux (not, however, by Mac OS).

If you are stuck with a “Windows compatible” drive and want to use it as an active “normal” drive in your studio, you probably can, as is. There are several inconveniences that you may notice (file size limitations, permission issues), but as external drives go, everything should basically work as expected.

A better option is to copy all of the data off the drive, re-format it as a UDF file system, and put all of the data back on. UDF brings along with it several benefits, including Window and Mac compatibility, and elimination of the quirks of Window hard drive formats.

If you are using a drive just on Linux systems (recommended, but not always possible), then you can keep your drives in a native Linux format. The only practical advantage of using a native Linux format on an external drive is that they happen to be very robust file systems. They are comparatively difficult to corrupt or break, they are fast, well-designed and maintained, case-sensitive, and have very few limitations in a pragmatic sense.

The immediate disadvantage of using native formats for external drives on Linux is that none of them were really designed for external use. That is, most of them assume that the drive is inside the computer. This tends to not matter much until you start swapping drives with other Linux users or computers, at which point file permissions can become a problem.

Permissions can be managed for external drives, but you have to be intentional and mindful of it.

A better option is to copy all of the data off the drive, re-format it as a UDF file system, and put all of the data back on. UDF brings along with it several benefits, including Window and Mac compatibility, and elimination of the quirks of Window hard drive formats.

Partly as an answer to the filesystem problem, a few Standards groups came up with UDF, the Universal Disk Format. It was mostly intended as the replacement for ISO-9660, and did become the official filesystem for CD-RW, DVD-RW, and Blu-Ray.

The down side is that it does not use journaling, making data recovery after a crash or accidental unplugging a little riskier. It does not use partitions, but that is not usually an issue for an external drive.

The good news is that it is open source, can use UTF-8 filenames that are as long as 255 bytes, file sizes and filesystem sizes of 2TB. Like FAT, UDF does not bother with permissions, making it ideal for external drives on Unix. At the very worst, even considering some of the features left out of UDF, it is a better and more flexible option than FAT, and has no patent issues to contend with (Micrososft sometimes sues companies for using random features in FAT).

Since it was primarily intended for optical media, creating a UDF volume is different from formatting a drive for any other filesystem.

The drive being formatted must have no partitions on it. This is entirely unlike any other filesystem, but it is necessary for some operating systems to accurately detect the UDF filesystem.

To get rid of the existing partition on a drive, zero out the first 4096 bytes of the drive.

dd if=/dev/zero of=/dev/sdx bs=512 count=4096

Note that the bytesize (bs) is not flexible. It must be 512.

Next, find out the block count for your drives:

$ df -i /dev/sdx
Inodes   IUsed    IFree
2040230   619    2039611

Finally, create the filesystem so that it spans the entire drive.

mkudffs --blocksize=512 \
--udfrev=0x0201 \
--lvid="myUdfDrive" \
--vid="myUdfDrive" \
--media-type=hd --utf8 \
/dev/sdx || echo "fail"

Now you can mount and use the drive on any platform.

[EOF]