Differences

This shows you the differences between two versions of the page.

--- drives [2015/07/14 21:09]
slackermedia
+++ drives [2021/06/03 19:48] (current)
@@ Line 1: / Line 1: @@
-[[{arrowp.png|border:0;background:none;width:0;display:inline-block;position:absolute;top:0;left:0;}dotdesktop|]]
+======Hard Drives and Linux======
-[[{arrown.png|border:0;background:none;width:0;display:inline-block;position:absolute;top:0;margin-left:2.5em;}mimetypes|]]
+Linux does hard drives astonishingly well. So well, in fact, that most hard drive rescue services default to Linux because its toolset is so rich.
-======Hard Drives and Linux======
+<WRAP info>
+**TL;DR** \\
+If you do not want to go into hard drive forensics for a living, you may be most interested in:
+\\
+  * [[#kvpm|kvpm]], a graphical hard drive manager
+  * [[#udf|udf]], a truly universal format that you should use for external drives under 2TB in size
+</WRAP>
 Hard drives can broadly be classified as either internal or external. Internal drives tend to be easier to manage, because they are not portable; they are integrated in some way with your operating system, even if it is just extra storage space that you use on an as-needed basis.
-Here is a quick profile of your internal drive:
+Here is a profile of your internal drive:
 =====Internal Drives=====
@@ Line 60: / Line 66: @@
 (The possible exception here is that you are getting a large drive that you want to use as shared storage space between a drive running Linux and a drive running some other OS. This is not recommended, but if you do this, then treat the drive as an [[#udf|external drive]]. Slackermedia does not support this, because the other OS adds a significant variable to how your data is being managed, so you are, respectfully, on your own!)
+{{anchor:intformat}}
 **To format an internal drive for use with Linux:**
 <WRAP indent>
-Determine the device node of the drive you are going to format by first seeing what drives are already part of your system:
+Determine the device node of the drive you are going to format by first seeing what drives are already part of your system.
+Use the ''lsblk'' command to view all block devices (hard drives) attached to your computer. If the ''lsblk'' command is not clear to you, or you want to double-check what it tells you, you can investigate further:
 <code>
@@ Line 88: / Line 97: @@
 In this example, there is a third drive not in use by the system, labelled ''sdc''. This is the new drive that needs formatting. Notice that it does have a partition on it already, but that's only because most all drives purchased from a modern computer store are pre-formatted, presumably so that users do not have to learn about formatting themselves.
+To prevent copy-paste errors, this article uses the ''/dev/sdX'' designation.
 Keep in mind that your drive in real life could be anything from ''sdb'' to ''sdz'', depending on how many actual drives you have plugged in. Usually, the first drive you plug in is going to come up as ''sdb'' because ''sda'' is the drive running your computer, but be aware of your actual setup and use your head. //You do not want to format the wrong drive.//
@@ Line 94: / Line 105: @@
 <code>
-$ su -c 'mount /dev/sdc1 /mnt/hd'
+$ su -c 'mount /dev/sdX1 /mnt/hd'
 $ cd /mnt/hd
 $ ls
@@ Line 114: / Line 125: @@
 <code>
 $ cd ~
-$ su -c 'umount /dev/sdc*'
+$ su -c 'umount /dev/sdX*'
 </code>
@@ Line 121: / Line 132: @@
 <code>
 $ su
-# parted /dev/sdc mklabel gpt
+# parted /dev/sdX mklabel gpt
 </code>
@@ Line 131: / Line 142: @@
 <code>
-# parted /dev/sdc print | grep Disk
+# parted /dev/sdX print | grep Disk
 </code>
@@ Line 139: / Line 150: @@
 <code>
-# parted /dev/sdc mkpart primary 1 2834020
+# parted /dev/sdX mkpart primary 1 2834020
 </code>
@@ Line 148: / Line 159: @@
 </WRAP>
-Now the drive has a partition; all it needs now is a file system. Remember, a partition is indicated by a number trailing the device node. In this example, the location of your new partition is ''/dev/sdc1''.
+Now the drive has a partition; all it needs now is a file system. Remember, a partition is indicated by a number trailing the device node. In this example, the location of your new partition is ''/dev/sdX1''.
 For a Linux native drive, use ''ext4'':
 <code>
-# mkfs.ext4 -L penguindrive /dev/sdc1
+# mkfs.ext4 -L penguindrive /dev/sdX1
 </code>
@@ Line 159: / Line 170: @@
 <code>
-# mkfs.jfs -L penguindrive /dev/sdc1
+# mkfs.jfs -L penguindrive /dev/sdX1
 </code>
@@ Line 181: / Line 192: @@
 </code>
-Now mount the drive by mounting all drives listed in ''/etc/fstab'':
+And finally, mount the drive by mounting all drives listed in ''/etc/fstab'':
 <code>
-mount /storage
+mount -a
 </code>
 </WRAP>
+=====External Drives=====
-/*
+External hard drives are more complex, but only because they tend to move around. You may not move it often, or you might literally have it on your keychain and take it everwhere with you. The bottom line is that external hard drives tend to visit not just one computer, indeed not even just one operating system, and as a multimedia artist, you are likely to have to deal with that frequently.
-Theoretically, it's comforting to know that the very format in which your data is stored is open source, because this way you can be sure that you will always have access to it. Short term, this hardly seems to matter, because no one ever (hardly anyone, anyway) has the problem of pulling out an old disk or drive and finding it suddenly unreadable because the file system is old and its specifications were never published.
-In practise, this is a false sense of security; there are plenty of artists who find that those charming projects of their youth are locked inside proprietary disk formats which simply cannot be read on most operating systems. They cannot be read until someone reverse engineers the format and releases an open source library that is able to look at a file system and understand how everything is organised, and how data can be stitched back together in a meaningful way. Most of the time, the miracle of reverse engineering happens when we are not looking, so it seems like there is no problem; closed source file systems and file formats get reverse engineered, and everything goes on as usual.
+====Incoming Drives====
-While there is some truth to taking the "time heals all wounds" approach, it is a calculated risk, and until enough time has passed for all of those file systems to be decoded, you still might find yourself without access to your old data. If you are willing to take that risk, it's your choice, but know that there are open source file systems available.
+In this context, an "incoming drives" is a drive you do not own but need to work with.
-Your internal drive, or at least the one running Linux, uses
+Good news: drives coming to you from clients or collaborators, more often than not, are plug-and-play **for reading** on Linux.
+There are only a few caveats:
+  * Macs mostly use HFS+ formatted drives. HFS+ is a relatively volatile file system; it is rather prone to failure and corruption, but luckily tools to repair them tend to be pretty good. Even so, it is safer to minimise contact with HFS+ drives; get the data you need from the drive, and disconnect it.
+  * Windows formats generally pose no issue; it is mostly all reverse-engineered by now, and so it's mostly a matter of attaching the drive and reading data from it as you would any other drive.
-External hard drives are more complex, but only because they tend to move around. You may not move it often, or you might literally have it on your keychain. The bottom line is that external hard drives tend to visit not just one computer, not even just one operating system, and as a multimedia artist, you are likely to have to deal with that frequently.
+**Writing** to a foreign drive is less simple.
-=====Incoming Drives=====
+**Mac Drives**
+<WRAP indent>
+HFS+ drives are crafted to be intentionally incompatible with other systems (the journal is not open source or even open spec); in order to write //to// HFS+ from Linux, you **must** disable journaling on the drive first.
-Good news: drives coming to your studio can, more often than not, be read by Slackware. There are only a few caveats:
+Disabling the journal on an HFS+ drive must be done from within Mac OS. If you do not have access to Mac OS, then you cannot write to the HFS+ drive, **do not** attempt to write to the drive; you could do serious damage to the drive's file system.
-Macs mostly use HFS+ formatted drives. HFS+ is a relatively volatile file system; it is rather prone to failure and corruption, but luckily tools to repair them tend to be pretty good.
+If you are stuck with a "Mac compatible" drive and want to use it as an active "normal" drive in your studio, with full read and write capabilities, then your best bet is to copy all of the data off of the drive, [[#extformat|re-format it]], and then copy the data back onto it.
+If you require the drive to remain compatible with a Mac as well as Linux (and, as a side benefit, Windows), then use the [[#udf|UDF]] format.
-=====Mac Compatible=====
+If these are not valid options for you, then use Mac OS to disable the journal on the drive. Mac OS may later re-activate the journal without notice, so you may have to do this often.
+</WRAP>
-If you have a drive that claims to be "Mac compatible" and you want to use it on both Linux and Mac OS, then it is probably formatted as **HFS+**. HFS+, in addition to being one of the least stable file systems on the market, is crafted intentionally to be incompatible with other systems; in order to write //to// HFS+ from Linux, you must disable journaling on the drive.
+**Windows Drives**
-You can, however, read //from// it without doing anything to the drive.
+<WRAP indent>
+Most Windows drive formats are well reverse-engineered at this point and can be both read from and written to without any special action taken on your part.
+</WRAP>
-Disabling the journal on an HFS+ drive must be done from within Mac OS. If you do not have access to Mac OS, then you cannot write to the HFS+ drive, **do not** attempt to write to the drive; you could do damage to the files.
+{{anchor:udf}}
+=====UDF: the Universal Universal Disk Format=====
-If you are stuck with a "Mac compatible" drive and want to use it as an active "normal" drive in your studio, with full read and write capabilities, then your best bet is to copy all of the data off of the drive, [[#format|re-format it]], and then copy the data back onto it.
+Partly as an answer to the problem of having no universally-accepted file system and partly out of the need for a good file system, a few Standards groups came up with **UDF**, the Universal Disk Format. It was mostly intended as the replacement for ISO-9660, and did become the official file system for CD-RW, DVD-RW, and Blu-Ray.
-If you require the drive to remain compatible with a Mac as well as Linux (and, as a side benefit, Windows), then use the [[#udf|UDF]] format.
+There are two significant disadvantages of UDF:
-If these are not valid options for you, then use Mac OS to disable the journal on the drive. Mac OS may later re-activate the journal without notice, so you may have to do this often.
+  * Maximum volume size is 2TB (on hard drives).
+  * It does not use journaling, making data recovery after a crash or accidental unplugging a little riskier.
-=====Windows Compatible=====
+It is open source, it can use UTF-8 filenames that are as long as 255 bytes, file sizes and file system sizes of 2TB. UDF does not bother with permissions, making it ideal for external drives.
-If you have a drive that claims to be "Windows compatible" and you want to use it on both Linux and Windows, then it is probably formatted as **NTFS** or **ExFAT**, with a small chance of it being **FAT32**.
+Since it was primarily intended for optical media, creating a UDF volume is different from formatting a drive for any other filesystem.
-None of these filesystems are particularly //good// but they are all well-supported by Linux (not, however, by Mac OS).
+Slackermedia recommends the UDF format for any external drive that you intend to use with more than just your own computer. It avoids both file permission frustration and file size limitations, but maintains all the other UNIX features that one would expect from a drive. It works well on thumbdrives as well as tradition drives. By being a universal format, it ensures that the data that matters to you the most is always available to you, regardless of what OS you happen to have on hand.
-If you are stuck with a "Windows compatible" drive and want to use it as an active "normal" drive in your studio, you probably can, as is. There are several inconveniences that you may notice (file size limitations, permission issues), but as external drives go, everything should basically work as expected.
-A better option is to copy all of the data off the drive, re-format it as a [[#udf|UDF]] file system, and put all of the data back on. UDF brings along with it several benefits, including Window and Mac compatibility, and elimination of the quirks of Window hard drive formats.
+====Formatting a Drive for UDF====
-=====Linux Compatible=====
+The drive being formatted must have no partitions on it. This is entirely unlike any other filesystem, but it is necessary for some operating systems to accurately detect the UDF filesystem.
-If you are using a drive just on Linux systems (recommended, but not always possible), then you can keep your drives in a native Linux format. The only practical advantage of using a native Linux format on an external drive is that they happen to be very robust file systems. They are comparatively difficult to corrupt or break, they are fast, well-designed and maintained, case-sensitive, and have very few limitations in a pragmatic sense.
+To get rid of the existing partition on a drive, zero out the first 4096 bytes of the drive. This effectivel **erases all of the data on the drive** so you should be doing this only on an empty drive or a drive with data on it that you do not wish to use.
-The immediate //disadvantage// of using native formats for external drives on Linux is that none of them were really designed for external use. That is, most of them assume that the drive is **inside** the computer. This tends to not matter much until you start swapping drives with other Linux users or computers, at which point file permissions can become a problem.
+<code>
+# dd if=/dev/zero of=/dev/sdx bs=512 count=4096
+</code>
-Permissions can be managed for external drives, but you have to be intentional and mindful of it.
+Note that the bytesize (bs) is not flexible. It must be 512.
-A better option is to copy all of the data off the drive, re-format it as a [[#udf|UDF]] file system, and put all of the data back on. UDF brings along with it several benefits, including Window and Mac compatibility, and elimination of the quirks of Window hard drive formats.
+Create the filesystem so that it spans the entire drive:
+<code>
+mkudffs --utf8 \
+--blocksize=512 \
+--udfrev=0x0201 \
+--lvid="penguindf" \
+--vid="penguindf" \
+--media-type=hd \
+/dev/sdx || echo "failed"
+</code>
+Now you can mount and use the drive on any platform.
+{{anchor:extformat}}
+====Formatting Drives on Linux====
+This section reiterates the usual steps in formatting a drive, with an emphasis on external drives (read the section on [[#intformat|formatting internal drives]] if you are adding a drive to you computer internally). If you are formatting to UDF (you probably should be), read the section on [[#udf|UDF]].
-Partly as an answer to the filesystem problem, a few Standards groups came up with UDF, the Universal Disk Format. It was mostly intended as the replacement for ISO-9660, and did become the official filesystem for CD-RW, DVD-RW, and Blu-Ray.
-The down side is that it does not use journaling, making data recovery after a crash or accidental unplugging a little riskier. It does not use partitions, but that is not usually an issue for an external drive.
+<WRAP indent>
+**1.** Identify the drive's device node.
-The good news is that it is open source, can use UTF-8 filenames that are as long as 255 bytes, file sizes and filesystem sizes of 2TB. Like FAT, UDF does not bother with permissions, making it ideal for external drives on Unix. At the very worst, even considering some of the features left out of UDF, it is a better and more flexible option than FAT, and has no patent issues to contend with (Micrososft sometimes sues companies for using random features in FAT).
+With the drive //unplugged// from the computer:
-Since it was primarily intended for optical media, creating a UDF volume is different from formatting a drive for any other filesystem.
+<code>
+# ls /dev/sd*
+sda  sda1  sdb  sdb1
+</code>
-    The drive being formatted must have no partitions on it. This is entirely unlike any other filesystem, but it is necessary for some operating systems to accurately detect the UDF filesystem.
+Then plug the drive in and do the same command again:
-    To get rid of the existing partition on a drive, zero out the first 4096 bytes of the drive.
+<code>
+# ls /dev/sd*
+sda  sda1  sdb  sdb1  sdc  sdc1
+</code>
-    dd if=/dev/zero of=/dev/sdx bs=512 count=4096
+The new node is the drive you have just attached.
-    Note that the bytesize (bs) is not flexible. It must be 512.
+**2.** Unmount the drive.
-    Next, find out the block count for your drives:
+It probably isn't mounted, but just in case...
-    $ df -i /dev/sdx
+<code>
-    Inodes   IUsed    IFree
+# umount /dev/sdX*
-    2040230   619    2039611
+</code>
-    Finally, create the filesystem so that it spans the entire drive.
+**3.** Create a fresh partition table:
-    mkudffs --blocksize=512 \
+<WRAP important>
-    --udfrev=0x0201 \
+From this point on, the device ''sdx'' (a rare drive node, since it would require you to have 23 drives attached to your computer) is used to protect people from copying and pasting commands and unintentionally erasing a real drive. Replace ''sdx'' with the actual label of your target drive.
-    --lvid="myUdfDrive" \
+</WRAP>
-    --vid="myUdfDrive" \
-    --media-type=hd --utf8 \
-    /dev/sdx || echo "fail"
-Now you can mount and use the drive on any platform.
+<code>
+# parted /dev/sdx mklabel msdos
+</code>
-[EOF]
+Or, for drives lager than 2TB:
+<code>
+# parted /dev/sdx mklabel gpt
+</code>
+**4.** Get the drive size:
+<code>
+# parted /dev/sdx unit MB print | grep Disk
+Disk /dev/sdx: 63417MB
+Disk Flags:
+</code>
+**5.** Create a partition spanning from 1 MB to the last available MB (which you will have gotten from the ''parted print'' command):
+<code>
+# parted /dev/sdx mkpart primary 1 63417
+</code>
+**6.** Put a file system on the partition you just created.
+<code>
+# mkfs.jfs -L rockhopper /dev/sdx
+</code>
+Or for an ''ext4'' file system (more common across Linuxes)
+<code>
+# mkfs.ext4 -L rockhopper /dev/sdx
+</code>
+Also consider using [[#udf|UDF]] for drives under 2TB.
+**7.** Unplug the drive and plug it back in.
+</WRAP>
+{{anchor:kvpm}}
+====KVPM Volume and Partition Manager====
+[[https://sourceforge.net/projects/kvpm/|KVPM]] is a graphical interface to the ''parted'' command and some related hard drive tools.
+Install KVPM from [[http://slackbuilds.org]]
+Launch **kvpm** from the **K Menu**.
+Once kvpm has launched, look at the list of drives for one that matches your drive in size and vendor. For safety purposes, you should **not** have any other external drives plugged into your computer at this point, so only the internal hard drive(s) and you target drive should be visible.
+Right-click on your target drive and choose **Add disk partition**.
+Accept the default options so that you are using all available space on the drive.
+[{{ kvpm_part.png | Create a partition on a drive.}}]
+One the new partition appears, right-click on the partition inside the drive and choose **Filesystem operations** → **Make filesystem**.
+[{{ kvpm_mkfs.png | Create a filesystem in the partition on your drive. }}]
+Use the filesystem type of your choice, give the drive a name (or "label"), and click **OK**.
+[{{ kvpm_fs.png | Name your drive and set it loose. }}]
+<WRAP centeralign>
+<wrap fa>[[dotdesktop|R]]</wrap> <wrap fa>[[start|S]]</wrap> <wrap fa>[[mimetypes|Q]]</wrap>
+</WRAP>
-[[{arrown.png|border:0;background:none;width:0;display:inline-block;float:right;}mimetypes|]][[{arrowp.png|border:0;background:none;width:0;float:right;}dotdesktop|]]