Partitions Over 2.1 TB

Introduction

As hard drives have increased in capacity over time, some clever workarounds have been engineered to keep breaking barriers, such as the move from CHS to LBA. Unfortunately, with older style MBR partitions, there is a limit of 2.1TB that simply cannot be worked around using an MBR partition. We have to instead use newer GPT style partitions.

I'm not going to delve into the history too much, you can google/wikipedia it yourself if you like.

So, if you've just purchased a snazzy new 3TB drive like a Western Digital Caviar Green, and you're scratching your head as to why you can't get the full capacity, here's the fix.

Assumptions

The below documentation assumes you have an existing boot drive with Amahi, namely /dev/sda. Your >2.1TB HDD will be /dev/sdb

The instructions also assume a Linux terminal, such as that on your Amahi box (either using its console, the Terminal application within the GUI, or SSH'd in)

Warning

BE ABSOLUTELY SURE THAT YOU ARE RUNNING THESE COMMANDS AGAINST THE CORRECT DRIVE. WE CANNOT ACCEPT RESPONSIBILITY FOR ANY DATA LOSS, HARDWARE MALFUNCTION, ETC!

OK, Let's do it

First we label the drive with a GPT partition

bash code
​ parted /dev/sdb mklabel gpt ​

Now we select sectors as a unit of measurement

bash code
​ parted /dev/sdb unit s ​

Now we make the partition.

bash code
​ parted /dev/sdb mkpart primary ext2 2048 -1 ​

Parted may complain about the last sector and will automatically suggest a correct sector, just say Yes to the prompt.

We make it as a primary partition, ext2 filesystem, and we carefully select the sectors. The first number MUST be divisible by 8, so you can try 40, 48, 56, 64 etc... For me these numbers threw up errors, so I just went to 2048 and everything worked fine, and in the grand scheme of a 3TB drive, the extra sectors don't really matter :)

The reason that the starting sector is divisible by 8 is for aligning the sectors. This is because HDD's have traditionally been based around 512-byte sectors, yet newer, higher capacity HDD's are now arriving based around 4096-byte (a.k.a. 4k) sectors. The partitioning will still work with 512-bytes, but the drive's performance will be drastically impacted - in some testing it can be twice as slow, in others up to 25 times slower, depending on the sector alignment. So it's important to get this right!!!

Alternative Parted usage

Instead of the sequence of one liners up there, you can enter into parted itself like this. I give this as I don't know how your version of parted is going to behave. The interactive stuff from parted has been removed:

bash code
​ parted /dev/sdb (parted) mklabel gpt (parted) unit s (parted) mkpart primary ext2 2048 -1 (parted) quit ​

Format the drive!

Finally, let's get that drive formatted to a nice ext4 volume:

bash code
​ mkfs.ext4 -T largefile4 /dev/sdb1 ​

Adding the drive to your HDA

Normally to add a drive to Amahi, you would run the script hda-diskmount, which searches for any unmounted drives and mounts them. Unfortunately, this script doesn't yet cater for GPT partitions. It runs an fdisk -l and searches out drives with a Linux system type, and with what it finds it mounts any that aren't mounted.

If we run fdisk -l by itself, we will see that our new drive doesn't return a system type of 'Linux', instead it returns 'GPT' as well as a warning:

bash code

​WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes 255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 1 4294967295 2147483647+ ee GPT Partition 1 does not start on physical sector boundary.​

So hda-diskmount correctly ignores the drive. You can either fix hda-diskmount, or you can manually mount the drives with the instructions below.

First we make our mount point

bash code
​ mkdir /var/hda/files/drives/sdb1 ​

Next we exercise good practice and backup our fstab file, I like to use `date +%Y%m%d` to give the name a date stamp e.g. /etc/fstab.20110731 represents a backup done on the 31st of July, 2011. This isn't necessary, you could just use something like 'fstab.old', but date-stamping your file backups is a good habit to get into

bash code
​cp /etc/fstab /etc/fstab.`date +%Y%m%d`​

Now we adjust the fstab file

bash code
​echo -e /dev/sdb1$'\t\t'/var/hda/files/drives/sdb1$'\t'ext4$'\t'defaults$'\t'1 1 >> /etc/fstab​

What this does is adds a line to the /etc/fstab file, telling it that when Amahi boots, it has to mount the drive. The $'\t' entries simply mean 'put in a tab space'

If you now run cat /etc/fstab you should see a nice entry at the bottom, similar to this

bash code

​cat /etc/fstab # # /etc/fstab # Created by anaconda on Thu Jul 28 23:56:48 2011 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/vg_hda-LogVol01 / ext4 defaults 1 1 UUID=cc1204dd-a4f8-42f8-8736-b5970251a865 /boot ext4 defaults 1 2 /dev/mapper/vg_hda-LogVol00 swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/sdb1 /var/hda/files/drives/sdb1 ext4 defaults 1 1​

Now you can either reboot to allow the fstab file to mount your new drive, or you can simply run mount -a

Now if you run mount by itself, you should see your drive listed e.g.

bash code

​mount /dev/mapper/vg_hda-LogVol01 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) fusectl on /sys/fs/fuse/connections type fusectl (rw) /dev/sdd1 on /media/520abb16-ac7c-4f65-9cc3-9e536c93dca2 type ext4 (rw,nosuid,nodev,uhelper=udisks) /dev/sdb1 on /var/hda/files/drives/sdb1 type ext4 (rw)​

A note about Western Digital drives

Western Digital make some nice hardware, but they are also incredibly stupid. Many of their hard drives will automatically park their heads every 8 seconds, which wears out the hard drives very quickly, in some cases the HDD's will reach their design limit (300k parks for consumer, 600k parks for enterprise) within 3-4 months!!! This isn't to say that the drives will necessarily die within 3-4 months, but it's still not good!

Fortunately we can fix this using a Western Digital tool named wdidle3. You can download it here.

Create a bootable usb drive using FreeDOS and unetbootin, then copy wdidle.exe onto the usb drive. Boot up your Amahi box to FreeDOS and enter the following commands:

bash code
​ c:\ wdidle3 /R wdidle3 /S300 ​

This cd's you to the usb drive's root, then we run wdidle3 /R to display the current drive status, then we run wdidle3 /S300 to set the park time to 5 minutes. Alternatively you could run wdidle3 /D to completely disable the parking.

When you're done, simply restart the PC!

n.b. Unetbootin was a bit finnicky for me, I found that if I created the drive with a FreeDOS live CD that it wouldn't work, but if I then re-created using unetbootin's automatic distribution download feature, the USB stick worked a treat. YMMV.