In 2006, a RAID array linking together multiple 500GB drives was required to reach the 2TB (2,000 GB) limit and operating systems were just starting to implement solutions to break through this barrier. Now, with single hard drives exceeding capacities of 4 TB (4,000 GB), the need to break this 2TB limit is extremely common, and your operating system may not be able to handle it. If you don’t plan ahead, your operating system will only be able to address the first 2 TB and all that extra storage beyond 2 TB will be unusable. Here is an overview of some of the methods you can use to get around the 2 TB limit.
Operating System Requirements:
- Windows XP 32-bit: If you are using Windows XP 32-bit, you are out of luck for native OS support. XP was designed well before this barrier was approached and was not designed to exceed it. Some drive manufacturers offer driver software to allows the full capacity of large drives to be accessed under XP 32-but (Hitachi GPT Disc Manager, Seagate Disc Manager), but user beware. This support is not native to the operating system and can lead to reliability and data recovery issues. Your best bet is to upgrade to a modern operating system.
- Windows XP 64-bit, Windows Vista, Windows 7, Windows 8, or later: All of these operating systems support GUID Partition Table (GPT), which is a newer drive formatting process required to access beyond 2TB of drive space.
- If using the disc as an additional data drive, either the 32- or 64-bit versions of Vista/7/8 can be used.
- The 64-bit versions of Vista/7/8 are required to use large drives as the boot disc (the computer must also be equipped with an newer-style EFI / UEFI BIOS.)
- Mac OS X 10.6 and later is required to access large volumes. GUID Partition Tables (GPT) are required.
- Linux requires kernel version 2.6.x and later (32-bit CPU limit of logical volume size limit is 16TB, 64-bit CPI limit is 8EB.) (Also, the kernel must be compiled with CONFIG_LBD enabled, which is almost always the default.) GUID Partition Tables (GPT) are required.
Breaking 2TB Option 1 – Use Appropriate Version of Windows with NTFS and GUID Partition Tables (GPT) partitions. It is possible for Windows to use NTFS partitions larger than 2TB as long as they are configured properly. Windows requires that the GUID Partition Tables be used in place of the standard Master Boot Record (MBR) partition tables. You will need Windows XP x64 Edition or Windows Server 2003 Service Pack 1, Windows Vista, Windows 7, Windows 8, or later for GPT support. (It is possible to mount and read existing GPT partitions under Windows XP and 2000 using GPT Mounter from Mediafour.; however, their MacDrive product does not support GPT partitions.) There are a few stipulations for GPT disks:
- First, the system drive on which Windows is installed can’t be a GPT disk because it is not possible to boot to a GPT partition unless you have the 64-bit version of Windows Vista/7/8 and a UEFI system BIOS.
- Secondly, an existing MBR partition can’t be converted to GPT unless it is completely empty. You must either delete everything and convert, but you might as well just create a new GPT partition at this point. Read this Microsoft TechNet article for more details on GPT. To create GPT partitions, use the diskpart.exe command line utility or right click in Disk Management Console (click here for more details.)
Breaking 2TB Option 2 – Use Linux with CONFIG_LBD enabled. Most Linux file systems are capable of partitions larger than 2 TB, as long as the Linux kernel itself is. (See this comparison of Linux file systems.) Most Linux distributions now have kernels compiled with CONFIG_LBD enabled (Ubuntu 6.10 does, for example.) As long as the kernel is configured/compiled properly, it is straight-forward to create a single 4TB EXT3 (or similar) partition. In general, this is applies to Linux kernels 2.6.x and later.
Breaking 2TB Option 3 – Use Standard Partitions and Create Multiple Volume Sets within a RAID array. A RAID array itself can be larger than 2 TB without presenting a volume set larger than 2 TB to the operating system. This way, you can use older file systems (that support only 2TB) and still have RAID 5 protection and more than 2 TB of total storage. To do this, put all 5 drives into a RAID set and create a 2 TB RAID Level 5 volume set — this will leave 2TB of the RAID set unused. Then create a second 2 TB RAID level 5 volume set. Boot into your operating system, create a partition on each of the 2TB virtual drives, and format each of the two 2TB virtual drives. The disadvantage is that there is not one single, large 4TB partition. The advantage is that 1) backwards compatibility for the file system and partitions and 2) they are both part of a RAID 5 array and are protected from single drive failures and only 1 drives worth of storage is sacrificed for RAID parity data.
- Example: 1 RAID array of five 1TB Drives -> 2 RAID level 5 Volume Sets that are 2TB each -> 2 standard NTFS (or any other) partitions that are 2TB.
Background info on RAID Requirements:
- Hardware RAID controller capable of 64-bit LBA addressing (for volume sizes greater than 2 TB). For this example, I’ll use an Areca ARC-1230 RAID card.
- Several hard drives to connect to the RAID controller to create a RAID array. For this example, I’ll assume (five) 1000GB SATA drives.
- Drives must be configured in a RAID level 5 Volume Set. For the first two examples, I’ll assume all five drives are members the same RAID level 5 volume set. RAID level 5 requires the space of 1 drive to be allocated for parity data, so total available storage space for 5 drives will be 4 drives x 1000GB = 4TB.
RAID Required Background Information: I’m assuming you already have an understanding of RAID 5, its benefits, and requirements. If not, read this Wikipedia article. Now, let’s discuss the difference between RAID sets, Volume Sets, and Operating System Partitions.
- RAID Sets are groups of drives that a RAID controller groups together to act as one single array. The individual disks are not visible to the operating system but rather are controlled by a hardware RAID controller.
- Volume Sets are create by the RAID controller and reside on top of RAID Sets. A Volume Set set is presented to the operating system as a single, virtual disk drive. This is a little confusing, but the RAID level (RAID level 5 in this example) is determined when the Volume Set is created (not when the RAID Set is created.) It is possible to have multiple Volume Sets residing on the same RAID set, and the Volume Sets may even use different RAID levels.
- Partitions are created by the Operating System and reside on top of Volume Sets. (Volume Sets appear as virtual disk drives to the operating system.) You can use have the Operating System create one or more formatted partitions on top of a volume set.
Note 1: RAID Capacity Expansion. If your RAID card supports online capacity expansion, it is possible to expand any of the configurations above. For options 1 and 2, expand the RAID Set, then Expand the Volume Set, then Expand the Operating System partition. For option 3, expand the Raid Set, Create a 3rd Raid level 5 Volume set, and then create a third operating system partition. To learn more about expanding a RAID array on an Areca controller running Windows, read this article.
Note 2: Software RAID. Software RAID adds an additional level of complexity to RAID. For that reason, I recommend using a Hardware RAID controller. Having said that, I think everything mentioned above is technically possible if you are using software RAID, but I’ve never messed with it some I’m not positive.