In the 1990's, breaking the 640 KB memory limit of DOS was a big deal. Now, with single hard drives reaching capacities of 1 TB (1,000 GB), it is very easy to create a RAID array larger than 2 TB, but your operating system may not be able to handle it. But if you don't plan ahead, your operating system will only be able to address the first 2 TB and all that extra storage beyond 2 TB will be unusable. Here is an overview of some of the methods you can use to get around the 2 TB limit.
Equipment Requirements:
- Hardware RAID controller capable of 64-bit LBA addressing (for volume sizes greater than 2 TB). For this example, I'll use an Areca ARC-1230 RAID card.
- Several hard drives to connect to the RAID controller to create a RAID array. For this example, I'll assume (five) 1000GB SATA drives.
- Drives must be configured in a RAID level 5 Volume Set. For the first two examples, I'll assume all five drives are members the same RAID level 5 volume set. RAID level 5 requires the space of 1 drive to be allocated for parity data, so total available storage space for 5 drives will be 4 drives x 1000GB = 4TB.
Required Background Information: I'm assuming you already have an understanding of RAID 5, its benefits, and requirements. If not, read this Wikipedia article. Now, let's discuss the difference between RAID sets, Volume Sets, and Operating System Partitions.
- RAID Sets are groups of drives that a RAID controller groups together to act as one single array. The individual disks are not visible to the operating system but rather are controlled by a hardware RAID controller.
- Volume Sets are create by the RAID controller and reside on top of RAID Sets. A Volume Set set is presented to the operating system as a single, virtual disk drive. This is a little confusing, but the RAID level (RAID level 5 in this example) is determined when the Volume Set is created (not when the RAID Set is created.) It is possible to have multiple Volume Sets residing on the same RAID set, and the Volume Sets may even use different RAID levels.
- Partitions are created by the Operating System and reside on top of Volume Sets. (Volume Sets appear as virtual disk drives to the operating system.) You can use have the Operating System create one or more formatted partitions on top of a volume set.
Breaking 2TB Option 1 – Use Windows with NTFS and GUID Partition Tables (GPT) partitions. It is possible for Windows to use NTFS partitions larger than 2TB as long as they are configured properly. Windows requires that the GUID Partition Tables be used in place of the standard Master Boot Record (MBR) partition tables. You will need Windows XP x64 Edition or Windows Server 2003 Service Pack 1, Windows Vista, Windows 7, or later for GPT support. (It is possible to mount and read existing GPT partitions under Windows XP and 2000 using GPT Mounter from Mediafour.; however, their MacDrive product does not support GPT partitions.) There are a couple of stipulations for GPT disks. First, the system drive on which Windows is installed can't be a GPT disk because it is not possible to boot to a GPT partition. Secondly, an existing MBR partition can't be converted to GPT unless it is completely empty; you must either delete everything and convert or create the partition as GPT. Read this Microsoft TechNet article for more details on GPT. To create GPT partitions, use the diskpart.exe command line utility or right click in Disk Management Console (click here for more details.)
- To summarize: 1 RAID array of five 1TB Drives -> 1 RAID level 5 Volume Set that is 4TB -> 1 NTFS GUID Partition Table Windows partition that is 4TB.
Breaking 2TB Option 2 – Use Linux with CONFIG_LBD enabled. Most Linux file systems are capable of partitions larger than 2 TB, as long as the Linux kernel itself is. (See this comparison of Linux file systems.) Most Linux distributions now have kernels compiled with CONFIG_LBD enabled (Ubuntu 6.10 does, for example.) As long as the kernel is configured/compiled properly, it is straight-forward to create a single 4TB EXT3 (or similar) partition.
- To summarize: 1 RAID array of five 1TB Drives -> 1 RAID level 5 Volume Set that is 4TB -> 1 EXT3 (or similar) Linux partition that is 4TB.
Breaking 2TB Option 3 – Use Standard Partitions and Create Multiple Volume Sets within a RAID array. A RAID array itself can be larger than 2 TB without presenting a volume set larger than 2 TB to the operating system. This way, you can use older file systems (that support only 2TB) and still have RAID 5 protection and more than 2 TB of total storage. To do this, put all 5 drives into a RAID set and create a 2 TB RAID Level 5 volume set — this will leave 2TB of the RAID set unused. Then create a second 2 TB RAID level 5 volume set. Boot into your operating system, create a partition on each of the 2TB virtual drives, and format each of the two 2TB virtual drives. The disadvantage is that there is not one single, large 4TB partition. The advantage is that 1) backwards compatibility for the file system and partitions and 2) they are both part of a RAID 5 array and are protected from single drive failures and only 1 drives worth of storage is sacrificed for RAID parity data.
- To summarize: 1 RAID array of five 1TB Drives -> 2 RAID level 5 Volume Sets that are 2TB each -> 2 standard NTFS (or any other) partitions that are 2TB.
Note 1: RAID Capacity Expansion. If your RAID card supports online capacity expansion, it is possible to expand any of the configurations above. For options 1 and 2, expand the RAID Set, then Expand the Volume Set, then Expand the Operating System partition. For option 3, expand the Raid Set, Create a 3rd Raid level 5 Volume set, and then create a third operating system partition. To learn more about expanding a RAID array on an Areca controller running Windows, read this article.
Note 2: Software RAID. Software RAID adds an additional level of complexity to RAID. For that reason, I recommend using a Hardware RAID controller. Having said that, I think everything mentioned above is technically possible if you are using software RAID, but I've never messed with it some I'm not positive.
to Mr. Unrein question – the built in array controller in the msa doesn't support logical volumes over 2TB. Sadly. Use the Compaq/HP ACU array config utility to configure your logical volumes (with raid) (looks like you'll get 3 logical disks 2, 2tb and maybe 1tb?) then use Microsoft's disk management mmc to create your spanned volume. Voila, 1 drive letter and about 5tb of usable fault tolerant space. The fault tolerance is what you chose during the logical drive setup. So if you choose raid 5, then you can lose 1 drive. If you do raid 6 you can lose 2 drives.
I told you that so I can ask this: using the scenario as Mr. Unrein's here, I want to extend the space on my MSA which uses sata drives connected to a Compaq Array controller 642.. On a different array controller, a 5300 series controller from Compaq are SCSI disks. I created 1 logical disk, raid 5. In the disk management console I converted the disk to dynamic. What issues does anyone see with extending the volume on to this dissimilar hardware?
Hardware and Software raids have their own pros and cons. While hardware raids are faster, a failure of the controller card means searching for a new one, and controller chips change very often. It may be very hard to find a replacement controller that can just accept your existing raid drives. Software raid avoids that issue, but there is a performance hit since your system processor is needed for parity calculation and that does take cycles regardless how optimized. I have 4x 2TB drives in a Radi5 array on Ubuntu Server, using the mdadm software raid, and it's working just fine. The performance hit doesn't matter since all that space is used for network shares.
In short, you need to know the pros and cons and decide for yourself what risks are worth what benefits.