As I mentioned in a previous post, I’ve build a media server to store all of my music and movies for playback throughout my house. I decided to store everything the original, lossless format. This means that music is stored at full CD bit rate, and more significantly, movies are backed-up with all menus and extras in the original file format. Although I could save a bunch of space by compressing everything to a different format, I want to keep everything in the original format. The drawback is that this requires quite a bit of disk space, more than I originally anticipated.
I justed added another 500GB disk to my RAID array, for a total of 1.5TB of usable space. The great thing about the combination of an Areca 1230 RAID controller and Windows Server 2003 is that I can grow the array / partitions / drives as I add disks. Because this is not exactly a straight-forward process, and because I’m bound to need to do this again, I’m documenting the process here for my reference as well as for the benefit of others.
It is basically a three-step process. First, the RAID Set on the controller card must be expanded. For example, I had a 3-disk RAID 5 array with 1TB usable space (one 500GB disk is devoted to storing parity recovery data, so it is not usable space). After expanding the RAID Set, I have a 4-disk RAID 5 array with 1.5TB usable space. However, this additional space is not visible to the operating system; the operating system can only see Volume Sets and their sizes, not RAID sets (which are only visible to the controller card).
So at this point, the RAID Set is 1.5TB, but the Volume Set on that RAID set is still 1TB. So, the second step is to expand the Volume Set on the RAID controller card. A Volume Set on the RAID controller card can be thought of as a virtual disk drive that is presented to the operating system by the controller card. So I could either create a new Volume Set, which would appear as a new disk drive to the operating system, or expand the existing volume set. Since I don’t want a bunch of useless drive letters, I expand the existing Volume Set using the RAID controller card utilities.
Now that the Raid Controller Volume Set is expanded, it appears as a new, larger 1.5TB drive to the operating system. However, this space is not allocated, so the third step is for the operating system to expand the size of an existing “partition” on this RAID Controller Volume Set. Now, here is the slightly confusing thing, the operating system refers to this ‘partition” as a “volume set”. This should not be confused with the RAID Controller Volume Set. The Operating System Volume Set is a “partition” within the RAID Controller Volume Set. At this point, the process is complete and an existing driver letter on the server will have more space available. Below are step-by-step instructions:
- Install the new drive in the server, connecting the drive to the RAID controller card
- Boot Windows Server 2003
- Go to the Areca RAID controller configuration webpage (must have the Areca ARC HTTP server software installed on the server, the default web address is http://127.0.0.1:81 if you are on the server itself, that machines physical IP address and port 81 otherwise. The default password is 0000)
- Expand the RAID Set using the “Expand RAID Set” feature.
- Continue to use the computer as usual. (It took 22 hours 50 minutes to add a 4th 500GB disk in background mode; it took 12 hours 20 minutes to add a 4th disk through the RAID bios a boot-up, but couldn’t use the PC at that time.) You can even reboot if you need to, the expansion will continue in the background with no issues.
- When the RAID Set expansion is complete, perform a Volume Set expansion using the same Areca Web Interface. I simply expand an existing Volume Set to take up all of the available free space with the newly expanded RAID Set. It took about 1 hour 15 minutes to go from a 1TB Volume Set to a 1.5TB Volume Set and 2 hours 17 minutes to go from 1.5TB to 2.0TB.
- Once the expansion is complete, reboot Windows Server 2003, hit the F8 key the very instant Windows starts to boot, and select Safe Mode Command Prompt
- Logon to Windows, a command prompt will open, and run the DiskPart.exe utility. Here are some more details on the utility from the Microsoft website. (Windows XP does have this utility now. Norton Partition Magic may be an easier-to-use alternative if this seems too complicated. Instructions for Partition Magic are beyond the scope of this post.) This process takes only a few seconds. Here are the exact commands I typed:
- diskpart.exe (starts the partdisk utility)
- help (displays a list of commands, for reference only)
- list volume (to list the operating system volumes available and figure out the number of the volume you want to expand; think of these as disk partitions)
- select volume 2 (this selects my D drive, the drive I want to expand, may be different for you)
- extend (this extends the selected volume to take up all available free space after the current partition; if you don’t want to use all available space, get more details from the Microsoft site listed above)
- list volume (to make sure the intended volume is in fact larger now)
- exit (to exit partdisk)
- CTRL+ALT+DEL (to reboot the computer back into “normal” mode)
Reboot Windows Server 2003; you’re done!
Well definitely sounds like staggered startup is working for you – I’m curious to see how your friends works out. I do not have the LEDs plugged in from the card, but perhaps I should get them up to see exactly how everything is responding. It would be pretty funny if 3ware accidentally made their cards require that jumper to be connected on the card for this to work. Thanks for all your help so far, I wish other people with this setup would find this blog – I couldn’t find anyone else on google who had this going on…
Nate
Nate – I do NOT have the LED’s connected to the controlled. The LED Activity comes through the Supermicro Enclosures. The RAID Controller LED Connections have not worked for anyone that I know so far so I didn’t even bother connecting them. So this would have nothing to do with this working at this point.
It seems that the only differences we have here are the Model of the cards, Motherboard and Enclosures. I suppose it is possible that this could have something to do with the enclosures but I don’t believe that the backplane has any intelligence associated with it.
Just to make 100% sure as I said I will power cycle my main server (with the 750’s) in a few hours when I get home. I will keep you posted as I get more information on the other systems that others have.
Nate – The Staggered Spin up functionality on my main server has been verified. This morning I rebooted the system, changed the “Method” to ATA-6 then power cycled. When I did this ALL of the drives spun up at the same time. This resulted in a power surge of just under 5 Amps of power which is almost 600 Watts of power. From there I went in to the controller and reset it back to OOB and power cycled again to compare results. This time the drives were spinning up one at a time and the power “Surge” never went past 2 Amps which is about 240 Watts. Just for the heck of it I went back in and changed the settings to spin up 2 drives at a time still with 4 seconds in between. This resulted in about the same power usage and visually the drives were showing me 2 at a time and I could hear them spinning up and the distinctive “Click” when the drive was ready.
I have e-mailed some of the others that I have built systems for to see if they could try this one of them said he will try today. This person has a somewhat similar set up as I do but the following differences. His controller is an old 9500 12 Port and he is using a Standard ASUS Motherboard (does not have any PCI-X Ports, he’s just using it in a regular PCI Slot). Again comparing Apples to Oranges in a way but I still want to hear the results.
In my testing of Staggered Spin-up or as some other companies might refer to as “Power On Standby” I opened a case with Seagate on their ST3750640AS 750 SATA Drives. After going back and forth with them trying to get something other than a reference to a Knowledge base article I finally got this answer.
**This drive fully supports staggered spin up but there is nothing that you need to do to “enable” the function on the drive itself. Usually these functions are fully controlled by the controller card at least on our drives. If you have any additional questions, let me know.**
At least this will answer this question but for others it will raise others. I also asked the same question to 3Ware who did the same thing by quoting nothing but KB articles which is why I went to Seagate as well. I did however find the proper “Method Setting” in one of their articles so I guess I did get some answers.
~ Pete
When I open the D drive under my computer on server it freezes for about 15 seconds before it will let me browse the directories (there’s only like 10 directories on the D: not like it’s loading thousands of entries here). When I went to do an analyze for defrag I saw that data (according to windows) was extremely fragmented. My question is, if I defrag through windows will it even do it appropriately (I don’t know if it knows how to handle parity bits), and if not, is there any such of way cleaning up the data on a drive so that it runs more efficiently.
Nate
I’m positive that Windows has zero visibility to the parity bits. I also doubt it will make much difference, since every file is broken into blocks and written across multiple drives. But it definitely won’t hurt anything. It might optimize the partition table or something, but I doubt you’ll see much difference speed-wise.
Lots of replies and didnt read through them all. But to point something out, there is no need to restart the server into safe mode to extend the partition using diskpart, likewise no reason to restart after this has been completed either. Simply rescan and refresh within disk manager. We do this all the time . Infact I have never restarted a server since nt4 to extend a partition or restart for the new disk space to show up either.
Thanks for the feedback. I think you have to go into safemode if you are expanding a system partition, which was the case for me.
I felt the need to add to this post – as I found this thread to be the most informative on expansion.
I was pretty sure what I was doing, going by the manual and having done it once in the past, but this thread re-assured me as to the procedure.
The once caveat of my adventure, was that I was taking a 6TB array to 9TB (RAID-6)without (and shoot me now) a full backup! So was was terribly afraid (rightfully so).
This is where I have something to add (besides really needing a backup):
My procedure was going fine – as I expanded the RaidSet on the controller – that took a day or so – left it running over the weekend. When I went to expand the VolumeSet, I selected Foreground Priority in the build – thinking that it should get it done faster. I hesitantly pressed verify and go. The progress of the controller showed 67% and slowly counting up – so far, so good.
Then, the defication hit the rotary oscillator!!!
I went to access the drive that the 6-9TB was supposed to house and the data was not there – the DRIVE was not THERE!!!
OH MY FREAKING GOSH !!! WHAT HAVE I DONE !!!
Do I pull the plug, and start recovery now? Do I wail bricks against my head for not having a backup? Do I start telling customers that files from the past 3 years are GONE!
I scoured the web – faintly remembering that I selected foreground tasking. NOTHING!! Nobody seems to have ever selected this.
Back to the manual – for the 7th time. Finally searching on the right terms. There it was:
“• Foreground Availability/Background Initialization
RAID 0 and RAID 1 volume sets can be used immediately after
creation because they do not create parity data. However,
RAID 3, 5 and 6 volume sets must be initialized to generate
parity information. In Backgorund Initialization, the initialization
proceeds as a background task, and the volume set is fully
accessible for system reads and writes. The operating system
can instantly access the newly created arrays without requiring
a reboot and without waiting for initialization to complete.
Furthermore, the volume set is protected against disk failures
while initialing. If using Foreground Initialization, the initialization
process must be completed before the volume set is ready
for system accesses.”
To highlight:
“If using Foreground Initialization, the initialization
process must be completed before the volume set is ready
for system accesses.”
So I said several prayers in hopes that it means that the volume is COMPLETELY offline until it is done. And then I waited….
…..
……..
…………
about 12 hours later, it came online. (then had to rebuild to a hot spare for a new drive that failed mid initialization – more pucker factor!)
I then ran diskpart.exe – thinking I couldn’t screw it up any more!
To my relief – the drive is now 9TB and all the data is there!! AMEN!! Time to clean out my shorts!!
I guess the good news is that it doesn’t take days when you choose foreground – but you do take it offline for that whole time.
BTW – my hardware consists of:
SuperMicro Xeon mobo with only one 1.8Ghz HT XEON cpu and 256Mb memory (hey, it’s only a file server)
Areca 24port SATA controller – PCI-X
Lian-Lu? V2100 case (I think – 21 drive bays) housing 16 Seagate 750Gb drives – 14 drives in RAID-6 (paranoid RAID-5 with two parity drives instead of one) two dedicated hot spares
The array is not effectively de-fragged and alot faster with it not being so full.
Anyway, Thanks to Carlton on this very nice bit of information on the web to help us get through it all!
Hammy.
Hello,
I was running in a nightmare… let me explain what happened:
– I bought the highpoint 2320 adapter some weeks ago, so I thought it has an actual BIOS
– I used 3 1T HDs to setup a standard RAID 5 with 1 partition. It worked fine.
– Then I added another 1T HD- The integration worked fine, verifiieng also showed no problems. But the resulting disk has now 3T available space and neither Win XP nor Vista 64 can read it. It seems that it has a standard MBR due to the first RAID installation.
– Now I upgraded to the newest BIOS and driver
– But still no access -> same error using Win XP and Vista x64
– I converted the RAID several times but it keeps the MBR, so still no access
– During converting the RAID to a new RAID 5 there is no option to choose VSS now, but the conversation completed without errors. But still no access.
– Unfortunately I can not convert it back to a 3 HD system, it says to less space.
I have now to find a way to access all data to backup it before deleting and creating a new array?
:
1 Is there a way to change the MBR to a working GPT or similar?
2 Can I switch to VSS?
3 Can I use 2 -3 new HDs to create a new Volume with VSS or GPT and then migrate the old RAID into the new one?
Thanks for your help,
Markus
Markus: Sounds like you’re having some major growing pains with your RAID array. I think this post [How to Break the 2TB (2 TeraByte) File System Limit] will answer your questions.
I grew very frustrated with RAID for the exact reasons you’ve mentioned coupled with the fact that a bug in the controller card caused my data to become completely inaccessible/unrecoverable. Because of this, I switched to Windows Home Server and haven’t looked back. It makes storage expansion super simple and you don’t have to worry about partitions or anything else. UnRAID is another alternative if storage density is more important to you and you’re willing to spend a little more time on it.
Good luck!
@Carlton Bale:
Thanks. The problem is that my data is not accessable anymore. Unfortunately the post you mentioned doesnt show an solution. It seems that the highpoint administration tool converted the working 2GB Raid 5 array to an non working 3 GB RAID 5 array both with MBR.
So I am now desperately seeking a way to acess the data again….
Thanks for any help,
Markus
Markus: Thanks for the more detailed explanation. I didn’t realize that you had already expanded the partition as well as the actual RAID array itself. In general, it’s a two-part process where you use the admin tool to change the size of the RAID Set/Volume Set and the operating system (or third-party utility) to expand the partition. I would think you would only have issues if you tried to modify the operating system partition; the OS should just ignore the additional drive space. Unfortunately, this doesn’t seem like the case.
I’m not at all familiar with the highpoint admin tool. First off, I’d recommend calling Highpoint and talking with their support staff. Hopefully they can help you out. If not, you may want to try posting on the StorageReview.com forums, though I didn’t find much help there. Finally, you can try to down-size the volume set (not the actual array, just the volume set on it) back to the previous size. If the admin tool can do that, it might make your data accessible.
It’s problems like this that caused me to abandon RAID in favor of something more flexible and recoverable, the strong points for Windows Home Server and UnRAID.
Best of luck to you!
Thank you very much for your help. Yes it seems to me that the controller did the expanding but the OS does not get aware of the new situation. I can see 3 partitions in the diskmanager tool. 2 with 2T and a small one in between (some GB). The 1st got a drive letter and is shown in explorer but not accessable. So I believe that it is just the partition information that is wrong due to the MBR…
I already set up a WHS and want to copy the data to it, but therefore I need access…..
Thanks again,
Markus
Hey guys I hope that this is not inappropriate to post here but I thought it might be ok based on the subject matter of this thread. I am upgrading one of my RAID Arrays and wanted to see if anyone was interested in the old HDD’s. Here’s the info
I am selling a total of Ten (10) slightly used Internal Seagate 750GB SATA II Hard Drives Model ST3750640AS. These drives are currently being used in a RAID Array and are being replaced by larger drives.
As you know Seagate Drives comes with the best in the business 5 year warranty. Each of these drives have approximately 4 years left in their warranties – Expires between 03-22-2012 & 11-07-2012) depending on the individual drive.
As mentioned these drives are currently in service and will be removed in approximately one week. Once they are removed they will be shipped out to whoever purchases them.
I am asking $650 for all Ten Drives and I will throw in CONUS shipping and Pay Pal Charges.
Correct me if I’m wrong, but isn’t the data redundancy that windows home server offers basically only a software RAID1. So if you have 1.5TB of data you will need another 1.5TB for it to be backed up in case a hard drive goes bad…. Right?
Yes, Windows Home Server does redundancy through 2 copies of the same file instead of through parity data. It’s not RAID 1, but it is similar in data usage. It is less efficient from a storage density standpoint but better from a data recovery standpoint due to no stripped data and independent drives that allow data recovery even with multiple drive failures.
Anyone else have any issues with a 3Ware Controller and the Seagate 1.5TB Drives? This is so odd… I’ve had a total of 7 Seagate ST31500341AS Drives connected to my 3Ware 9550SX-16ML Controller in a RAID 5 since mid October and haven’t had any issues.
Last week I had one of the drives “Time Out” and went off line. I was able to rebuild the array by using the same drive by removing it and reinserting it. Just last night I had yet another drive to the same exact thing. I haven’t made any changes to the array whatsoever. No drive added or removed, the amount of data on the Array has increased slowly but I still have 1.9TB Free.
I did a little research and heard that there has been some issues with the drives firmware and there are newer versions out there.
Of course 3Ware says this is a Seagate issues and Seagate says this is a 3Ware issue.
Why would this work for 3 Months without a single hiccup only to have this happen twice in one week. We all know that if the second drive fell off line when the first one was off I would have been SCREWED!!!
Any thoughts???
I’d side w/3ware on this one (for now). Their name is as bulletproof as seagates was (until they started releasing drives >=1TB).
I’d say the reasons you’re having the drives “time out” is directly related to the problems the drives themselves are having with timing out while streaming data from them. For instance, if you’re even doing something as low intensity as playing an mp3 the song (directly from the drive no raid involved) will stop for 3-10 seconds before resuming play. In RAID configurations (due to how raid works) apparently the problem is amplified since the controllers are thinking that the drives are now offline or unavailable when really there’s just some odd delay issue going on with the drives.
Newegg reviews have documented this issue very well, so much that seagate even has a somewhat admittance of guilt – does this sound like it might be you?
ST31500341AS 1.5 TB
Some Seagate Barracuda 7200.11 1.5TB hard drives may show uncharacteristic operation when used with Mac and Linux operating systems in multi-drive configurations. Users may experiences pauses in video streaming applications or a dropped drive from RAID arrays. Customers seeing these symptoms should contact Seagate Technical Support for a firmware upgrade at [email protected] or http://www.seagate.com/www/en-us/about/contact_us/.
I’m having the exact same issue with a 1TB seagate drive, but support is currently refusing to admit that the problem is their drive. I’m pretty disappointed w/seagate since between home and work all the drives (except for a raptor 10K) are seagates (so over 30 seagates total) and I’ve never had an issue with them (until the larger than 750GB models). Mistakes happen I’m fine w/that, but the way they’re handling my problem with the drive I have is pretty frustrating.
Nate
I thought I would post an update, Seagate finally responded to the Support Case I opened after about 2 weeks (even though they said it would be 24 hours), they released a new firmware for several different drives of which my 1.5T’s were included.
I waited about a week to make sure that they didn’t pull down the firmware then applied it to a single drive to make sure that I didn’t have any issues. After a few days I upgraded the rest of them and **Knock on Wood** so far so good.
One thing I noticed right off the bat is that the Activity LED on the drives went back to normal. With the original firmware it was opposite, the LED was on when the drive was idle and off when it was inactive. I felt like this was a very good sign.
I’m switching over to an Unraid server and am going to use some of my drives from my raid5 server. Does anyone know if it’s possible to ‘shrink’ a partition in w2k3 server (google says probably for 2K3) and if it’s possible to shrink a raid5 array much like it’s possible to expand it. Hmmm..
I’ve never heard of shrinking RAID arrays. Your best bet may be getting a few cheap drives, using them temporarily, and then selling them on eBay, but that is a pain to do.