As I mentioned in a previous post, I've build a media server to store all of my music and movies for playback throughout my house. I decided to store everything the original, lossless format. This means that music is stored at full CD bit rate, and more significantly, movies are backed-up with all menus and extras in the original file format. Although I could save a bunch of space by compressing everything to a different format, I want to keep everything in the original format. The drawback is that this requires quite a bit of disk space, more than I originally anticipated.
I justed added another 500GB disk to my RAID array, for a total of 1.5TB of usable space. The great thing about the combination of an Areca 1230 RAID controller and Windows Server 2003 is that I can grow the array / partitions / drives as I add disks. Because this is not exactly a straight-forward process, and because I'm bound to need to do this again, I'm documenting the process here for my reference as well as for the benefit of others.
It is basically a three-step process. First, the RAID Set on the controller card must be expanded. For example, I had a 3-disk RAID 5 array with 1TB usable space (one 500GB disk is devoted to storing parity recovery data, so it is not usable space). After expanding the RAID Set, I have a 4-disk RAID 5 array with 1.5TB usable space. However, this additional space is not visible to the operating system; the operating system can only see Volume Sets and their sizes, not RAID sets (which are only visible to the controller card).
So at this point, the RAID Set is 1.5TB, but the Volume Set on that RAID set is still 1TB. So, the second step is to expand the Volume Set on the RAID controller card. A Volume Set on the RAID controller card can be thought of as a virtual disk drive that is presented to the operating system by the controller card. So I could either create a new Volume Set, which would appear as a new disk drive to the operating system, or expand the existing volume set. Since I don't want a bunch of useless drive letters, I expand the existing Volume Set using the RAID controller card utilities.
Now that the Raid Controller Volume Set is expanded, it appears as a new, larger 1.5TB drive to the operating system. However, this space is not allocated, so the third step is for the operating system to expand the size of an existing "partition" on this RAID Controller Volume Set. Now, here is the slightly confusing thing, the operating system refers to this 'partition" as a "volume set". This should not be confused with the RAID Controller Volume Set. The Operating System Volume Set is a "partition" within the RAID Controller Volume Set. At this point, the process is complete and an existing driver letter on the server will have more space available. Below are step-by-step instructions:
- Install the new drive in the server, connecting the drive to the RAID controller card
- Boot Windows Server 2003
- Go to the Areca RAID controller configuration webpage (must have the Areca ARC HTTP server software installed on the server, the default web address is http://127.0.0.1:81 if you are on the server itself, that machines physical IP address and port 81 otherwise. The default password is 0000)
- Expand the RAID Set using the "Expand RAID Set" feature.
- Continue to use the computer as usual. (It took 22 hours 50 minutes to add a 4th 500GB disk in background mode; it took 12 hours 20 minutes to add a 4th disk through the RAID bios a boot-up, but couldn't use the PC at that time.) You can even reboot if you need to, the expansion will continue in the background with no issues.
- When the RAID Set expansion is complete, perform a Volume Set expansion using the same Areca Web Interface. I simply expand an existing Volume Set to take up all of the available free space with the newly expanded RAID Set. It took about 1 hour 15 minutes to go from a 1TB Volume Set to a 1.5TB Volume Set and 2 hours 17 minutes to go from 1.5TB to 2.0TB.
- Once the expansion is complete, reboot Windows Server 2003, hit the F8 key the very instant Windows starts to boot, and select Safe Mode Command Prompt
- Logon to Windows, a command prompt will open, and run the DiskPart.exe utility. Here are some more details on the utility from the Microsoft website. (Windows XP does have this utility now. Norton Partition Magic may be an easier-to-use alternative if this seems too complicated. Instructions for Partition Magic are beyond the scope of this post.) This process takes only a few seconds. Here are the exact commands I typed:
- diskpart.exe (starts the partdisk utility)
- help (displays a list of commands, for reference only)
- list volume (to list the operating system volumes available and figure out the number of the volume you want to expand; think of these as disk partitions)
- select volume 2 (this selects my D drive, the drive I want to expand, may be different for you)
- extend (this extends the selected volume to take up all available free space after the current partition; if you don't want to use all available space, get more details from the Microsoft site listed above)
- list volume (to make sure the intended volume is in fact larger now)
- exit (to exit partdisk)
- CTRL+ALT+DEL (to reboot the computer back into "normal" mode)
Reboot Windows Server 2003; you're done!
This is a GREAT article and very helpful for anyone looking at single volume large capacity storage. However due to the ever increasing drive capacities and storage needs, it needs to be amended/enhanced with the following critical information:
When using an Areca controller (the most reliable I have ever used) you should setup your >2TB support as follows:
1) Try to create an initial RaidSet LARGER than 2TB to start off, so that the controller prompts you to choose how to handle that much data.
2) Use "LBA64" >2TB support option!
3) Once the RaidSet is created, create the VolumeSet.
4) In Server 2003 create an NTFS partition using GPT not MBR [you can convert from MBR to GPT (just right click-in Disk Manager) but only if there is no data on the partition as it will be lost!]
NOTE: If you already have data, and are not configured as described above, you will HAVE to backup your data and redo the config! No other workaround!
Now going forward, whenever you add more HD's to the controller and after you expand the Raidset & VolumeSet, you will need to boot into SAFE mode in windows and run DISKPART to expand the volume. If you try to do it while in normal startup mode it will NOT work!
With some of the Areca controller models handling 24 drives, following these guidelines you can have over 512TB in ONE NTFS volume as drives get larger, and never worry about space!
Enjoy!
Stefan.
Http://solutions.semotech.com
Hello,
I will try and explain my case and maybe someone can help me with this problem that I have.
I had Win XP 64bit with 1 areca ARC-1120 SATA raid controller. On the raid controller I had 5×750GB Seagate 7200.10 HDDs on a single RAID5 volume of total 3TB.
When this volume was first configured I made it Dynamic Disk and used GTP so that Windows XP could read the greater than 2TB volume.
Everything was working fine but 2 days ago I decided to upgrade to Vista Ultimate 64bit.
I installed vista 64bit then chipset, SATA, Ethernet drivers. After that I installed the areca raid drivers (used latest) open Computer Management and import the 2.7TB foreign disk to Vista 64bit. It worked just fine, I even opened some files from windows explorer.
After that I installed some applications and video drivers (EVGA 8800GTS) and on the next reboot my volume was missing and when I opened Computer Management the volume was offline. From there on I couldn't re-import the disk.
After that I decided to install Win XP 64bit as a secondary OS to check if everything is ok from there. After raid driver installation (used latest) I couldn't even import the foreign disk from computer management under XP 64bit. I get this error in event viewer.
INTERNAL Error - The disk group contains no valid configuration copies (C10000B6).
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
The strange thing is that if I open the archttp software my volume is there and it’s shown at Normal state.
I also installed R-Studio software and that software can read my volume just fine.
I just cannot understand why I cannot import the volume on XP or Vista 64bit.
Any help will be greatly appreciated.
Hardware Config
Motherboard: Supermicro X7DAL-E+
CPU: Intel Xeon 5148
RAM: 4GB Kingston FB-DIMM
Video Card: EVGA 8800GTS 320MB
SATA RAID: Areca 1120 8port PCI-X
Hard Drives: 5×750GB Seagate 7200.10 series
Still on topic of expanding the capacity of a storage server, I have ran into a new problem. My 650watt Thermaltake power supply cannot take any more hard drives.
I currently have 10 seagate 750gbs in the array + another hd for OS and a dvd rom. After installing the 11th hard drive the machine will power on and then power itself off after about 1-2 seconds. If I unplug the power of the new hd, the server works fine. This was not a problem I anticipated since my current calculations I should at MAX only be using 400watts (probably more like 350), unless the raid controller card takes an inordinate amount of power to work, but I don't that since it's just pci-express and doesn't even require the extra power running to it.
Anyways, so I'm wondering for those with large arrays, what type of power supply do you have?
Oh yeah and 1 more thing, I did attempt booting the machine with the 11th hard drive powered and un-powering the dvd rom, but received the same result. I was estimating that the hd's only used about 25 watts each, now I'm thinking it's much more than that.
Nate
Nate - There are SO many variables when it comes to power consumption in any system. You have to consider not only the hard drives and the number of them, you also need to consider the CPU, Video card and lets not forget cooling as well.
What you are really running in to here is the initial surge. When a system powered up and the RAID Controller sends the start command to the drive each of them will start spinning at the same time which puts a huge load on the power supply. SCSI Drives dealt with this a lot better in allowing a delayed spin up command to be issued by the controller. While most SATA RAID Controllers support this most current drives do not. The only ones that I have found that do are Western Digital Drives and even so that was something I have to dig the information up for (you need to install a jumper on it then set the command in the RAID Controller).
If you would put your system on a power strip that has an AMP Meter or use some other method to measure the amount of current your system draws you will see that when you power up the system it might take as much as 10Amps when the drives all spin up but once they settled down it will drop down to something much lower probably in the 2 - 3Amp range which is roughly 240 to 360 Watts.
I have a system with 7 drives in it right now and it is taking only 1.2 to 1.3 amps of power once running. That is roughly 150 Watts of power.
A friend of mine built pretty much the same exact system (Same case, Motherboard, Hot Swap Cages and Drives) but went with an insane CPU and an even more insane video card. When I did some work on it I was shocked to see that it was drawing a constant 4.5 amps of power which is 540 watts! He's got a total of 15 drives in his so I thought that was the difference. When I saw the dramatic difference I did some checking and removed half the drives and it was still drawing just about the same amount of power. So that shows that the CPU / Video Card (I'm assuming the video card is the issue seeing that it has a direct power connection in to it).
With that said I am using Mushkin Power Supplies and haven't had any issues whatsoever. Both of the systems I mentioned are running the 550 Watt Dual-Rail versions which take the initial surge very nicely.
Hope this helps!
Hard drive consume only about 8-10 watts once they are running. However, the initial start-up can require 2-3 times more power and I'm guessing that is your issue.
I have 10 drives connected to my Areca RAID controller + 1 system drive connected to the motherboard. I have an Antec Neo Power 500 ATX12V 500W Power Supply. The difference is that the Areca RAID controller supportes staggered spin-up, so the drives come online one after the other. The controller allows me to specify the interval between drive spin-ups; I use 2 seconds as the interval.
In my experience, it has nothing to do with the SATA hard drives and everything to do with the hard drive controller. All SATA drives will allow staggered spin-up; the key it controlling and coordinating that. My Areca RAID controller has a BIOS that loads at startup. It pauses the boot process while the drives are being powered-up. Once all drives are online, the controller bios allows the PC to continue booting.
I have 3Ware Controllers and they support Staggered Spin up but according to them Seagate Drives SATA Drives do not support it??? I searched high and low on their website and couldn't come up with anything. I can say that without any doubt the drive all spin up at the same time with 3ware as well as Adaptec controllers with Delay Spin Up enabled and the settings set properly.
I have not used any other controllers except for SCSI which almost all support Delayed Spin up that actually works.
If someone knows how to get my 3Ware Controllers to work with Staggered Spin up please let me know.
I think spin-up, spin-down are standard SATA commands. The Areca controller supports spin-down of the drive during normal operation for power savings. But maybe some drives spin-up as soon as they receive power and don't give the controller a chance to stagger spin-ups. I'm very pleased with the Samsung drives I have, much quieter than the Seagate drives.
Yes, thank you for the replies, that makes a lot of sense. I should have been more specific in my system though, it's pretty barebones. It's headless (no keyboard, mouse, monitor) so I use onboard video (I plug a monitor + keyboard in when I need to expand the array) and ethernet. The one card plugged in on the motherboard is the 3ware raid controller. I can't remember the processor, but it's nothing power consuming since I built this as a storage only machine. That's why I was so shocked about having a power issue, but after you explained the initial power surge it makes complete sense. I will look on my 3ware controller for a staggered startup option - I hope it has one because if it doesn't then my only other option is to upgrade the power supply (which will be a LOT).
However, since the controller supports hot-swapping, what if I have the computer on and then plug in the power to the drive? I've never done that before, but it's an option. Obviously if I ever power it down (which I only do when I'm installing a new hd), I'll have to unplug the power again before powering it on, but it seems viable.
As for measuring the power, I have my storage server plugged into an apc unit with an lcd screen so I will have to watch it and see if it updates quickly enough to show the draw.
Nate
Nate -
Yes the Controller Supports Hot-swapping but if you would power the system on without the drives plugged in the controller will see them as missed and fail them. This could lead to issues and possibly data loss. I don't think I would go this route.
I don't know what model 3Ware controller you have but it should support staggered spin up but as I mentioned my controller (9550-16 and a 9550-8) it doesn't work with my Seagate 750's for some reason. I actually opened up a web case this morning with 3Ware to see if I can get this resolved.
To add to the initial power draw of a hard drive. According to the Specs of a typical 750GB Seagate Hard Drive the start up power draw can be just under 35 Watts of power but once it is up and running it settles down to right around 13 watts of power. So you can see how your power supply can have an issue with this initial surge.
Also what I meant to say in my last post is that if you remove the power from the drives then connect them the controller will see them as missing during the initial post and fail them. This would be bad!
Pleblanc - I have a similar setup to you. Seagate's and a 3ware controller. I have the 9650-16ML and I can see in the web interface where it says staggered start-up, but I can't change those numbers (or enable/disable it) and the help file is useless as it says it can be changed, but doesn't say how.
I'm guessing this can't be changed in the web interface and needs to be changed in the other interface during boot-up. In the help file though it does say that you can enable it, but that if the hard drives don't support it, the setting will just be ignored (so even though it will appear to be working in the controller setup, the staggered setup doesn't actually happen). Now I'm thinking I should have opted for the enterprise editions of the hard drives rather than the regular seagate 750s…. Hopefully the regular ones will support it, but I'm doubtful.
Nate
Pleblanc - Yeah I would have to boot into the motherboard's bios each time, give the hd power, then exit bios, and then the motherboard bios runs again followed by the 3ware controller which would then see the HD, therefore the controller would have never known that the HD wasn't initially there. It's a poor solution don't get me wrong, but I might have to do it temporarily as I have an extra 750 in the machine right now not being used and I need the space.
The real solution is to get staggered startup working, but I don't see any mention of that feature on newegg's site. I have the 750's that end in "AS" - when I get home I will have to play around with it more - currently I'm just remotely connecting to the server and looking for the option in the web interface which I'm not seeing.
Nate
Hey everyone I thought I would let you know about something I discovered over the last few days.
As mentioned above I have a 3Ware 9550SX Controller. Recently I expanded my Array from 7 to 8 drives this process took about 2 days to complete. This is where it gets interesting, I have my controller set to auto-verify the array once a week. This past Wednesday my usual verify kicked in and I was surprised to see that I got a few hundred sector relocate errors.
This made me wonder why this wasn't caught when I expanded the array. The reason behind that is because the controller does NOT verify the data when it expands. To confirm this what I did was remove the failing drive then reinsert it and rebuild the array. This process completed successfully in only about 5 hours or so (which I thought was impressive) I was kind of surprised to see that I didn't have any errors. Once the rebuild completed I kicked off a verify and sure enough an hour or so in to it the same drive started to show errors / issues.
So a nice little safety tip is to Make sure that once you expand your array you immediately kick off a verify. This holds true especially if you don't have auto-verify enabled (which I do not believe is the default) or if you have verify scheduled for sometime in the future. Also if you do have "Follow Schedule" enabled in the Controller then you might have to Set Ignore Schedule to get it to kick off immediately.
I have an open case with 3Ware support and I mentioned this to them and they confirmed that when expanding the system does NOT verify the data. This to me seems WAY more risky then I would have ever thought. I guess that is why they say to make sure you have your data backed up huh?
BTW - I opened a case with Seagate to see if their ST3750 AS Drives support Staggered Spin up cause 3Ware won't answer the question. They keep referring me back to the drive manufacturer.
Nate - I thought I would drop you a note and let you know what I have discovered with my setup. As I mentioned I have a 3Ware 9550SX-16ML Controller and 8 Seagate ST3750640AS Drives. I thought that I tried the various Staggered / Delayed Spin up settings in the controller but wanted to take another look at them as they were not working.
I looked through my 3Ware settings through the 3DM software but couldn't really find anything that would let me change this setting. I was only able to when I rebooted the system and accessed the controller directly.
I am very happy to announce that I got my delayed spin up working. I am not sure what the default settings are for the controller but the config that worked for me was to choose a single drive at a time (just to make it easier for me to follow the start up), 4 second delay and OOBSATA for the method (this was not the default).
I power Cycled the system and sure enough the drives spun up one at a time. I can't believe that it took me messing with this myself to figure out. I have cases open with both 3Ware and Seagate and get nothing but each company telling me to check with the other company. Well this means that these drives ARE Power On Standby Capable.
It is worth mentioning that in my set up I have my drives in Supermicro Hotswap Enclosures but I do not believe that has anything to do with this working.
Well I'm glad you got it working, because I sure can't seem to get it! I saw in one of 3ware's manuals about that staggered option only being able to be set in it's bios on bootup so I got that step, but the staggered startup was already enabled for me w/1 drive and 1 second delay and on sata. I played around with the numbers a bit (1 drive at a time, 6 seconds inbetween, 2 drives at a time, 4 seconds - all that stuff), but still no luck.
I watched the power meter on my ups unit and sure enough when I plug in that extra drive it hits 480watts-ish and the machine shuts off a second later. As soon as I unplug it the machine takes up about 470watts and boots fine (then shortly after goes to 200-220, and once booted stayed around 180).
I thought the problem might be in the motherboard bios (not letting a pci-e card have controller over power or something), but I didn't see any option that would make a difference. I would say that the drives are not compatible with staggered startup, but since you've got them to work I'm pretty stumped.
I'll play around some more, but if I get it working I'm guessing it will be a bios option somewhere and with this tyan motherboard they've got a million sub-menus and settings. I of course checked in windows that the 3ware controller accepted the changes correctly and everytime I would also turn off and unplug the computer wait a minute and attempt to boot it back up (with the additional drive plugged in). Everytime the power would surge to 480watts and the machine would shut off. Not the way I wanted to start the new year, but oh well, I'll get it working somehow! Any ideas?
Nate
Do you have the "Staggered Method" set to SATA OOB or ATA-6? This is what made the difference for me, I believe mine was set to ATA-6 SATA6 and when I changed it I got it working.
I don't believe that anything in the Motherboard BIOS should prevent this from working. I believe the way that this works is the Controller is waiting to send out the Start Signal to the drive based on the staggered start up parameters.
From what it sounds like this should be supported by any SATA II Drive as it is part of the standard. I do however know that the Western Digital Drives need a jumper installed on it to work.
Other background info my machine (don't think it matters), the OS does not boot from the storage array, but a single and separate sata drive, I have pulled out all of the 1.5gb jumpers on my seagate 750s to make them run at 3.0gbs, I do not have BBU unit installed on the raid controller.
If you changed any other settings anywhere, let me know. Thanks.
Nate
It sounds like we have very similar set ups but just to confirm my set up…
I also boot of another HDD that is connected directly to the motherboard and not the 3Ware controller. I do not have a BBU installed on the controller and have had my drives set to 3Gbs since day one (and confirmed through 3Ware Software). Finally I do not have the BIOS enabled on the controller as I do not boot off it.
So I believe the only differences we have are motherboard, I am using an ASUS Workstation / Server Board and I also am using Supermicro CSE-M35T Hatswap Enclosures for my drives.
thanks for the great tips guys.
I run a areca 1220 raid card with 4*500GB disks, I just added a 5th to expand the capacity. However, when i expanded to server2003 "dynamic" partition is has create two 'blocks' both called Data (D:)
is this correct or should there be only one 'block' called 'D' - the array is reporting the right size (2tb) but doesnt look optimal to me.
thanks again
Ian
Sounds correct to me if I understand your description. You expand the RAID set, then you have to expand the NTFS data partition to fill the newly available space.
Yeah the drives power just is not being controlled by the raid card, they all just instantly spin as soon as it's turned on, hmmm. In 3ware bios it's been turned on to OOB-SATA, but I even tried ata just as a fluke to see if I could that to work. I've played around with several motherboard settings, but no luck there either (although I did find one about enabling sata raid - which I did, but I'm guessing that one is more for software raid - but now that I type that I don't think my mb has a software raid). I guess I could try contacting 3ware's support, but from the sounds of it, they'll be of no assistance.
It's also lame that the power supply I have is for 650watts and craps out as soon as it peaks above 480 watts for a second.
I turned 3ware bios "off," but all that did was make the controller say "bios not installed" during it's initialization. I'll have to start digging through the motherboard manual, since I can't possible see how the settings for the 3ware controller are off.
Nate
Ian - I don't have the RAID Controller that you have so I can't speak exactly to your situation but I have expended my arrays several times without an issue.
Once you finished expanding the array through the controller are you saying that the additional space showed up within Disk Manager at the end of the same bar but at the end? If this is the case then all you have to do is run Diskpart.exe and expand the NTFS Partition.
Couple of things to keep in mind here though, With the set up you mentioned you would have just went over the 2TB limit of a "Regular" Partition. Did you make this a GPT Disk within Windows? Also some RAID Controllers have something called Auto-Carving that will report anything over 2GB Partitions to the operating system as a separate disk.
So the bottom line is that if you expanded the array properly and the partition within windows is GPT you can keep expanding the same drive without any issues.
Pete, what case was it that you modified to accept the SuperMicro drive cages?
What do you guys use to backup your 4+ TB arrays, or are you using them strictly as backup servers?
Jim - I have built several servers with 3 of the SuperMicro Hotswaps in each of them and haven't had any issues whatsoever. I am using AeroCool Masstige Cases. Each of them have a total of 10 5-1/4" Bays which allows for 3 of the hotswaps and one CD or DVD-ROM Drives.
Here is a picture of a few that I've built recently. http://www.pete-man.com/images/RaidServer.JPG
As for backup that is really the 64 thousand dollar question isn't it? My large servers are Video Servers so pretty much everything I have on there I have the original for. Obviously there are many, many backup solutions that are capable of this kind of backup but they are NOT cheap. Hopefully someone somewhere will come up with a low cost backup solution for these types of applications. I certainly don't plan on backup this amount of data on upwards of 1000 single layer DVD's
Yeah this is my storage server and the backup is that it's raid 5 with a ups unit. If my apt burns down, well then I have insurance and my storage server will be a small problem compared to the real stuff I lose.
Anyways, I am just about ready to give up w/my problem, I am absolutely stumped. I updated the driver and firmware for the controller (which btw is amazing, the controller was in use the whole time the firmware updated, no down time or annoying command prompts necessary), I played around with numerous motherboard settings - I thought I had it with enabling the memory cache to write through during bootup (the cache for irq port 16, which is the port that the raid controller is on, I saw the memory range in the driver info in windows), but that didn't work. Out of frustration I enabled everything to write in cache and then I wasn't able to get into bios (or bootup) so I had to flash it.
I think I am going to buy the motherboard that Pleblanc is using and try that - and I think tomorrow I will try to talk to tyan support, but that's pretty much a waste of my time. I don't really know what else could be different or going on, I have checked and re-checked everything, the power just always surges up every time the button is pressed. I really wish there was some software app out there to test the hard drive to see if staggered startup would work (perhaps the firmware on my 750's isn't compatible or something).
Nate
Assuming it's not my motherboard, then the only thing it could be IMO would be the firmware on one or all of these seagates, Out of the 10 drives I have, I have 3 different firmwares, they are
3.AAJ
3.AAC
3.AAK
I'm thinking of turning off auto-rebuild and unplugging all the older firmware versions (I'm assuming the lower the alpha letter the older it is) and just trying to see if I can get the newer ones to stagger-start. Pleblanc if you don't mind can you let me know what firmware versions you have so if you have one of the ones I have I can just use that initially for my test. Thanks,
Nate
Nate - I don't know if the firmware of the drives would have anything to do with it or not but just to compare I have a total of 8 drives in my array and only 2 firmwares at this point they are as follows. 3 Of them are 3.AAK and the remaining 5 are 3.AAE according to the 3Ware software. Lets say for the sake of argument that this would have something to do with it then some of the drives would spin up and others wouldn't???
Again I personally do not think that this has anything to do with the motherboard. Maybe it has something to do with the SuperMicro Enclosures that I am using but I don't think that is the case either. From what I understand the way this is supposed to work is the controller has to tell the drives not to spin up until it tells them to. If the controller doesn't tell them to "Wait" then they will spin up upon power up which is what they are doing. So as long as these drives connected to the controller which they are then it should work. I don't know what else could be the issue.
Also I forgot to mention that I am also using my RAID 5 as my backup so I would have to suffer data corruption or loss of 2 drives at the same time for data loss to occur. Hopefully **Knock on Wood** that doesn't happen anytime soon
I agree Pleblanc, I don't think the motherboard or the firmware versions is the issue, but I'm reaching here since I can't think of anything else and am looking for any differences between a working setup (yours) and one that's not (mine). I am going to try just the AAK's initially since we have that in common. If you have any other ideas, please let me know! I'm desperate!
Nate
Well on port 2 of the controller I had all AAK's, so I attempted to just boot up with just that plugged into the controller (but everything has power, so I was hoping that it would stagger the 4 AAk's that were on the 2nd port), but it didn't work (same result, 480 watts).
I even tried putting it on another PCI-E port in the motherboard, but didn't work. I will say that when you boot up into BIOS upon failure (since it's not seeing all the drives) it tells you that if you've unplugged them just to plug them right back in and do a re-scan. This gave me confidence that my workaround that I don't want to do will be okay. That being to manually plug in the power to the additional drives AFTER bootup has started. This is of course not what I want to do, but it's not really that big of a deal since I never have to turn off machine and a restart shouldn't produce that same huge jolt of power.
To me the problem has to either be 1) faulty 3ware card 2) motherboard is not allowing the card to have control over the devices 3) I just have drives that aren't compatible 4) is your setup really working?
I say #4, because I noticed that it sounds like my drives stagger start sometimes, but the power draw shows otherwise. Are you 100% sure that your drives truly are stagger-starting?
To anyone thinking about using these seagates I'd say they are fantastic drives, but be aware that stagger-start may not work. If I could do it all over again I would have spent a little bit more on the enterprise drives since I presume they HAVE to be stagger-start compatible.
Nate
Nate - I don't know what to say… I will make 100% sure that it is working on my end when get home in the morning. I can say that once I set the "Method" to OOB on the controller and set the Staggered Settings to 1 Drive every 4 Seconds without any doubt the controller was kicking on one drive every 4 seconds. I would hear them individually spin up then click once they were up to speed then it would go to the next drive. I didn't see the huge spike in power either.
Keep in mind that I have these drives in the Supermicro Enclosures which have Activity LED's on them. Each time one of the drives was powering on I would see that LED come on and stay on until the drive was up to speed and the heads engaged with the noticeable click then it would move on to the next drive.
I have another system in my network that has Western Digital Drives in it. I haven't tried that one yet but that would be comparing apples to oranges seeing that the drives aren't Seagate and I've been told that the WD's need to have a jumper installed on them to enable Staggered Spinup / Power On Standby.
One of the other systems I've built recently is with a friend of mine. He's got some WD drives as well as the same Seagate 750's. He is going to give it a try as well to see what happens on his. At the same time one of the systems I built recently has a total of 15 drives in it (all Seagate 750's) connected to two controllers. I should have access to that system sometime this week at which time I will check to see if it works on his.
Well definitely sounds like staggered startup is working for you - I'm curious to see how your friends works out. I do not have the LEDs plugged in from the card, but perhaps I should get them up to see exactly how everything is responding. It would be pretty funny if 3ware accidentally made their cards require that jumper to be connected on the card for this to work. Thanks for all your help so far, I wish other people with this setup would find this blog - I couldn't find anyone else on google who had this going on…
Nate
Nate - I do NOT have the LED's connected to the controlled. The LED Activity comes through the Supermicro Enclosures. The RAID Controller LED Connections have not worked for anyone that I know so far so I didn't even bother connecting them. So this would have nothing to do with this working at this point.
It seems that the only differences we have here are the Model of the cards, Motherboard and Enclosures. I suppose it is possible that this could have something to do with the enclosures but I don't believe that the backplane has any intelligence associated with it.
Just to make 100% sure as I said I will power cycle my main server (with the 750's) in a few hours when I get home. I will keep you posted as I get more information on the other systems that others have.
Nate - The Staggered Spin up functionality on my main server has been verified. This morning I rebooted the system, changed the "Method" to ATA-6 then power cycled. When I did this ALL of the drives spun up at the same time. This resulted in a power surge of just under 5 Amps of power which is almost 600 Watts of power. From there I went in to the controller and reset it back to OOB and power cycled again to compare results. This time the drives were spinning up one at a time and the power "Surge" never went past 2 Amps which is about 240 Watts. Just for the heck of it I went back in and changed the settings to spin up 2 drives at a time still with 4 seconds in between. This resulted in about the same power usage and visually the drives were showing me 2 at a time and I could hear them spinning up and the distinctive "Click" when the drive was ready.
I have e-mailed some of the others that I have built systems for to see if they could try this one of them said he will try today. This person has a somewhat similar set up as I do but the following differences. His controller is an old 9500 12 Port and he is using a Standard ASUS Motherboard (does not have any PCI-X Ports, he's just using it in a regular PCI Slot). Again comparing Apples to Oranges in a way but I still want to hear the results.
In my testing of Staggered Spin-up or as some other companies might refer to as "Power On Standby" I opened a case with Seagate on their ST3750640AS 750 SATA Drives. After going back and forth with them trying to get something other than a reference to a Knowledge base article I finally got this answer.
**This drive fully supports staggered spin up but there is nothing that you need to do to "enable" the function on the drive itself. Usually these functions are fully controlled by the controller card at least on our drives. If you have any additional questions, let me know.**
At least this will answer this question but for others it will raise others. I also asked the same question to 3Ware who did the same thing by quoting nothing but KB articles which is why I went to Seagate as well. I did however find the proper "Method Setting" in one of their articles so I guess I did get some answers.
~ Pete
When I open the D drive under my computer on server it freezes for about 15 seconds before it will let me browse the directories (there's only like 10 directories on the D: not like it's loading thousands of entries here). When I went to do an analyze for defrag I saw that data (according to windows) was extremely fragmented. My question is, if I defrag through windows will it even do it appropriately (I don't know if it knows how to handle parity bits), and if not, is there any such of way cleaning up the data on a drive so that it runs more efficiently.
Nate
I'm positive that Windows has zero visibility to the parity bits. I also doubt it will make much difference, since every file is broken into blocks and written across multiple drives. But it definitely won't hurt anything. It might optimize the partition table or something, but I doubt you'll see much difference speed-wise.
Lots of replies and didnt read through them all. But to point something out, there is no need to restart the server into safe mode to extend the partition using diskpart, likewise no reason to restart after this has been completed either. Simply rescan and refresh within disk manager. We do this all the time . Infact I have never restarted a server since nt4 to extend a partition or restart for the new disk space to show up either.
Thanks for the feedback. I think you have to go into safemode if you are expanding a system partition, which was the case for me.
I felt the need to add to this post - as I found this thread to be the most informative on expansion.
I was pretty sure what I was doing, going by the manual and having done it once in the past, but this thread re-assured me as to the procedure.
The once caveat of my adventure, was that I was taking a 6TB array to 9TB (RAID-6)without (and shoot me now) a full backup! So was was terribly afraid (rightfully so).
This is where I have something to add (besides really needing a backup):
My procedure was going fine - as I expanded the RaidSet on the controller - that took a day or so - left it running over the weekend. When I went to expand the VolumeSet, I selected Foreground Priority in the build - thinking that it should get it done faster. I hesitantly pressed verify and go. The progress of the controller showed 67% and slowly counting up - so far, so good.
Then, the defication hit the rotary oscillator!!!
I went to access the drive that the 6-9TB was supposed to house and the data was not there - the DRIVE was not THERE!!!
OH MY FREAKING GOSH !!! WHAT HAVE I DONE !!!
Do I pull the plug, and start recovery now? Do I wail bricks against my head for not having a backup? Do I start telling customers that files from the past 3 years are GONE!
I scoured the web - faintly remembering that I selected foreground tasking. NOTHING!! Nobody seems to have ever selected this.
Back to the manual - for the 7th time. Finally searching on the right terms. There it was:
"• Foreground Availability/Background Initialization
RAID 0 and RAID 1 volume sets can be used immediately after
creation because they do not create parity data. However,
RAID 3, 5 and 6 volume sets must be initialized to generate
parity information. In Backgorund Initialization, the initialization
proceeds as a background task, and the volume set is fully
accessible for system reads and writes. The operating system
can instantly access the newly created arrays without requiring
a reboot and without waiting for initialization to complete.
Furthermore, the volume set is protected against disk failures
while initialing. If using Foreground Initialization, the initialization
process must be completed before the volume set is ready
for system accesses."
To highlight:
"If using Foreground Initialization, the initialization
process must be completed before the volume set is ready
for system accesses."
So I said several prayers in hopes that it means that the volume is COMPLETELY offline until it is done. And then I waited….
…..
……..
…………
about 12 hours later, it came online. (then had to rebuild to a hot spare for a new drive that failed mid initialization - more pucker factor!)
I then ran diskpart.exe - thinking I couldn't screw it up any more!
To my relief - the drive is now 9TB and all the data is there!! AMEN!! Time to clean out my shorts!!
I guess the good news is that it doesn't take days when you choose foreground - but you do take it offline for that whole time.
BTW - my hardware consists of:
SuperMicro Xeon mobo with only one 1.8Ghz HT XEON cpu and 256Mb memory (hey, it's only a file server)
Areca 24port SATA controller - PCI-X
Lian-Lu? V2100 case (I think - 21 drive bays) housing 16 Seagate 750Gb drives - 14 drives in RAID-6 (paranoid RAID-5 with two parity drives instead of one) two dedicated hot spares
The array is not effectively de-fragged and alot faster with it not being so full.
Anyway, Thanks to Carlton on this very nice bit of information on the web to help us get through it all!
Hammy.
Hello,
I was running in a nightmare… let me explain what happened:
- I bought the highpoint 2320 adapter some weeks ago, so I thought it has an actual BIOS
- I used 3 1T HDs to setup a standard RAID 5 with 1 partition. It worked fine.
- Then I added another 1T HD- The integration worked fine, verifiieng also showed no problems. But the resulting disk has now 3T available space and neither Win XP nor Vista 64 can read it. It seems that it has a standard MBR due to the first RAID installation.
- Now I upgraded to the newest BIOS and driver
- But still no access -> same error using Win XP and Vista x64
- I converted the RAID several times but it keeps the MBR, so still no access
- During converting the RAID to a new RAID 5 there is no option to choose VSS now, but the conversation completed without errors. But still no access.
- Unfortunately I can not convert it back to a 3 HD system, it says to less space.
I have now to find a way to access all data to backup it before deleting and creating a new array?
:
1 Is there a way to change the MBR to a working GPT or similar?
2 Can I switch to VSS?
3 Can I use 2 -3 new HDs to create a new Volume with VSS or GPT and then migrate the old RAID into the new one?
Thanks for your help,
Markus
Markus: Sounds like you're having some major growing pains with your RAID array. I think this post [How to Break the 2TB (2 TeraByte) File System Limit] will answer your questions.
I grew very frustrated with RAID for the exact reasons you've mentioned coupled with the fact that a bug in the controller card caused my data to become completely inaccessible/unrecoverable. Because of this, I switched to Windows Home Server and haven't looked back. It makes storage expansion super simple and you don't have to worry about partitions or anything else. UnRAID is another alternative if storage density is more important to you and you're willing to spend a little more time on it.
Good luck!
@Carlton Bale:
Thanks. The problem is that my data is not accessable anymore. Unfortunately the post you mentioned doesnt show an solution. It seems that the highpoint administration tool converted the working 2GB Raid 5 array to an non working 3 GB RAID 5 array both with MBR.
So I am now desperately seeking a way to acess the data again….
Thanks for any help,
Markus
Markus: Thanks for the more detailed explanation. I didn't realize that you had already expanded the partition as well as the actual RAID array itself. In general, it's a two-part process where you use the admin tool to change the size of the RAID Set/Volume Set and the operating system (or third-party utility) to expand the partition. I would think you would only have issues if you tried to modify the operating system partition; the OS should just ignore the additional drive space. Unfortunately, this doesn't seem like the case.
I'm not at all familiar with the highpoint admin tool. First off, I'd recommend calling Highpoint and talking with their support staff. Hopefully they can help you out. If not, you may want to try posting on the StorageReview.com forums, though I didn't find much help there. Finally, you can try to down-size the volume set (not the actual array, just the volume set on it) back to the previous size. If the admin tool can do that, it might make your data accessible.
It's problems like this that caused me to abandon RAID in favor of something more flexible and recoverable, the strong points for Windows Home Server and UnRAID.
Best of luck to you!
Thank you very much for your help. Yes it seems to me that the controller did the expanding but the OS does not get aware of the new situation. I can see 3 partitions in the diskmanager tool. 2 with 2T and a small one in between (some GB). The 1st got a drive letter and is shown in explorer but not accessable. So I believe that it is just the partition information that is wrong due to the MBR…
I already set up a WHS and want to copy the data to it, but therefore I need access…..
Thanks again,
Markus
Hey guys I hope that this is not inappropriate to post here but I thought it might be ok based on the subject matter of this thread. I am upgrading one of my RAID Arrays and wanted to see if anyone was interested in the old HDD's. Here's the info
I am selling a total of Ten (10) slightly used Internal Seagate 750GB SATA II Hard Drives Model ST3750640AS. These drives are currently being used in a RAID Array and are being replaced by larger drives.
As you know Seagate Drives comes with the best in the business 5 year warranty. Each of these drives have approximately 4 years left in their warranties - Expires between 03-22-2012 & 11-07-2012) depending on the individual drive.
As mentioned these drives are currently in service and will be removed in approximately one week. Once they are removed they will be shipped out to whoever purchases them.
I am asking $650 for all Ten Drives and I will throw in CONUS shipping and Pay Pal Charges.
Correct me if I'm wrong, but isn't the data redundancy that windows home server offers basically only a software RAID1. So if you have 1.5TB of data you will need another 1.5TB for it to be backed up in case a hard drive goes bad…. Right?
Yes, Windows Home Server does redundancy through 2 copies of the same file instead of through parity data. It's not RAID 1, but it is similar in data usage. It is less efficient from a storage density standpoint but better from a data recovery standpoint due to no stripped data and independent drives that allow data recovery even with multiple drive failures.
Anyone else have any issues with a 3Ware Controller and the Seagate 1.5TB Drives? This is so odd… I've had a total of 7 Seagate ST31500341AS Drives connected to my 3Ware 9550SX-16ML Controller in a RAID 5 since mid October and haven't had any issues.
Last week I had one of the drives "Time Out" and went off line. I was able to rebuild the array by using the same drive by removing it and reinserting it. Just last night I had yet another drive to the same exact thing. I haven't made any changes to the array whatsoever. No drive added or removed, the amount of data on the Array has increased slowly but I still have 1.9TB Free.
I did a little research and heard that there has been some issues with the drives firmware and there are newer versions out there.
Of course 3Ware says this is a Seagate issues and Seagate says this is a 3Ware issue.
Why would this work for 3 Months without a single hiccup only to have this happen twice in one week. We all know that if the second drive fell off line when the first one was off I would have been SCREWED!!!
Any thoughts???
I'd side w/3ware on this one (for now). Their name is as bulletproof as seagates was (until they started releasing drives >=1TB).
I'd say the reasons you're having the drives "time out" is directly related to the problems the drives themselves are having with timing out while streaming data from them. For instance, if you're even doing something as low intensity as playing an mp3 the song (directly from the drive no raid involved) will stop for 3-10 seconds before resuming play. In RAID configurations (due to how raid works) apparently the problem is amplified since the controllers are thinking that the drives are now offline or unavailable when really there's just some odd delay issue going on with the drives.
Newegg reviews have documented this issue very well, so much that seagate even has a somewhat admittance of guilt - does this sound like it might be you?
ST31500341AS 1.5 TB
Some Seagate Barracuda 7200.11 1.5TB hard drives may show uncharacteristic operation when used with Mac and Linux operating systems in multi-drive configurations. Users may experiences pauses in video streaming applications or a dropped drive from RAID arrays. Customers seeing these symptoms should contact Seagate Technical Support for a firmware upgrade at discsupport@seagate.com or http://www.seagate.com/www/en-us/about/contact_us/.
I'm having the exact same issue with a 1TB seagate drive, but support is currently refusing to admit that the problem is their drive. I'm pretty disappointed w/seagate since between home and work all the drives (except for a raptor 10K) are seagates (so over 30 seagates total) and I've never had an issue with them (until the larger than 750GB models). Mistakes happen I'm fine w/that, but the way they're handling my problem with the drive I have is pretty frustrating.
Nate
I thought I would post an update, Seagate finally responded to the Support Case I opened after about 2 weeks (even though they said it would be 24 hours), they released a new firmware for several different drives of which my 1.5T's were included.
I waited about a week to make sure that they didn't pull down the firmware then applied it to a single drive to make sure that I didn't have any issues. After a few days I upgraded the rest of them and **Knock on Wood** so far so good.
One thing I noticed right off the bat is that the Activity LED on the drives went back to normal. With the original firmware it was opposite, the LED was on when the drive was idle and off when it was inactive. I felt like this was a very good sign.