40Tb Media Server Performance issues

[ashrafi]

read an article stating "The Windows Server® 2008 operating system attempts to align new partitions out of the box, yet disk partition alignment remains a relevant technology for partitions created on prior versions of the Windows® operating system."
reference:http://msdn.microsoft.com/en-us/library/dd758814(v=sql.100).aspx

 

[Rubycon]

areca has 4gb of cache for 24 ports. how much gig's of BBWC can you go up to?

I would be scared you know what if I had to depend on 4GB of WB cache dumping to slow drives. Eventually it will slow way down until it practically gets deadlocked. 15K SAS drives can take that kind of punishment. Desktop drives - even the fastest 7.2K ones - struggle severely. If one adds logical drives or OS partitions to the mix the scenario becomes complicated and performance/reliability is likely to be degraded.

 

[HappyCracker]

Regarding the partition alignment, this was a problem on Windows (and Linux) up to Server 2003. Server 2008 is supposed to have it fixed out of the box. The problem is, the operating system would detect 63 sectors per track, but should be picking up 64 sectors per track. This causes misalignment of partitions and could result in double reads when only one was needed; it's extra work for the drives. The simplest method was to stick a volume of size one sector at the beginning of the drive, then create your data volume after that.

Since you're running 2008, and if your partitions were created in 2008, I would not worry about it.

 

[pjkenned]

I would be scared you know what if I had to depend on 4GB of WB cache dumping to slow drives. Eventually it will slow way down until it practically gets deadlocked. 15K SAS drives can take that kind of punishment. Desktop drives - even the fastest 7.2K ones - struggle severely. If one adds logical drives or OS partitions to the mix the scenario becomes complicated and performance/reliability is likely to be degraded.

Well that works "OK" if you are using as a 1-10 user file server scenario where you upload files once and then are doing sequential reads... so long as you are able to clear the buffer as fast as you are sending data I think you are OK with a fast controller. That setup is not great.

 

[ashrafi]

Yes , that is a relief , i am using 2008 and all partitions are made in 2008.
would somebody enlighten me the optimum allocation size for gpt patitions?
i hear so much about stripe cache size ( i have come to conclusion - larger stripe for big chunks of data and smaller for fast / small chunks of data).
there must be a relation between all this... some set method used to configure optimum raid in a perfect environment!

 

[Rubycon]

Most controllers default to 64KB and that is generally a good place to start. With large sequential transfers 128KB (or larger if you controller allows) would also be good if not better.

 

[ashrafi]

The max on Dell perc6i , i believe would be 512K , and minimum is as low as 8k .
and i think 64-128k would be decent.

 

[classy]

Is the operating sytem and the data seperate?

 

[classy]

Also what kind of data is it?

 

[classy]

I would be scared you know what if I had to depend on 4GB of WB cache dumping to slow drives. Eventually it will slow way down until it practically gets deadlocked. 15K SAS drives can take that kind of punishment. Desktop drives - even the fastest 7.2K ones - struggle severely. If one adds logical drives or OS partitions to the mix the scenario becomes complicated and performance/reliability is likely to be degraded.

Well that would be true, it you are talking smaller number of spindles. With 20 spindles, even with those slow drives, he shouldn't run into problems copying data.

 

[Rubycon]

Well that would be true, it you are talking smaller number of spindles. With 20 spindles, even with those slow drives, he shouldn't run into problems copying data.

Actually I was referring to Emulex's comment regarding 4GB Areca cache which I have experience with. With other controllers this may or may not happen.

These WDC disks were not designed for this kind of application. They are typically found in mybook products or used as a single drive in pcs, etc. Their error / failure rate is alarmingly high from what I've been reading of late.

 

[Emulex]

the cache is used to optimize the read/write nature of raid-5/6.

Most people would do raid-50 or raid-60 with that many spindles and use hot spares.

i have a 12 drive raid-5 and with 512meg of BBWC with 75 write 25% read cache it struggles still. the better solution would be raid-10,50, or 60

 

[classy]

Actually I was referring to Emulex's comment regarding 4GB Areca cache which I have experience with. With other controllers this may or may not happen.

These WDC disks were not designed for this kind of application. They are typically found in mybook products or used as a single drive in pcs, etc. Their error / failure rate is alarmingly high from what I've been reading of late.

Oh I agree. These are not drives for a raid array of this size. But my point is that doing just a copy to the array, with array not serving any other purpose, with that many spindles it should not take 3 days to copy 3T of data. Also quite a few people have touched on possible various problems, but he says it degrades. That cache stripe of 512 is a rather large size, so that s why I asked what kind of files is he copying.

 

[ashrafi]

I have 2 Partitions , One for OS (Raid 1 ) and rest for Data ( Gpt , Raid 6 )
Normally i would transfer Render projects , animation files , High def content , Posters , Videos .
Coming to other question I have to setup one more Server in similar capacity.
This time i wana go right and select something better , but i cannot invest in RE or any Raid class / SAS /Enterprise grade drives.
I Heard consumer grade Hitachi' Deskstar is very RaidFriendly , Need some suggestions.
For Expander I would still use Chenbro , Better recommendation appreciated.
Raid card I am looking at Adaptec 5445.

http://www.adaptec.com/en-us/products/controllers/hardware/sas/performance/sas-5445/

Once I get finalized I will continue to Configuration steps.
Thanks Everyone!

 

[ashrafi]

And .. This server is for Dumping HD content .. It would not be used like a File server , with people trying to retrieve information all the time .. only limited users ( 10 15 ) will have access to copy data to the server. ( one of the assumptions when i compared Green Drives- Performance for Price ).

I have to start working on Second Server( as mentioned in earlier post) would be at forefront for Active Content Access - like a file server relaying 50Gb-250Gb files back and forth .

 

[ViviTheMage]

kind of jumping in the middle of this .. but i'd split your array in two.

10 disks raid 6

10 disks raid 6

or just do a massive raid 10. I think raid 5/6 have a sweet spot aroun d8-10 disks.

 

[classy]

I have 2 Partitions , One for OS (Raid 1 ) and rest for Data ( Gpt , Raid 6 )
Normally i would transfer Render projects , animation files , High def content , Posters , Videos .
Coming to other question I have to setup one more Server in similar capacity.
This time i wana go right and select something better , but i cannot invest in RE or any Raid class / SAS /Enterprise grade drives.
I Heard consumer grade Hitachi' Deskstar is very RaidFriendly , Need some suggestions.
For Expander I would still use Chenbro , Better recommendation appreciated.
Raid card I am looking at Adaptec 5445.

http://www.adaptec.com/en-us/products/controllers/hardware/sas/performance/sas-5445/

Once I get finalized I will continue to Configuration steps.
Thanks Everyone!

I use Adaptecs at work. Two 3805s and one 52445 for a large fileserver. The 52445 has 20, 2TB HITACHI Deskstar 7K2000 HDS722020ALA330 drives. Runs beautifully. I backup to it using Symantec Backup Exec. I back up my virtual environment to it as well. I can write over 1T in backups in about 3 1/2 hours. Their cards work well, but make sure you use only the drives that were tested, make sure they all are exactly the same, same firmware as well. Make sure you intialize fully. Now here is a tip about adaptec cards. Install storage manager to monitor the array. But don't assume your array is healthy by looking at when it says the array is optimal. That just means all the drives are online. You need to check the STATUS of the drives to know whether you array is healthy. Trust me, don't learn the hard way, lol .

 

[ashrafi]

@classy : Did you optimize your raid card or configured raid on default setting ? ( I mean 64k Stripe size , Enable read Cache and son on- Raid Configs )
I believe these Deskstars are very raid friendly(unlike Green WD drives - which are not manufactured for Raid anyway) .
do you have Background consistency check enabled for these drives in Storage manager!

 

[classy]

@classy : Did you optimize your raid card or configured raid on default setting ? ( I mean 64k Stripe size , Enable read Cache and son on- Raid Configs )
I believe these Deskstars are very raid friendly(unlike Green WD drives - which are not manufactured for Raid anyway) .
do you have Background consistency check enabled for these drives in Storage manager!

Ours is a 256K stripe and no the background consistency check is not on. Here is a link to a good pdf on best practices
http://download.adaptec.com/pdfs/mi...aptec_RAID_Maintenance_Best_Practices_v2b.pdf
Its only 6 pages. My suspicion with your setup is drive firmware. They are not really raid drives and the firmware is just not optimized for them to work in an array, let alone one of that size.

 

[ashrafi]

Thanks for the link , Did you update Deskstar's firmware to latest ? I havent checked the Adaptec Website for Harddrive support , but I am assuming this drive should be mentioned in their compatible list of drives.
I am inclined on buying these or new 3Tb for another server build.

 

[ViviTheMage]

Thanks for the link , Did you update Deskstar's firmware to latest ? I havent checked the Adaptec Website for Harddrive support , but I am assuming this drive should be mentioned in their compatible list of drives.
I am inclined on buying these or new 3Tb for another server build.

spend the extra on RE equivalents ... you'll have zero issues.

 

[classy]

Thanks for the link , Did you update Deskstar's firmware to latest ? I havent checked the Adaptec Website for Harddrive support , but I am assuming this drive should be mentioned in their compatible list of drives.
I am inclined on buying these or new 3Tb for another server build.

All the drives had the same firmware. It was done several months ago, so its probably not the latest firmware. I doubt if the 3TB versions have been tested.

 

[classy]

spend the extra on RE equivalents ... you'll have zero issues.

Not a bad point, can't argue this at all.

 

[fleshconsumed]

Here are the recent Tests ,

http://www.flickr.com/photos/45577352@N02/

I'm just curious, am I missing something? Those tests look good, a lot better than 30mbps streaming speed in the first post. Did you do something between when you first posted this thread and ran benchmarks, or are you still getting 30Mbps even though benchmarks say you should have 200MB/s speeds?

 

[Rubycon]

I use Adaptecs at work. Two 3805s and one 52445 for a large fileserver. The 52445 has 20, 2TB HITACHI Deskstar 7K2000 HDS722020ALA330 drives. Runs beautifully. I backup to it using Symantec Backup Exec. I back up my virtual environment to it as well. I can write over 1T in backups in about 3 1/2 hours. Their cards work well, but make sure you use only the drives that were tested, make sure they all are exactly the same, same firmware as well. Make sure you intialize fully. Now here is a tip about adaptec cards. Install storage manager to monitor the array. But don't assume your array is healthy by looking at when it says the array is optimal. That just means all the drives are online. You need to check the STATUS of the drives to know whether you array is healthy. Trust me, don't learn the hard way, lol .

Hitachi drives work awesome in multi user setups IME.
Got a bunch of Seagate Constellation 2TB SAS drives to try out next. (crosses fingers)