Raid Controller for SSD's

[jmelgaard]

Because I am slowly running out of space on my old 2 x 120 GB Vertex 2 SSD's I am looking into maybe switching to 2 x 240 GB Vertex 3 instead and use the old ones elsewhere.

However the old ones are raiding using an on-board Intel raid controller which is ofc. SATA II, so I was thinking of moving to a dedicated hardware raid controller.

If you ask "Why, what do you need the bandwidth/performance for?" the only answer I can provide you with is "Because I can".

The thing is, what dedicated controller should one choose for this, I am familiar with the LSI MegaRaid controllers as I am using an LSI MegaRAID SAS 9260-8i in my ESXi server, but I am thinking that is maybe just a little overkill for only connecting 2 SSD drives, so might as well save a bit there since the price tag for this is obviously already going to be quite high, not that that is a major problem, but was just thinking of finding a "non-sas" controller.

Obviously I need to be able to boot from it.

 

[yinan]

Eventually you will need more space so you might as well just go for overkill like I did and get the Areca 1880 it is speedy.

 

[jmelgaard]

Eventually you will need more space so you might as well just go for overkill like I did and get the Areca 1880 it is speedy.

Seems very similar to the one I mentioned from LSI (which after digging a bit more is not so odd since they are both based on the LSI2108 ROC chip it seems), and I can't get any Areca products from local dealers.

While it is true that I will eventually need more space, it's unlikely that I will ever get to fully utilize what a 8 port card offers, after all I have a 14TB NAS on the side so it is mostly due to the first installation taking up allot of application space.

Since then I have really only increased the usage with about 50GB over the past year, so looking at MegaRAID SAS 9260-4i as an alternative (and since I can't find any dealers locally for any Areca products), and even that is a bit overkill. (So still searching for alternatives, therefore this post)

Chances are that before I need more space, new generations of SSD's has arrived and it will be better to simply replace them rather than add more to an existing array.

 

[mfenn]

The big issue with using RAID controllers with SSDs is to make sure that the controller itself can sustain enough IOPs to keep up with the drives. I'd actually look into the 9265-8i because it can sustain 200K IOPS versus 66K IOPS for the 9260 series.

 

[jmelgaard]

The big issue with using RAID controllers with SSDs is to make sure that the controller itself can sustain enough IOPs to keep up with the drives. I'd actually look into the 9265-8i because it can sustain 200K IOPS versus 66K IOPS for the 9260 series.

Yes I have read as much, but as far as I could understand, FastPath eliminates much of that issue.

http://thessdreview.com/our-reviews...rage-manager-software-fastpath-and-cachecade/

That just brings the next problem which is that I can get the LSI cards, but as for the key that unlocks those premium things, I have yet to find a dealer here in Denmark, but depending on the price I may trust such a small component enough to order it overseas from e.g. UK.

The 9265-8i is absolutely better but so far unavailable, and I haven't seen anything indicating that it is on it's way to DK... more sadly it does not seem to come in a cheaper 4 port version as the 9260 does (have not figured if that also support the premium unlocker), however I may just have to eat the price tag and say to hell with it...

I still hope to get some good alternatives though, which hopefully also available in the market here, really not many brands here for Raid controllers when it comes down to it.

 

[mfenn]

I still hope to get some good alternatives though, which hopefully also available in the market here, really not many brands here for Raid controllers when it comes down to it.

That is true, and there are even fewer companies who make the actual IO controllers.

 

[jmelgaard]

That is true, and there are even fewer companies who make the actual IO controllers.

Ok so ended up going with the 9265-8i since I actually found a local dealer, they just listed under the LSI store part number (LSI00278) for some reason.

http://store.lsi.com/index.cfm?category=18&subcategory=47

But I wrote to them and verified it was that controller, and then ended up being crazy enough to go with 3 x 240GB Vertex 3 >.<... (SO MUCH FOR NOT GOING OVERKILL >.<)

They had to order the controller, so it will be 2 days before they have that in stock and I am guessing another 2-3 days before I then have it...

Now I just hope I can restore my windows installation onto it, would hate to reinstall completely since it would properly be wise.

 

[somethingsketchy]

If you are changing controllers (i.e. going from on-board RAID to now a discrete RAID card), you would be better off just reinstalling Windows.

Chances are the both the LSI controller and the motherboard controller are two different manufacturers, and they may not be all that compatible.

Plus it'll be a fresh start with the new 240GB Vertexes on a dedicated controller.

 

[jmelgaard]

If you are changing controllers (i.e. going from on-board RAID to now a discrete RAID card), you would be better off just reinstalling Windows.

Chances are the both the LSI controller and the motherboard controller are two different manufacturers, and they may not be all that compatible.

Plus it'll be a fresh start with the new 240GB Vertexes on a dedicated controller.

I think you misunderstood that a bit, we are not talking "moving the disks" but using Image backups like Norton Ghost, Acronis or the built in system image for Windows 7. (I Have the ultimate edition).

So the Raid array won't be maintained, the partitions will however. I did the same thing when moving from a single Intel 160GB drive to my 2 Vertex 120GB drives with no problems (Single disk to Raid volume, same controller), but I am more concerned if the "restore" utility will recognize the volume on the LSI controller with no issues.

Considering that I am running out of 2x120GB in installations, it is rather much to reinstall everything, and would potentially take a long time.
Fresh starts are always good, but it might not be worth the time.

 

[mfenn]

I think you misunderstood that a bit, we are not talking "moving the disks" but using Image backups like Norton Ghost, Acronis or the built in system image for Windows 7. (I Have the ultimate edition).

So the Raid array won't be maintained, the partitions will however. I did the same thing when moving from a single Intel 160GB drive to my 2 Vertex 120GB drives with no problems (Single disk to Raid volume, same controller), but I am more concerned if the "restore" utility will recognize the volume on the LSI controller with no issues.

Considering that I am running out of 2x120GB in installations, it is rather much to reinstall everything, and would potentially take a long time.
Fresh starts are always good, but it might not be worth the time.

He was probably actually referring to the fact that when Windows often refuses to start when you change the system drive (or LUN) to one that requires a different controller. Acronis has a "universal restore" capability in their Enterprise products that works around this.

 

[jmelgaard]

He was probably actually referring to the fact that when Windows often refuses to start when you change the system drive (or LUN) to one that requires a different controller. Acronis has a "universal restore" capability in their Enterprise products that works around this.

Hmmm.... I Do have an Intel Disk, maybe I should try there Disk Data Migration tool as it is provided by Acronis, it really made the swap at work extremely easy, and as far as the statement, all it says is that it needs a Intel disk in the system...

Would also be far more easy, so will properly proceed with that during the weekend.

For now I need to evaluate the Tweaking, if there is anything to teak at all, Random writes did not really improve over the old disks, however ATTO rapports sequential reads around 1.5GB/s and AS SSD around 1.25GB/s...

AS SSD puts the IOPs up and around 144.000 IOPS the the best outcome:

 

[mfenn]

For now I need to evaluate the Tweaking, if there is anything to teak at all, Random writes did not really improve over the old disks, however ATTO rapports sequential reads around 1.5GB/s and AS SSD around 1.25GB/s...

You've got to really hit them with a high queue depth in order to see an improvement. Otherwise the bottleneck is in the IO stack and not the drive. Your 64-thread read numbers are right where they should be and your write numbers are just a little low.

AS SSD puts the IOPs up and around 144.000 IOPS the the best outcome:

That's really damn good for a two drive array (33&#37; above spec).

So in short, the drives are performing above spec everywhere except for high queue depth random write, where they're only a little low.

EDIT: I just noticed that your drive shows as 670GB. Did you put the Vertex 2 drive in the same array as the Vertex 3? If so, I'm not surprised that the Vetex 2's contributions are next to nil.

 

[jmelgaard]

You've got to really hit them with a high queue depth in order to see an improvement. Otherwise the bottleneck is in the IO stack and not the drive. Your 64-thread read numbers are right where they should be and your write numbers are just a little low.

Ill check that out the QD when i get home (meaning making some tests with higher QD),

EDIT: I just noticed that your drive shows as 670GB. Did you put the Vertex 2 drive in the same array as the Vertex 3? If so, I'm not surprised that the Vetex 2's contributions are next to nil.

No I guess you didn't keep track of the entire post , I ended up ordering 3 x 240GB Vertex 3 >.<... (So much for not going overkill heh)...

In any case I would never expect the performance gain to be linear, but question is how much one could actually expect.

 

[DreamWarrior666]

Have they found a way around the issues with RAID + SSD and no TRIM? I suppose given the lack of TRIM in RAID is a driving factor to picking a Sandforce solution? Is this a real concern? Just curious....

 

[jmelgaard]

Have they found a way around the issues with RAID + SSD and no TRIM? I suppose given the lack of TRIM in RAID is a driving factor to picking a Sandforce solution? Is this a real concern? Just curious....

With the new upcoming RevoDrive 3 series it looks like we are moving a step closer meaning that OCZ is pushing TRIM support into those cards, and since they are essentially just raided SSD disks then I guess trim for raids is a step closer.

That said it may take long before companies like LSI etc. will incorporate it into their products, yet they are beginning to focus allot on SSDs at their end, the LSI 9265-8i is as an example specially designed to leverage the high amount of IOPS SSDs can deliver.

But AFAIK it does not support TRIM. So for now a drive with ITGC or similar is to preferable.

 

[DreamWarrior666]

With the new upcoming RevoDrive 3 series it looks like we are moving a step closer meaning that OCZ is pushing TRIM support into those cards, and since they are essentially just raided SSD disks then I guess trim for raids is a step closer.

That said it may take long before companies like LSI etc. will incorporate it into their products, yet they are beginning to focus allot on SSDs at their end, the LSI 9265-8i is as an example specially designed to leverage the high amount of IOPS SSDs can deliver.

But AFAIK it does not support TRIM. So for now a drive with ITGC or similar is to preferable.

Funny OCZ should be pushing TRIM since their solution is less prone to require it, though it still helps them. Should be interesting to see it come along.

 

[mfenn]

Ill check that out the QD when i get home (meaning making some tests with higher QD),



No I guess you didn't keep track of the entire post , I ended up ordering 3 x 240GB Vertex 3 >.<... (So much for not going overkill heh)...

In any case I would never expect the performance gain to be linear, but question is how much one could actually expect.

Ah my bad, I see it now. In that case, I'd say that your numbers still look pretty good.

Your sequential read numbers are right at sped, and your sequential write numbers right where you would expect for random data. Remember that Sandforce gets its high write speeds by compressing the data, so by interleaving data (RAID 0), you are making the data more random.

Your random read numbers are still off the charts, making me think that the controller is helping you out a log with cached data. The only thing that is a little low is your random write (66&#37; of spec), but I personally wouldn't be too worried about that because you could be running into bottlenecks elsewhere (80K IOPS is a lot to be generated on the fly).

EDIT: As for TRIM, yeah that is an issue, but there isn't too much you can do about it. Sandforce's internal garbage collection is pretty good.

 

[jmelgaard]

Your random read numbers are still off the charts, making me think that the controller is helping you out a log with cached data. The only thing that is a little low is your random write (66% of spec), but I personally wouldn't be too worried about that because you could be running into bottlenecks elsewhere (80K IOPS is a lot to be generated on the fly).

Obviously the cache helps out, but for AS SSD I don't see an option to increase the area it spans over, I am not sure how it works but I assume it creates one big file and then accesses random locations in that file for 4KB of data?...

In any case, with 1GB Cache on the card it would require a rather big file in order to make a fair attempt to eliminate that I guess...

The settings is just:
- Read Policy: No Read Ahead.
- Write Policy: Write Through
- IO Policy: Direct IO
- Access Policy: Read Write
- Disk Cache Policy: Disabled
- Strip size: 64KB

Think thats what there is to mention.

 

[mfenn]

Obviously the cache helps out, but for AS SSD I don't see an option to increase the area it spans over, I am not sure how it works but I assume it creates one big file and then accesses random locations in that file for 4KB of data?...

In any case, with 1GB Cache on the card it would require a rather big file in order to make a fair attempt to eliminate that I guess...

The settings is just:
- Read Policy: No Read Ahead.
- Write Policy: Write Through
- IO Policy: Direct IO
- Access Policy: Read Write
- Disk Cache Policy: Disabled
- Strip size: 64KB

Think thats what there is to mention.

You are correct about the way AS-SSD (and all filesystem level) benchmarks work. You can try running CrystalDiskMark as well which lets you set the file size to test.

 

[jmelgaard]

So I Ran a set of tests again, and I am thinking that I must have changed some settings since the benchmarks I posted here because AS SSD turns even better results.

Just for reference:
- Sinlge drive review from here: http://www.anandtech.com/show/4316/ocz-vertex-3-240gb-review
- Specs from OCZ + AS SSD data: http://www.ocztechnology.com/res/manuals/OCZ_Vertex3_Product_sheet.pdf

AS SSD Benchmarks.

Did the test several times with close to same results, looking at the spec sheet above it seems they have also achieved much higher results with AS SSD than what they have put in the specs... And I have the cache added to that.

Crystal Disk Mark gives some indication about the cache helping, however the results is hard to interpret fully. If I do a full test, the numbers are clearly lower for a 4GB file, but if I only do a 4K random at QD32 run they are the same.

In any case it shows something, the full run on 4GB was even one of the lower results, for all the runs i did it was somewhere between 350 and 380 MB/s...

I have written in the iops for the QD32 test.

Crystal Disk Mark - 1GB File.

Crystal Disk Mark - 4GB File.

For throughput ATTO seems to achieve higher results that AS SSD and CDM, so just added a single Screen for that as well...

In any case I doubt ill have much issues with IO speeds anymore, not that I had that on 2 x Vertex 2 drives anyways but hey ...

 

[mfenn]

Those test results look really good to me! Enjoy