Reading dd perf test result Q

[Sequences]

I recently setup a freenas instance on a laptop to play around with. I have two of the following drives in RAIDZ0 configuration: http://www.amazon.com/gp/product/B00...?ie=UTF8&psc=1

I ran the following command to test write performance: dd if=/dev/zero of=temp.dat bs=xk count=25k, where bs=xk went from 4k to 1024k. What I think this is doing is writing 25k blocks to a file temp.dat where each block is xk large. Is this correct?

My results, all in bytes/sec:
4k: 231,643,525
8k: 405,123,077
16k: 621,108,792
32k: 850,625,719
64k: 894,870,764
128k: 1,110,215,931
256k: 1,069,449,486
1024k: 998,503,067

I just want to make sure I am reading this correctly: at around 128k per block, this setup can write at 1.1GB/s sequentially?

 

[dave_the_nerd]

The command looks right. The results are flatly impossible, so it's probably just going, "hey, this jerk is writing zeroes - let's just pretend we're done so he can go play Candy Crush."

 

[Cerb]

You're testing how quickly the system can read from /dev/zero, , fill up buffers for a file, and sync those buffers to the contents of the file system for the drive's mount point. The drive's performance may not show up as a significant part of the results until you've written many GBs, that way. The drive only matters once the buffers are large enough, old enough, or take up too much memory, causing the OS to flush them. Unless mounted sync, that can be done while writing to the in-memory buffers, and the results, even if mounted sync, will vary by filesystem. ZFS may complicate testing and results, too.

Use a tool made to test drives, like Iometer, to have any hope of getting realistic results. You will probably find the peak sequential rate to be around 250MBps.

 

[Sequences]

The command looks right. The results are flatly impossible, so it's probably just going, "hey, this jerk is writing zeroes - let's just pretend we're done so he can go play Candy Crush."

They didn't look right to me, hence this sanity check post. Thanks for the check.

You're testing how quickly the system can read from /dev/zero, , fill up buffers for a file, and sync those buffers to the contents of the file system for the drive's mount point. The drive's performance may not show up as a significant part of the results until you've written many GBs, that way. The drive only matters once the buffers are large enough, old enough, or take up too much memory, causing the OS to flush them. Unless mounted sync, that can be done while writing to the in-memory buffers, and the results, even if mounted sync, will vary by filesystem. ZFS may complicate testing and results, too.

Use a tool made to test drives, like Iometer, to have any hope of getting realistic results. You will probably find the peak sequential rate to be around 250MBps.

I ran it again with higher counts and the numbers stayed consistent. I take it from your post that this would be a much better test?

What I want to accomplish is getting some benchmark numbers so that later on I have something to compare against. Ideally, this will become part of my diagnostic toolset for maintaining my setup. I see that you mentioned iometer. I guess I will need to compile it to use it?

 

[dave_the_nerd]

I ran it again with higher counts and the numbers stayed consistent. I take it from your post that this would be a much better test?

What I want to accomplish is getting some benchmark numbers so that later on I have something to compare against. Ideally, this will become part of my diagnostic toolset for maintaining my setup. I see that you mentioned iometer. I guess I will need to compile it to use it?

Yeah, you'd be installing it from the local ports directory, probably.

http://olddoc.freenas.org/index.php/Installing_non-PBI_Software

Once you've installed Dynamo (the I/O engine) you connect to it with the IOMeter GUI from a windows machine.

If you're just looking for sequential numbers, you can also generate a test file (do your dd command again but if=/dev/random) and and move that around. Random data won't be easy to compress/skip/cache.

 

[Cerb]

I ran it again with higher counts and the numbers stayed consistent. I take it from your post that this would be a much better test?

The problem is that just writing data from memory to a file makes great utilization of the OS and FS implementation, and not so much the drive. If you have compression on in ZFS, FI, writing zeroes should never appear to get limited by the HDDs (I don't know for sure, but ZFS may have special treatment for all-0 blocks, too). Almost all the work that is going on is going in your RAM, then only the finalized result goes to disk. The speed differences you see are mostly the overhead of the OS managing memory outside the file system, and file system synchronizing data in memory. You need to run something that requests syncs, flushes, and/or direct access, to get around that, to see what your performance might be, if you start getting limited in IOPS.

P.S. Also check out the fio project.

I would expect the RAID 0 to max out around 250MBps and 300 IOPS, with such tests.