To me it seems rather simple, although maybe someone can explain where I'm misunderstanding.
You can look at *any* SSD benchmark, it's it's clear that sequential reads are much faster than random. Obviously there's a major difference between a random read and a fragmented file, but regardless, there's a difference there.
I fully understand that its orders of magnitude less of a problem on a SSD, but theoretically, fragmentation leads to lower performance, no?
Computer: Controller, give me blocks 44245 - 44827 please.
SSD: Okay, here you go. *BLEEAARGH*
Computer: Controller, give me blocks 1484 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 44827 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 11295 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 90210 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 49583 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 60601 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 48108 please.
SSD: Okay, here you go. *pfft*
Computer: Controller, give me blocks 10201 please.
SSD: Okay, here you go. *pfft*
HDD's map block numbers too. They have a small reserve of blocks that they can take into use (remap) when original blocks become unusable. After that some blocks in the sequence are physically less "near" than before. Overall, HDD's do less mapping and such remap is more drastic to performance (particularly, because the reason to remap, a "bad block", might mean that more will follow.)Your computer gives everything a block number. On an HDD, those correspond to physical locations on a platter. (Which is why mechanical optimizations like defragging are helpful.)
The SSD shows the block numbers to the computer, and then maps each block number to a particular page of flash memory.
mikeymikec, sequential reads are up to 5 times faster than random reads.
I suggest you read my post further up, to understand what the difference is between a fragmented file system, and a defragmented file system.
At this point, the drive geometry matters much less, than for HDDs, but the FS virtual geometry still matters, as long as we linearly address the SSD (never mind how it translates those addresses - this is almost irrelevant, except when writing a FS defragmenter specifically for SSDs, where this internal address translation can be exlpoited to defrag with O(n) (n = number of files) writes to flash outside the addressing table.)
They're completely different things.
Sequential access:
Random access:
HDD's map block numbers too. They have a small reserve of blocks that they can take into use (remap) when original blocks become unusable. After that some blocks in the sequence are physically less "near" than before. Overall, HDD's do less mapping and such remap is more drastic to performance (particularly, because the reason to remap, a "bad block", might mean that more will follow.)
Regarding the sequential/random on SSD.
Prepare multiple SSD's so that each seem to contain an non-fragmented file and perform the sequential access test on them. The 'prepare' step is that each drive's controller must map the blocks of the file differently; sequential in same memory chip, striped over lines, sequential but in reverse, random, etc. Now the test truly measures the effect of the mapping within the drive and nothing else.
I don't defrag SSD's.
Ok, so where does fragmentation fit into that?
That's the thing. On an SSD, it doesn't.
On an HDD, the *BLEARGH* becomes a bunch of *pfft*s because the HD has to go here, then go here, then go there, then go there, to get all the pieces of the file. So that's why everybody has spent the last 30 years pissing themselves about defraying their stuff.
You're mistaking controller latency for fragmentation behavior.
As for sequential vs linear from reading a single file: I assume the difference comes through read-look-ahead, which is described here: https://www.mail-archive.com/forum@t13.org/msg02556.html (for the case of mechanical drives, where it's even more of a crucial function)
This would make a seek (i.e. moving to a non-sequential LBA) clean the buffer and restart the read look-ahead at the new position. The performance gains are simply from the controller already having the required block in the buffer, before the OS's command to read it actually arrives over SATA.
Let's try explaining this another way.
A hard disk's method of retrieving files is physical / manual. A head has to be physically moved to the correct location on the platter (as well as the platter being in the right position for the head to read the data), kind of like a librarian sitting in a library. Ask the librarian to find a book and they'll walk to the correct location, pick the book up, come back to the front desk and give it to you.
Now let's say there's a big file, or in the librarian example, you're asking for the entire series of a particular encyclopedia. On a recently defragmented disk, or in a well-organised library, the contents of that encyclopedia (or large file) will be located in the same place (continuous block), so the librarian can just go to one location in the library with a trolley and pick up all the volumes of that encyclopedia in one go (and so the head does one data seek, finds it and reads it), rather than having to walk around the library because the library is untidy (or disk is fragmented, so the head is having to make lots of journeys). As for the disk, its read speed goes way up for a large file because it doesn't have to spend time seeking for bits all over the platter.
Defragmentation (or tidying the library) is good so that time isn't wasted seeking all the bits of a large file.
SSDs do not have a drive head. There is no movement inside the SSD. If a file has been splurged in a thousand fragments across the SSD, the retrieval time is the same as if the file was saved in a single fragment; the SSD still has to request the portions of the file in just the same way.
Defragmenting an SSD is just as pointless as hypothetically defragmenting RAM or the cache levels in the processor, except with an SSD there is a limited amount of writes it can do in its lifetime, and you're wasting those writes in an utterly useless exercise.
Isn't that inconsistent with the idea that a sequential read is a single command? Isn't look ahead there for the purpose of buffering blocks that haven't been requested? I don't see how that plays into a scenario where it's already been requested.
Its both a hardware issue and a software issue.
Hardware
--------
By requesting each block individually you swamp the cable with individual requests, and there is a lot of latency in the SATA protocol that mean that isn't brilliantly efficient. Its much more efficient to do a nice big DMA transfer as the channel can just send data. If you go have a detailed read of the SATA protocol you'll see there is quite a difference with how individual block verses sequential reads are setup.
Software
---------
The program that is measuring the speed has to talk through the operating system. To do a sequential read it sets up an InputStream with the operating system and then proceeds to read from the same stream continuously, simply just calling next with the same buffer over and over, probably something like 4k although maybe its megabytes in size.
However for a 4k read gets a random access file and then does seeks before calling reads with a smaller 4k buffer. So it seeks then reads and seeks and reads and has to keep going through that cycle all the way through the operating system.
The combination of the two dramatically decreases performance, and you can see a dramatic uptick with a queue depth of 32 which is effective making lots of calls in parallel.
And yet for the same reason that people buy fast ram, I'd like to keep my SSD operating as close as possible to it's theoretical max performance. I'm not particularly concerned about SSD life, I think it's a problem that's blown way out of proportion. A defrag every now and then isn't going to destroy it.
Honestly, it's more of a theoretical thing, more of a "why not?" kind of thing, not something to obsess over.
I wonder how much of the 4k latency hits inside the SSD, and how much outside.
I suspect that some of the remaining performance difference even when "spamming" the bus with parallel requests (i.e. high qd) comes from the cache size of the controller (talking L2 equivalent here). Since if the controller has to send for the LBA-to-local address mapping from DRAM or flash, there'll be another delay in the chain. I also wonder how much SRAM these controllers generally carry on board, and whether anyone does anything else but linearly pre-streaming the next LBA in a classic read-ahead scenario.
But yeah, DMA is definitely where a lot of the magic happens when you want big throughput. When the SATA controller can write directly into an assigned memory space, and just waits for the data from the disk, things are as good as they'll get. I'd expect a fragmented file would open one DMA "session" per fragment - the FS shouldn't fall back to block-operation unless specifically asked for -- I'd imagine this is only important for synchronized concurrent R/W access (yup, DBs, unbuffered/synced logs, that kind of thing).
Of course, a good I/O subsystem will try to get a lot of reads from cache, but that only helps to read non-dirty LBA's that hit the I/O subsys frequently enough.
I think the gist of it remains: don't defragment SSDs; but if you're aware of fragmentation negatively impacting you, design around it, using partitions, placeholders or independent disks. Workloads which could lead to dramatic fragmentation are imaginable (and therefore, probably real...)
To what extent can a SSD read in parallel? Leaving the bus out of it, just straight to it's internal buffer. Can it access each NAND chip at the same time, maybe even multiple parts of the same chip at the same time? Or is it ultimately still a serial device like a HDD?
Go into the Properties page of the SSD volume. Make sure "scheduled Defragmentation" is turned off.