Extreme Overprovisioning

[Belial88]

Z87X-UD3H
8GB 2666 CL11 hynix CFR
i7-4770k
w7 Ultimate x64
samsung 830 128gb

Basic question - overprovisioning increases performance (and/or reduces wear), as I understand. So why not overprovision a significant portion of your SSD?

Like if you are using 50% of your drive, why not overprovision to 51%? Because of the inconvenience of having to change that overprovision every time you added (or lost) data? So is that an issue of extremely marginal performance increase - but a performance increase nonetheless - and convenience? And even then, why not just overprovision on your 50% used drive, 55-60% (and maybe only have to change it anytime you installed a huge program, so once every few months or so).

That's really all there is to it. The rest of it is just me explaining myself so I avoid getting dumb 'you won't notice the performance boost' responses that don't answer the question at all.

I am acutely aware that these performance gains are insanely insignificant, but that's not the question here, so please no responses like that.

I've used only 40-50GB on my systems in the last ~4 years. I only play a single game (i play at a competitive level, and can say with confidence it'll be the only game I play for at least another 5 years). And that's including a 10% overprovision and 8GB reserved for Intel Rapid Start. I also regularly reinstall my OS, I'm very good about deleting stuff when done (i hate a downloads folder with things in it), and even when I do download movies, music, etc, they go somewhere else.

In about 3 years I will be adding on ~8GB for an expansion, but otherwise I will not be installing anything major anytime soon, and I haven't in the last 4 years. So, since this is how I am as a user, why not set my overprovision to ~50gb, ie over 50%?

again, before i get dumb responses about marginal returns, I am well aware of that. If it's a free performance boost, even if it's infinitesimally small, there is zero cost so why not do it if I, as a user, will never use more than 40-50GB (and in 3 years i can shrink it or i can leave ~10gb un-overprovisioned to leave room for that expansion).

Thanks. It's really a simple question, I just know I'm going to get a ton of answers saying things like it's an unnoticeable performance increase or what if you have to add more space in the future, to which I'll say it's free, and i can account for that as well as definitively say I will not be needing more space in the future.

 

[Cerb]

Basic question - overprovisioning increases performance (and/or reduces wear), as I understand. So why not overprovision a significant portion of your SSD?

Because Windows 7+ supports TRIM.

With Vista (depending on storage driver), XP, Linux (except maybe EXT4+discard), FreeBSD, or use of any number of RAID options, may remove the ability to use TRIM, at which point you'll want to add more OP, instead.

TRIM is hardly perfect, but it effectively makes the free space on your file system equivalent to added over-provisioned space. If you want to add OP instead, go right ahead (it will generally lead to better performance consistency, if nothing else). But, TRIM is why hardly anybody bothers.

 

[Mushkins]

Because it's not a free performance boost, and you're not a typical use-case. Like you said, every time you add/delete/etc data from the drive you'd need to recalculate and reconfigure the OP. That time and effort is worth something to most people, typically more than the already agreed unnoticeable gains from redoing that extreme level of OP practically on a daily basis.

If you want to overprovision to that extent because it fits with what you use the system for and you want every teeny tiny bit of performance, nobodys gonna say "don't do it." But to answer the original question, most people don't bother even considering that level of OP because it's just not worth the time or effort for 99% of people, and that last 1% is even arguable. You'd gain more performance by simply buying a better/faster SSD even with its default configuration, unless you're already at the absolute top of the list, but even then something new is always just a few months away. Not to mention that in order to do it, you're giving up usable space, which is already the biggest premium for an SSD.

 

[BrightCandle]

It largely depends on how much data you tend to write and remove constantly and how full the drive will typically be. If most of the files are static you'll see no benefit at all. If the drive is filled at full speed, then emptied and that is done over and over then overprovisioning to 50% would show a more dramatic impact. Its really dependent on the usage pattern of the disk and on the nature of the data you store on it and for how long.

 

[GlacierFreeze]

Yep, waste of time.

 

[imagoon]

The only time over-provisioning an SSD makes sense is when the writes exceed the slack space on the drive between GC cycles. IE due to the 256 binary / decimal thing you generally end up around 7% slack so on a 256gig drive you would need to write 17.92 gig at 100% of the drives write capacity (since at any pause the drive can kick of some GC processes) to see performance loss.

Also since changing the provisioning on drives is a destructive process, most people don't mess with it to avoid disk wipes.

 

[Belial88]

I dont see it as a waste of time. It takes 20 seconds to set up a partition of one size or shrink/grow it...

I have a 128gb SSD, I've used 40GB in 3 years, I'll use an extra 8GB over the course of ~3 years, so I'll set up my partition so I have 40/60GB usable space.

I don't see the problem. If you aren't going to answer the question, butt out. So I'll set up a max provision then, the performance gain is negligible and further still because of TRIM on newer SSDs, but still there.

Also since changing the provisioning on drives is a destructive process, most people don't mess with it to avoid disk wipes.

So changing the partitions over and over might potentially cause a total drive wipe? So any time you change a partition, you are risking total drive wipe?

 

[tidy]

.

 

[mrpiggy]

Extreme overprovisioning is only a performance or longevity/reliability boost if certain conditions are met. You aren't meeting those conditions. Put it this way. You are basically saying "I have a Ferrari that can do 200mph, but I only drive 55mph max no matter what. How much performance is using racing grade gasoline and different Z-rated racing tires going to give me?"

It's not with your stated usage. Sure you can argue that "theoretically" the gas and tires are going to increase the performance of your Ferrari at 55mph. So what? Just because you don't want people to waste your time with "you won't notice the performance boost' responses that don't answer the question at all." doesn't mean the answer is wrong when it's your usage parameters and question that's stupid in the first place.

 

[imagoon]

I dont see it as a waste of time. It takes 20 seconds to set up a partition of one size or shrink/grow it...

I have a 128gb SSD, I've used 40GB in 3 years, I'll use an extra 8GB over the course of ~3 years, so I'll set up my partition so I have 40/60GB usable space.

I don't see the problem. If you aren't going to answer the question, butt out. So I'll set up a max provision then, the performance gain is negligible and further still because of TRIM on newer SSDs, but still there.



So changing the partitions over and over might potentially cause a total drive wipe? So any time you change a partition, you are risking total drive wipe?

Over provisioning has nothing to do with partitions. You have to use the drives tools to change the ratio. The drive will then report itself as physically smaller to the computer. The blank space outside a partition is still valid data space as far as the SSD is concerned and it will treat it as such.

IE 256GB drive -> 50% slack via the drive tools -> appears to the computer as 128GB SSD.

 

[Phynaz]

I just know I'm going to get a ton of answers saying things like it's an unnoticeable performance increase or what if you have to add more space in the future, to which I'll say it's free, and i can account for that as well as definitively say I will not be needing more space in the future.

Why bother asking if you already know the answer?

 

[Belial88]

Because I did not know the answer. I did not know definitively if there was actually a performance boost with extreme over-provisioning, or, if there was some 10-25% performance sweet spot. Most of the articles on line say "Use 25%" or "Use X%" but don't really say if more was better or if they even tested more. I also didn't know if it still held true today - which it seems it holds less true but still true.

So I learned something. Took a lot of bad posts, but I still learned it. Hence, I will set up my extreme overprovision.

Extreme overprovisioning is only a performance or longevity/reliability boost if certain conditions are met. You aren't meeting those conditions. Put it this way. You are basically saying "I have a Ferrari that can do 200mph, but I only drive 55mph max no matter what. How much performance is using racing grade gasoline and different Z-rated racing tires going to give me?"

It's not with your stated usage. Sure you can argue that "theoretically" the gas and tires are going to increase the performance of your Ferrari at 55mph. So what? Just because you don't want people to waste your time with "you won't notice the performance boost' responses that don't answer the question at all." doesn't mean the answer is wrong when it's your usage parameters and question that's stupid in the first place.

I don't think that's a fair analogy. I think it would be more like if the gas and tires were completely free, and my question was "Hey I have a 200mph Ferrari, and I also have a choice of either Z-rated racing tires or basic tires with no cost difference between either, and the racing grade gas in my area is the same price as normal gas, shouldn't I use the racing grade gas because it'll be better for the engine over the long term, and shouldn't I use the racing tires because while 99.9% of the time they won't make a difference, they might make a difference on a 55mph hairpin turn while entering the highway on a rainy day?'

That's the point. 70GB of my SSD will be unused, forever unused, and another 10gb will take about 3 years to be used. So why not set the OP for that 70GB at least?

Over provisioning has nothing to do with partitions. You have to use the drives tools to change the ratio. The drive will then report itself as physically smaller to the computer. The blank space outside a partition is still valid data space as far as the SSD is concerned and it will treat it as such.

IE 256GB drive -> 50% slack via the drive tools -> appears to the computer as 128GB SSD.

I'm not sure I understand. I thought overprovisioning was just creating an unallocated space partition? I'm not sure I understand your post here.

 

[kmmatney]

The thing is, I don't think there is ANY gain at all, even a small one, for your usage. Here is a quote from the Samsung web site:

"ETC: What Is Over Provisioning?

Over Provisioning refers to the practice of leaving a chunk of free space available for use by the SSD controller. The SSD will perform better if it has free space available to use as swap space. Swap space is used to perform routine SSD maintenance (TRIM and Garbage Collection) in the background during idle time, allowing the SSD Controller to prepare free blocks for the OS to use in the future. Because the SSD performs best when writing to free blocks, the result is a better user experience through reduced wait time"

All the OP space is used for is a swap space during TRIM and garbage collection. If you are filling up your drive, then you can probably use the extra swap space. However, since you are only using a fraction of the space, the existing swap space should be plenty.

The Anandtech reviews show small gains with extra OP, but those tests are writing huge amounts of data. Looks at the consistency graphs in any Anandtech review - the X-axis is in time (seconds) and it goes out to 2000 seconds (33 minutes)! In your usage, I doubt you ever write to the drive for more than a few minutes at a time, since you are hardly using any space.

So I think the default OP is more than adequate for your usage, and I don't think you'll see ANY gain, not even a small one. You just don't need the extra swap space, as you don't have much data to move around. The Anandtech tests are misleading IMO, as they are filling up the hard drive.

 

[UaVaj]

there is definitely a gain. however the gain from the intended usage is moot.

same reason why "everyday typical usage" going from single ssd to dual ssd (raid 0) yields no noticeable difference.

 

[imagoon]

I'm not sure I understand. I thought overprovisioning was just creating an unallocated space partition? I'm not sure I understand your post here.

You should read through here:

http://www.edn.com/design/systems-design/4404566/Understanding-SSD-over-provisioning

SSD over-provisioning has zero to do with partitioning. Page 4 goes it to a lot detail about what it really is. Just partitioning 1/2 the drive simply leave 50% of the user space unallocated and leaving the 7% of the over-provision space untouched. That 7% while be where the copy on writes and remaps will be landing. The 50% of unused disk space will not really be targeted because the SSD considers that space to possibly contain valid user data.

 

[F1shF4t]

You should read through here:

http://www.edn.com/design/systems-design/4404566/Understanding-SSD-over-provisioning

SSD over-provisioning has zero to do with partitioning. Page 4 goes it to a lot detail about what it really is. Just partitioning 1/2 the drive simply leave 50% of the user space unallocated and leaving the 7% of the over-provision space untouched. That 7% while be where the copy on writes and remaps will be landing. The 50% of unused disk space will not really be targeted because the SSD considers that space to possibly contain valid user data.

From that article the OP should look at Figure 4 on page 4. He's using an OS with Trim so all the free space will be treated as over-provisioning. Even in linux you can just use fstrim. In other words doing any sort of partitioning schemes for it is a complete waste of time.

The only place it would make sense is in really write heavy workloads.

 

[UaVaj]

The only place it would make sense is in really write heavy workloads.

which is clearly NOT the intended usage.

 

[imagoon]

From that article the OP should look at Figure 4 on page 4. He's using an OS with Trim so all the free space will be treated as over-provisioning. Even in linux you can just use fstrim. In other words doing any sort of partitioning schemes for it is a complete waste of time.

The only place it would make sense is in really write heavy workloads.

Exactly and not exactly.

Exactly: TRIM does mark the free space as usable assuming the firmware uses the extra zero'd space as a landing zone then it works as described.
Not Exactly: Depending on the drive firmware, the unpartitioned area will never get TRIMed because the OS only sends TRIM commands to the disk about sectors it never uses. The later 1/2 of the disk may not end up in the over-provision pool. It varies a lot but some drives won't keep more than over-provision pool zeroed to reduce writes and latency. Others are really optimistic.

And exactly: He isn't describing a write heavy load like SQL so yeah it is fairly pointless. It only really has value on high write systems, sometimes in RAID arrays since TRIM doesn't always translate properly nor is really wanted and for longevity such as on a higher load SQL server. You would just take a drive like the Intel SSDs and change the over-provision on the 240GB drives to 200GB or 180GB.

I would harbor a guess that in a lot of normal use cases now TRIM isn't really needed for an SSD anymore.

 

[Belial88]

So I think the default OP is more than adequate for your usage, and I don't think you'll see ANY gain, not even a small one. You just don't need the extra swap space, as you don't have much data to move around. The Anandtech tests are misleading IMO, as they are filling up the hard drive.

The factory OP, or a default 10% OP?

That's very interesting what you say though, that I'd get a larger-negligible performance boost without OP at all than using any OP or a large OP. That's the kind of answer that I'm very interested in.

SSD over-provisioning has zero to do with partitioning. Page 4 goes it to a lot detail about what it really is. Just partitioning 1/2 the drive simply leave 50% of the user space unallocated and leaving the 7% of the over-provision space untouched. That 7% while be where the copy on writes and remaps will be landing. The 50% of unused disk space will not really be targeted because the SSD considers that space to possibly contain valid user data.

I'm having a bit of difficulty understanding your post, I think there's a few grammar issues in your post, I'm not sure. I thought you create an Over-Provision by creating Unallocated Space blocks? If that's not how... how do you set up an overprovision?

I love where the discussion is going now but now I'm not so sure what the best course of action is. I will never, ever, ever, ever use more than 60GB on my SSD anytime in the far future, I can say that with 100% certainty, and I don't mind the 2 minutes it takes to set up an OP or not even if the performance gain is near-zero, but now it sounds like you are saying that there's a performance boost with more non-OP'd space and I'm now not sure how you set up an OP.

Thanks for the posts

 

[Cerb]

I'm having a bit of difficulty understanding your post, I think there's a few grammar issues in your post, I'm not sure. I thought you create an Over-Provision by creating Unallocated Space blocks? If that's not how... how do you set up an overprovision?

The blocks need to be in a state the SSD considers as spare, for it to work (such as just having been sent a list of them all by TRIM, by being securely erased and not written to, or by having the size set). By making a partition, in that state, that will end up leaving those now-spare LBAs never touched, the drive can use them as added OP. Or, you can use drive utilities, if they give you the option, to change the apparent size (I'm pretty sure Samsung's has that feature). The net effect is the same, so long as you start with all LBAs in an effectively TRIMed state.

However, the main thing you'll get with more OP is performance consistency. With a light workload, and TRIM, the difference will be negligible. Now, if running without TRIM, or running a hard workload, or running into perceptible performance consistency problems, then adding some can help, sometimes, and might be worth going ahead and doing. But, most of the time, it's not--and, queued TRIM should take care of the linked problem, as the future rolls around to becoming now.

 

[Ben90]

I can't be bothered to read this thread, but I'm sure it's already been mentioned. Instead of over-provisioning, just stop adding data to the drive when you feel its getting "too full". They call me the CS King, throwing common sense down like nobody's business.

 

[imagoon]

I'm having a bit of difficulty understanding your post, I think there's a few grammar issues in your post, I'm not sure. I thought you create an Over-Provision by creating Unallocated Space blocks? If that's not how... how do you set up an overprovision?

Read through the link then. They explain it in detail.

 

[glugglug]

In theory, a perfect TRIM algorithm yields the same benefit as overprovisioning

Also, by decreasing the free space on your usable partition, you increase fragmentation, which you would think wouldn't matter with an SSD. But since you are probably using 4KB clusters on an 8KB or 16KB page size SSD, the fragmentation will lead to write amplification defeating the purpose of the overprovision if you overdo it.

 

[Cerb]

Also, by decreasing the free space on your usable partition, you increase fragmentation, which you would think wouldn't matter with an SSD. But since you are probably using 4KB clusters on an 8KB or 16KB page size SSD, the fragmentation will lead to write amplification defeating the purpose of the overprovision if you overdo it.

But the LBAs are 4K, so that would occur even with larger cluster sizes--the SSD isn't aware of your FS structure, beyond what it's told by TRIM commands. Meanwhile, if the write is large enough, the SSD should be able to figure out that it can write a single page (or more), when given larger writes.

If you increased your cluster size, you'd be trading writing a <4K file or tail into 16K NAND with writing a <4K file or tail into a 16K cluster that then needs 16K of NAND. So, not only was there not a net gain for the flash (net loss, really), but now the files on the system are taking up more space, leaving less free space for both the SSD and OS to play with.

FI, when the SSD needs to move some data, it won't be able to move 4 4K chunks into 1 16K page. Yeah, they hurt on the first write, but now it's not so bad. Meanwhile, if they were 16K chunks, the same data being moved would take 4 pages (on non-SF drives). Also consider that every one of those files that's small, or has a small tail, is taking up 4-12K more space each, and that would be space that could add up to hundreds of MBs, or even a few GBs, depending on what kind of data you work with a lot, of free space on the file system (so there could be non-negligible gains even with an SF drive).

So, basically, not having some free space on your drive is going to be a substantial problem either way. Hopefully caching will take care of the page size problem, though, with SSDs, as NAND shrinks further (be it multimode, like is out now, or something else, like an MRAM write cache).

 

[glugglug]

But the LBAs are 4K, so that would occur even with larger cluster sizes--the SSD isn't aware of your FS structure, beyond what it's told by TRIM commands.

Yes, the LBAs are 4KB, but if the clusters are larger (and aligned properly), the OS will always write the contiguous LBAs making up the cluster as a group, so the SSD will always be able to handle it as a single write (assuming it doesn't need to erase something to make space first).