Backblaze and Seagate

[bigboxes]

I also have an equal number of 2TB Seagate drives. All running problem free.

 

[Dave2150]

Seagate drives are just crap compared to WD.

Seagate are unreliable and cheap, wouldn't trust my data on them.

 

[bigboxes]

Seagate drives are just crap compared to WD.

Seagate are unreliable and cheap, wouldn't trust my data on them.

You mean like my two dead green drives? Or my dead WD drive when I lost a lot of unrecoverable data? You mean those WD drives? Maybe you mean the 74GB Raptor that is still flawless after 8.5 years? Or maybe you mean the lone green drive that hasn't bitten the dust. Let me know which WD drive you are referencing.

My point is that all makes have bad models. I've got a crapload of 1.5TB/2TB/4TB Seagate drives that are chugging along just fine.

 

[zir_blazer]

I know that one of the main complains about BackBlaze methodology is that they're using the HDs in a way that its not intended for them to be used. However, as far that I know, that is how usually construction materials are tested: Their useful life is calculated based on results from their endurance under stressful conditions to force acelerated aging.
Basically, you can take that if an specific HD model does well with them, they must at the very least be equal or more reliable on light usage scenarios than the ones that fails. And that ABSOLUTELY MEANS that Seagate models are much worse than their competition, even if their reliability was equivalent under those light scenarios. And remember, we are not talking about the HDs themselves, since replacing parts isn't that hard, is the invaluable data on them (Which if you did your homework you would have a backup of, but maybe you just have the critical files and not all since you don't have money to purchase another multiTB HD). So, who would you trust more? Hitachi workhorses that you can throw in BackBlaze inferno and pull them alive, or Seagate fragile diskses?

 

[eton975]

-snip-

This.

 

[smakme7757]

As far as i know Jotta Cloud (A Norwegian cloud company) also does this. They buy in consumer Hard Drives with a 5 year warranty and then just send them back under warranty when they fail. As most drives fail within two years even drives with a 3 year warranty is good enough.

The savings are worth the hassle with postage as far as they were concerned.

So it's not incompetence, it's just risk and economic management.

 

[coercitiv]

Here are the facts as far as have been reported by both Backblaze and the various tech sites.

Backblaze by their own admission buys in harddrives based on price with no other consideration being paramount.

Did those cheap drives also come with a different MTBF from their manufacturers?

Backblaze by their own admission bought Seagate drives alost exclusively during the harddrive crisis after the floods because other manufacturers were not able to supply.

Did Seagate or any other manufacturers warn customers of reliability concerns due to the floods?

Backblaze by their own admission buys in external USB drives to take them apart.

Are USB drives more prone to failure under certain temperature/vibration conditions? Are they significantly different than other retail drives from the same manufacturer?

Backblaze by their own admission stated that when they could use the drives even after they had been deemed faulty in a single drive environment.

Can you please clarify this statement?

Now to my mind whoever was/is responsible for purchasing mission critical equipment at Backblaze is criminally irresponsible.

Why? Did they lose data? Did they end up spending considerably more money? What damage has been done?

One thing which I have not seen in any reporting is what kind of Power Supply Units they use in their drive farms. If they use the same kind of criterion there (cheapest possible) then what surprises me least is the amount of drive failures they have had but rather that there have not been even more.

However, you know little about the PSUs being employed. Are you suggesting they use bad PSUs wit Seagate drives and reliable ones with other HDD brands?

Backblaze has bought in higher end Hitachi drives when the price has fallen due to the introduction of newer model. They buy in newest models Seagate drives because of capacity and price irregardless of the STATED USE by Seagate.

So they bought faulty Seagate drives to begin with?

What I find horrific is that in a so-called "professional" data storage company they get a high percentage of their drives from breaking apart external USB drives.

You already mentioned this, did anything change since 5 paragraphs ago?

It's like there was a dire need for engines, so they bought a few thousand cheap VW Golf cars, stripped the engines out of them, bunged them into tractors and then turn around and say that VW is crap because so many of the engines they built into the tractors failed under load.

Are you suggesting a VW Golf engine is not reliable enough to work as a tractor engine?

If Backblaze is too effing stupid to use the right tool for the right effing job then they cannot blame anyone but themselves.

But did they blame anyone? Seems to me you don't realize what they've been saying all along with these reports: it's cheaper to use cheap drives. They use more drives to achieve the same reliability, but they still end up paying less. This shouldn't happen with properly priced enterprise drives.

Instead of focusing on Seagate, you might want to learn what is valuable from their study: every manufacturer has it's bad period/drive, and when building a data center one should create a purchasing strategy as to avoid exposing yourself to such an event.

BTW, my questions were all rhetorical, you are welcome to answer them (and I will gladly read you posts further on) but I won't pursue any of the points above any further.

What I would really like to see on this thread is any kind of other study made for drives in this generation that paints a different reliability picture than the one from Backblaze.

 

[VirtualLarry]

I reported your post and here is exactly what I said:
If you are going to have a go at me then at least try to make a cogent argument instead of taking your intellectual cues from the likes of Faux News.

I reported your post because you said "STFU" to another member here. Flaming other members in the technical forums is not allowed. Please see the forum rules, and try to be civil.

Edit: Nevermind. I see that the mods took some action.

 

[VirtualLarry]

I reported your post and here is exactly what I said:

And please tell me where you see "8x5" specified with regard to HDs. Again another claim I cannot see verified anywhere.

What you seem to mistake is "being available 24/7" (powered on but not spun up) with "being accessed 24/7" (powered on and spun up).

If you are going to have a go at me then at least try to make a cogent argument instead of taking your intellectual cues from the likes of Faux News.

http://www.seagate.com/files/www-co...us/docs/7200rpm-drive-spec-mb578-7-1201us.pdf

Straight from Seagate's mouth: "POH profile" (For Barracuda drives) - "8x5"

Edit: Next time, try Facts, not Flaming.

Edit: Btw, I found that PDF with a web search for "Seagate 8x5". It was in the top results. At least make an attempt.

 

[Sabrewings]

Most consumer drives should work very well in a RAID environment. Take WD reds and greens for example. Reds are your NAS drives and greens are your eco drives. Both are constructed the same, just with different firmwares. Specifically, the reds have a low TLER counter so that if they hit a bad sector they report it bad quickly and the RAID controller reconstructs the data if possible depending on the RAID setup in use. For greens, they have a longer TLER (which can't be changed, unfortunately) since they're designed to work alone and it is assumed that they should try until they can read the sector. This causes a lack of communication to the RAID controller and that drive will be kicked from the array. In this situation I would then replace the drive. If one bad sector occurred more can follow.

That's the differences (at least on the WD side). This nonsense that they're being used outside of their test envelopes is a poor argument to start with. That would be fair if they used enterprise drives from other brands but consumer Seagates. The main takeaway is that Seagates were failing sooner than their competitors. Will your Seagate (of that model) fail tomorrow? Maybe, maybe not. The point is that the life expectancy is lower.

And since when did staying on all the time become bad for HDDs? It's always been spinning up and parking the heads that was the most wear, not idling. Despite that, I have four WD blues that have been in a RAID 5 since 2011 and have not had a single one fail yet. That system has over 30,000 hours operation in that time, so they've seen their fair share of on time as well as spin up/down cycles. Unless they're actively working on something (which is a lot, actually) I turn them off at night.

BTW, RAID 5 is an excellent way to increase data redundancy and read performance. You simply can't rely on one safety net if it is something you can't bear to lose. I take weekly backups to external drives of everything on our PCs, even the RAID arrays. If a drive drops out of an array, that array is shut down until a fresh drive is installed and it is left alone until the rebuild is done (never happened to my home machines, but that's the plan if it does).

Some things can't be replaced, so take every precaution you can regardless of your storage redundancies.

 

[BFG10K]

I am a bit cheesed off about seeing posts using the reporting of Backblaze to paint Seagate harddrives in a bad light.

And yet, after that entire nonsensical rant, the other brands did better under identical circumstances. How exactly is that Backblaze's fault?

 

[mikeymikec]

@ Nec_V20

I'm wondering why you're mentioning RAID when criticising Blackblaze; do you realise that when multiple drives are run in a RAID configuration, the drives themselves don't work any differently, in any scenario it's the storage controller's job to tell the drive what to store.

I'm also wondering why you're saying that the person responsible is "criminally irresponsible" for buying such drives. The acronym "RAID" originally stood for:

Redundant Array of Inexpensive (or 'Independent') Disks

The point of RAID in the vast majority of cases is in the first letter: Redundancy. Drive failures are to be expected, so a competent sysadmin plans for it with as much budget as they can feasibly throw at it.

Your impression that RAID came about purely because businesses were running out of space is simply inaccurate. Using RAID for that reason makes very little sense as one can simply connect up another HDD to a computer, whereas you would need to reinitialise RAID0 every single time you add another disk "for more space"... what's the point? RAID0 (aside from obvious reliability concerns) makes things unnecessarily complicated if your only goal is to increase data storage capacity. The only valid reason to use RAID0 is to increase performance (with a large number of "ifs" and "buts"). Yet you then go on to talk about mirroring which does not increase the data storage capacity! 2x 250GB HDDs in RAID1 configuration = 250GB total capacity (ignoring GB/GiB).

If there were disks out there considered to be so reliable that redundancy isn't a cost-effective priority, then a lot of sysadmins would invest that money elsewhere. If say instead processors were deemed 'not particularly reliable', then server boards would be designed to allow redundancy in that department instead.

If a sysadmin is finding that a particular model of drive is failing a lot more often than others, the obvious thing to do is not to use that model any more. Blackblaze apparently did this and reported their findings, and for some reason you have a problem with this.

The only (on topic) reason why I might be inclined to question a sysadmin's competence when picking drives for such a scenario as Blackblaze's (as I understand it) would be if the person had been picking drives that the manufacturer had clearly marked as something like, "this drive is not designed to withstand stress caused by running 24x7, running it in such an environment will void the warranty and cause the drive to fail prematurely", which is a warning I'm not sure I've seen before on a HDD, possibly because (at least in my experience), HDDs tend to be more reliable if they're not being spun up and down on a regular basis.

 

[redzo]

The only (on topic) reason why I might be inclined to question a sysadmin's competence when picking drives for such a scenario as Blackblaze's (as I understand it) would be if the person had been picking drives that the manufacturer had clearly marked as something like, "this drive is not designed to withstand stress caused by running 24x7, running it in such an environment will void the warranty and cause the drive to fail prematurely", which is a warning I'm not sure I've seen before on a HDD, possibly because (at least in my experience), HDDs tend to be more reliable if they're not being spun up and down on a regular basis.

This is the tricky part. ST warrants a AFR of less than 1% for the baracuda working only 5 days a week and not 24/7 in RAID 0 or 1 max. If you exceed any of ST's specs you are on your own as far as the Annual Failure Rate is concerned, and: reliability becomes a lottery, failure rate increases. This is basically what the manufacturer says/specs.

My point is:

1. BB is to blame for the way they've used some drives.

2. But this does not necessary mean that their entire report is useless. Can't you still make a clue of the reliability across multiple brands? After all, the drives were working under the same conditions. But this is were I'm gonna get a little picky: you need to take price / capacity / model into account because some cheapo drive has all the chances to turn up as the less reliable drive.

 

[iluvdeal]

I find it interesting how some want to disregard Backblaze's findings due to their beyond the specs use of the drives. Aren't extreme tests and benchmarks what geeks pay the most attention to when hardware tech sites do their reviews? This is the same community which prizes overclocking right?

You'd think they would rejoice such test results were made public of a real world case involving a huge amount of drives, but nope, it's the opposite reaction from people like Nec_V20 who seems enraged about it for some reason. If you're going to rail against Backblaze, why not take aim at every tech site including Anandtech who uses results from extreme, out of spec testing?

 

[poofyhairguy]

You'd think they would rejoice such test results were made public of a real world case involving a huge amount of drives, but nope, it's the opposite reaction from people like Nec_V20 who seems enraged about it for some reason. If you're going to rail against Backblaze, why not take aim at every tech site including Anandtech who uses results from extreme, out of spec testing?

It is awesome that the data is available. I for sure think more highly of certain Hitachi drives than I did.

The problem is when the data is used to extrapolate things that aren't true. Like just because one Seagate drive model is bad then ALL Seagate drives or bad. Or heck, even just because the 2011 versions of that drive were bad doesn't mean new ones are. The first 3TB Seagate drives had five platters, current ones have three. Product and product lines can change over time. "3TB Seagate Drive" could mean a dozen actual variations within a couple of models.

 

[Auric]

VW Golf TDI engines are renowned for durability and sold for industrial and marine applications.

 

[Red Squirrel]

Their stats compare to other non enterprise drives, so I think their numbers are still something to be considered. Just because a drive is not meant to be used in such an environment does not mean it should fail so fast. If the other non enterprise drives are putting up with that environment and Seagates don't, well that still gives a user a general idea of which drives to avoid if they want to do a raid setup or any setup for that matter.

 

[smitbret]

It is awesome that the data is available. I for sure think more highly of certain Hitachi drives than I did.

The problem is when the data is used to extrapolate things that aren't true. Like just because one Seagate drive model is bad then ALL Seagate drives or bad. Or heck, even just because the 2011 versions of that drive were bad doesn't mean new ones are. The first 3TB Seagate drives had five platters, current ones have three. Product and product lines can change over time. "3TB Seagate Drive" could mean a dozen actual variations within a couple of models.

Once again, Poofy, you've broken things down to what they really are.

99% of us will not be using our HDDs in a manner that is similar to Backblaze. No one is saying they should assume that their results will be similar.

I would be willing to bet that most people could run 3TB Seagates for 5 years in their PCs and have no problems at all. I have had 4 x 2TB Seagates in my FlexRAID for the last 2 1/2 years and they have been awesome for speed, noise and temp. If 2TB was still the sweet spot then I would get more. However, 3TB and now 5TB are the better value.

Enter Backblaze data. I can look at large samples of drives from multiple manufacturers, all used under the same high stress environment, thus removing external factors. I can clearly see that the 3TB Seagates have a much higher failure rate than WD, Hitachi and possibly Toshiba when used in the same environment. Now I can decide if it is worth it to pay $75 for the Seagate or get what is probably a little better HGST or Toshiba for $10-20 more.

I don't know how it could be any more clear that 3TB Seagates are more fragile than their counterparts. I wouldn't pay the extra for a 3TB WD Red, either. That doesn't mean they will fail when used as directed but I can decide not to take the chance and I don't. It is worth a little extra investment to get the drive that is much less likely to fail under stress.

How is it not obvious?

 

[Auric]

Yeah, for volume buyers small percentages add up to real monies but individual end-users are unlikely to regret spending a bit more -especially since shoddy has become the new normal.

 

[coercitiv]

Enter Backblaze data. I can look at large samples of drives from multiple manufacturers, all used under the same high stress environment, thus removing external factors. I can clearly see that the 3TB Seagates have a much higher failure rate than WD, Hitachi and possibly Toshiba when used in the same environment. Now I can decide if it is worth it to pay $75 for the Seagate or get what is probably a little better HGST or Toshiba for $10-20 more.

I don't know how it could be any more clear that 3TB Seagates are more fragile than their counterparts. I wouldn't pay the extra for a 3TB WD Red, either. That doesn't mean they will fail when used as directed but I can decide not to take the chance and I don't. It is worth a little extra investment to get the drive that is much less likely to fail under stress.

Also, keep in mind Backblaze publishes new data periodically, so with new data sets we are likely to see newer Seagate drives return to normal as far as failure rate is concerned.

Furthermore, one should look at this data just as astronomers look at space imagery: by the time you have the data and are able to compile it, the phenomenon your are observing is likely gone. (unless you plan to buy same revision drives)

 

[B-Riz]

I find it interesting how some want to disregard Backblaze's findings due to their beyond the specs use of the drives. Aren't extreme tests and benchmarks what geeks pay the most attention to when hardware tech sites do their reviews? This is the same community which prizes overclocking right?

You'd think they would rejoice such test results were made public of a real world case involving a huge amount of drives, but nope, it's the opposite reaction from people like Nec_V20 who seems enraged about it for some reason. If you're going to rail against Backblaze, why not take aim at every tech site including Anandtech who uses results from extreme, out of spec testing?

+1

What I find most interesting is that the anomaly that caused the failures is still unknown, to us at least.

The reason the drives, or a certain population of drives fails, is never explicitly expounded upon in the BB data; BB just gives the data and lets us make our own conclusions.

Was it a firmware quirk or a bad production run / sub par parts used in manufacturing?

 

[eton975]

+1

What I find most interesting is that the anomaly that caused the failures is still unknown, to us at least.

The reason the drives, or a certain population of drives fails, is never explicitly expounded upon in the BB data; BB just gives the data and lets us make our own conclusions.

Was it a firmware quirk or a bad production run / sub par parts used in manufacturing?

Well, it wasn't that long after the Thailand floods when they bought the drives. The Backblaze article lists that as a definite possibility for the high fail rates.

 

[RecoveryForce]

I've only gone back a couple of weeks and may go back further, if I have time. But, for now, I'll do my best to add new cases as they come in.

What I've done is created a thread on my company's forum listing Seagate DM (and variants) cases. Perhaps, as the list gets longer, we can extrapolate the exact models to avoid.

http://www.recoveryforce.com/forums/viewtopic.php?f=12&t=157

 

[therealnickdanger]

"Lies, damned lies, and statistics."

Exposure is typically the primary factor when considering rates of any kind. Take car crashes, for example. Interstates typically have more crashes than any other type of roadway, but for the simple reason that they have more cars every day. When adjusted for volume, the crash rate is incredibly low compared to other road types. However, in the same stroke, Interstates are considerable safer from a death/injury perspective than any other road, especially when factoring in volume. So how do you find a "bad" portion of Interstate?

The only valuable statistic is one that is weighted against its peers. Backblaze's Seagate drives (they had tons of them) performed poorly compared to other drives of similar size. Going by Amazon and Newegg reviews for similar drives, Seagate is hit or miss. Anecdotally, all but one of my Seagate drives has failed, but none of my Hitachi, HGST, or WD have failed. I can't, in good conscious, use or recommend Seagate products to anyone. I don't think that's unfair. It's not an attack on Seagate or people that have had good luck with them as there are outliers in every sample.

 

[eton975]

I've only gone back a couple of weeks and may go back further, if I have time. But, for now, I'll do my best to add new cases as they come in.

What I've done is created a thread on my company's forum listing Seagate DM (and variants) cases. Perhaps, as the list gets longer, we can extrapolate the exact models to avoid.

http://www.recoveryforce.com/forums/viewtopic.php?f=12&t=157

Seems to be a lot of head problems with the 1TB drives.

Isn't the main problem with trying to do this that you only see the failed drives, without seeing all the healthy drives that made it? (Unless you're getting waaaaaaaaaaaaaayyyyyyy more of a particular model, the cause of which can't be attributed to more being sold.)