What kind of computer would you build to hold 10,000 cd's?

[lastig21]

One of the professors on campus has asked me to build him a computer that will hold all of his 10,000 cd's. The cd's will be converted to mp3 probably at a 192k bitrate. I was wondering what type of setup you would build for this guy? Should I go with a SCSI Raid 5, multiple IDE drives, or some other means I haven't even thought of yet? Also, what kind of motherboard/cpu would you build with this setup. I wouldn't think anything overly powerful would be needed, but that is just my own thought.

What kind of powersupply/supplies would I need for this kind of machine. Lets assume 9 harddrives (8 storage and 1 system), 2 optical drives, and a P4 2.2ghz system.

Does anyone know the harddrive size limitation of Windows 2000/XP?

 

[mrzed]

I would go for IDE RAID, unless this box will be serving a lot of workstations as an MP3 server, you don't need a lot of performance out of the drives. But anyone who gets the job of ripping 10,000 CD's to MP3 (you?) is going to want to make damn sure there is some form of backup/redundancy on the job. Maxtor D540X 160Gb drives are cheap, reasonably fast, and huge.

I agree that you won't need anything too powerful for CPU, mobo. Depends on what else if anything he will be using the system for.

Hold on: I just did a quick calculation, and at 192k, assuming an average CD of 65 minutes, you are looking at over 1000 Gb for 10,000 CD's.

You may have to look into some serious storage array here. You would need 7@160Gb not including redundancy. This guy really has 10,000 CD's? Cripes.

 

[TheCorm]

Well......as for space, say each CD contains an average of 50mins of Audio, each Minute taking up 1.25mb of space when encoded, thats 62.5mb per CD.

So 625,000mb of space is needed....625GB of hard disk space!

So, say you put in 4x160GB Hard Drives....If you did it without raid you could have them all on the raid channels, but as standalone drives.

If you wanted to go SCSI Raid 5....well.....lot's of hard disks will be needed. As for motherboard and CPU, theres too many choices of motherboard, I would suggest a high end Pentium 4/Athlon XP so that you can speed up the MP3 encoding, CD-ROM drive that support digital audio extraction and is 50x or better.

Has this guy got a few years free to spend encoding, sorting and naming the tracks then?

Corm

 

[owensdj]

Sounds like you need a computer that has a large amount of fault-tolerant space, but the disk access doesn't have to be very fast like what you'd need in a server. I'd go with a RAID 5 IDE card with 4 of the 120GB 8MB cache Western Digital ATA100 hard drives. That would give this computer 360GB of fault-tolerant space for less than $800 in drives plus the IDE RAID card. You'll need to do some calculations to see if 360GB is enough space to hold all of his music. If you need more space, you'll want to use a 6 or even 8 port IDE RAID 5 card.

 

[Sunner]

Look into a 3Ware Escalade 7850 card.

It's RAID5 capable, you can attach 8 disks to it, and it supports ATA133, so Maxtor's 160 GB disks will work fine.

Have a look here.

 

[jfall]

Your going to need one kick-a$$ reliable/fast CD-ROM in there as well.

 

[lastig21]

The professor was a music reviewer for years, so he easily has 10,000 cd's. I believe he will need about 600-700 gigs of harddrive space for the music he currently has, and am going to leave him some room for expansion. I know ide is cheaper, but is it possible to run an 8 device ide raid 5? Can you point me to any controllers that support that? Also, would you personally trust an ide raid for data integrity? I don't trust ide drives, and would probably want a backup of the drives. 150 dvd's for backups?

Sunner already answered the Raid controller question - thanks.

 

[Adul]

Originally posted by: Sunner
Look into a 3Ware Escalade 7850 card.

It's RAID5 capable, you can attach 8 disks to it, and it supports ATA133, so Maxtor's 160 GB disks will work fine.

Have a look here.

I like sunner suggestion. No need for high perfomance IDE HD. SO maxtors 160 GB drives will be fine. Nice thing about raid 5 is you only need one additional drive for raid 5.
So lets say you get a TB+ of storage going. you will need 8 160 GB Maxtor drives. one of which is used for parity in the raid 5 array. This setup will efectively giive you about 1120 GB of storage. That should be enough for all those cds. Now comes the question on HTF do you organize all this crap?

* Ultra ATA/133 interface
* 64-bit PCI technology
* Support up to 8-drives through independent non-blocking switch ports
* RAID 0, 1, 10, 5 and JBOD
* 64-bit ASIC design with fast on-board microprocessor
* Hot swap and hot spare capability
* Greater than 180MB/sec on reads
* Greater than 127MB/sec on writes
* Delivers ultimate RAID 5 performance

Now that is smoking

 

[wfbberzerker]

I like sunner suggestion. No need for high perfomance IDE HD. SO maxtors 160 GB drives will be fine. Nice thing about raid 5 is you only need one additional drive for raid 5.

why do you only need 1 extra drive for parity?

 

[Adul]

since you'll need a 64Bit PCI slot, you may want to look at a anthlon MP mb. they start at around 200 bucks. Now if you went the SCSI route, well you can look at segates barracuda 180.

At 180 GB per drive, you can shell out 1800 bucks per drive. Now you need an ultra 160 controler. Mylax(sp?) makes a nice one.

 

[Adul]

Originally posted by: wfbberzerker

I like sunner suggestion. No need for high perfomance IDE HD. SO maxtors 160 GB drives will be fine. Nice thing about raid 5 is you only need one additional drive for raid 5.

why do you only need 1 extra drive for parity?

this is not a mirroring setup. Raid 5 can work with a minumum of 3 drives. One of which is for parity. OF you want more space add more drives to the array.

Let me double chekc this to be sure I got it right though

 

[lastig21]

As it stands right now, I am just trying to price a cost effective computer for this guy. I doubt that I will do any of the encoding, unless I need some extra cash. I will probably suggest that he have his students do some for extra credit.

Assuming it takes about 5 minutes to rip a cd (including the swapping of disks), I calculated the time up a couple of days ago. It would take like 33 24hour days to encode all those cd's. It will be a huge project to encode all those songs.

 

[Adul]

write up on raid 5.

now if you realy want redunancy, how abouta raid 5+5 array

you need at least 9 drives for this setup.

When then IDE will be your most cost effective solution. Just ad a tape backup system capable of holding all that info. Ahs for song titles, if you use cdex, it can grab all the info from the net using CDDB.

 

[lastig21]

I was planning on using cdex. I really like its simplicity at ripping cd's to mp3's. Do you like the tape back up idea as compared to dvd media?

 

[Adul]

you can hold a lot of data with a tape backup, and dvd media can get pretty expensive.

 

[Adul]

mmm here is a nice option.

http://rss.seagate.com/products/srssDrives/STDL42401LW-KY.html

Simplifies things geatly

 

[Pariah]

For an array of that size, I wouldn't even bother with a backup, that's why you use RAID to add a level of protection for the data. The only media capable of handling data amounts that large would be enterprise tape backup options which are very expensive. If you absolutely require a backup, building a second array would be the cheapest, fastest , and easiest route to take.

You don't need a MB with 64bit slots to use the above mentioned 3Ware card. Because performance is not a criteria, it would be better to do without for cost reason anyway.

 

[Mavrick007]

Does the guy love every cd he's ever reviewed? I should hope not. Therefore if it was me, I would not keep all those cds. I'd keep maybe a couple hundred and sell the rest.

Heh He's going to need to start playing them and listen to them nonstop for like a year and a half without any replays.

 

[Pariah]

Originally posted by: Adul
mmm here is a nice option.

http://rss.seagate.com/products/srssDrives/STDL42401LW-KY.html

Simplifies things geatly

Actually, that's a bad idea which isn't simple at all. That drive comes up with a 240GB capacity by using a 6 slot loader of 40GB tapes. Unfortunately, that 40GB capacity includes an assumed 2:1 compression ratio for a 20GB uncompressed tape. MP3's are already a compressed format so you will get zero compression when storing. A low estimate 750GB array would require 38 tapes which each would have to be catalogued for their content, which I would not consider simple. Now looking at cost, the drive itself costs $3175, using $125 for a 5 pack of tapes, we come up with an additional $1000 for 40 tapes ringing up a total of $4175 not including SCSI controller. 750GB would require 5 160GB Maxtor drives plus a parity drive for RAID 5 at a cost of $1626 at Buy.com. For the cost of the tape solution you could build 2 additional backup RAID 5 arrays plus still have over $900 left over.

 

[lastig21]

I don't think the guy would want to spend the time sorting out which albums/songs he likes. He is a music fanatic though, and may very well like 90% of the stuff he has. His biggest concern is room though. He has a 4 bedroom house, 3 of which are filled with albums. 10,000 are cd's (in one room), and he has 20,000 albums on vinyl in the other 2. I think he knows nobody would ever want the job of encoding the vinyl, so he just looking at the cd's for now.

 

[Adul]

Originally posted by: Pariah

Originally posted by: Adul
mmm here is a nice option.

http://rss.seagate.com/products/srssDrives/STDL42401LW-KY.html

Simplifies things geatly

Actually, that's a bad idea which isn't simple at all. That drive comes up with a 240GB capacity by using a 6 slot loader of 40GB tapes. Unfortunately, that 40GB capacity includes an assumed 2:1 compression ratio for a 20GB uncompressed tape. MP3's are already a compressed format so you will get zero compression when storing. A low estimate 750GB array would require 38 tapes which each would have to be catalogued for their content, which I would not consider simple. Now looking at cost, the drive itself costs $3175, using $125 for a 5 pack of tapes, we come up with an additional $1000 for 40 tapes ringing up a total of $4175 not including SCSI controller. 750GB would require 5 160GB Maxtor drives plus a parity drive for RAID 5 at a cost of $1626 at Buy.com. For the cost of the tape solution you could build 2 additional backup RAID 5 arrays plus still have over $900 left over.

Didnt think about having another array.

 

[Armitage]

Originally posted by: Pariah
For an array of that size, I wouldn't even bother with a backup, that's why you use RAID to add a level of protection for the data.

RAID only protects you from hardware failure. If you have a filesystem corruption, you're SOL. I would definitely want a backup system of some type. It might actually be cheapest to build two such systems, as you suggested. Put them in different locations and synchronize over a network.

Originally posted by:Maverick007
Does the guy love every cd he's ever reviewed? I should hope not. Therefore if it was me, I would not keep all those cds. I'd keep maybe a couple hundred and sell the rest.

He may not like all of them, but this collection is a part of his life's work. It's probably not important that he likes them as it is that the collection is complete.

On that note, he might consider requesting the assistance of the university library in converting and preserving the collection, including the vinyl. If the school has a library science department, it could turn into a masters degree for somebody.

Remember that storing the data is actually a smallish part of the problem. Being able to search & retrieve the data effectively will be a real challenge. And I suspect that CDDB will likely come up with alot of misses on a collection that large (and potentially obscure).

On the subject of preserving the collection ... he might want to consider an open format like Ogg Vorbis instead of the proprietary MP3 format. Years & years from now MP3 could be dead & forgotten, and you'll be stuck with no players for current systems, and maybe no legal codec for making your own.

 

[Armitage]

Originally posted by: lastig21
I think he knows nobody would ever want the job of encoding the vinyl, so he just looking at the cd's for now.

That's what grad students are for

But seriously ... with some clever scripting and the right kind of turntable, you could probably automate alot of that. Same thing goes for the CDs as well. The really tedious bit will be inputting the information for each item (album title, artist, tracklist, date, notes).

You might consider scanning the cover art as well.

 

[DaveSimmons]

Everyone seems to be pushing RAID but (a) the parity wastes an extra drive (b) any extra read speed from striping isn't really needed.

Why not just have 4-5 NON-RAID drives? Then if one fails it doesn't "bring down" the array and a dead drive can be replaced with a slightly different size/model such as a 200 GB when they arrive. Use the built-in error checking of the drives to catch errors.

For backup, one fast way is using one IDE channel for a removable hard drive cage. This does require a second set of drives (but you get one of them "free" by dropping the RAID idea).

Also, for ripping Plextor SCSI CD-ROM is excellent, another good one is the IDE ASUS 50X.

 

[GoodRevrnd]

Nobody discussed the processors to be used?! He'll obviously need a Quad Xeon for maximum encoding speeds!


BTW: Put in some backdoors on his comp so we can have ourselves a nice little MP3 server. >: )