Yes, it is best to get a silent case. But any old case you have can be made silent. For example use passive (no fan) CPU cooling and the largest diameter, slowest turning fans you can find. Then glue on sound deadening foam to all the large sheet metal parts. Pasive cooling and LARGE diameter fans are the key
But the #1 thing I did that works 100% is to place the computer in a closet. Place it on a top shelf and screw some ceiling tiles to the walls/ceiling near the computer to absorb reflected noise then shut the closet door. Use the above ideas first, passive cooing and so on but the closet door does wonders
Or simply buy a MacBook "air". Those are dead silent right out of the box. (You can place your ear on the case and still it is silent. They cost less then you think. You pay about $1,000 but after a couple years they sell used for about $700. So $300 out of pocket gets you a dead silent computer. Put all your files on a NAS and plug in a 30" LCD monitor if you need a big screen. Your cost is the purchase minus the resale. Look on Craigslist for resale values.
I'm using a system sort of like this, a big NAS for about 1TB of photos and 300GB of music and loads of other stuff and then a small notebook (Macbook pro, Sony, Tosiba, the "Air" and Kindle. The NAS server in the closet is dead-on silent as are the Apple branded products. The Sony and Tosiba have semi-loud fans that some times run.
When building your silent computer you can learn a little from the MacBooks what they did is to use a solid aluminum case. If you open that computer there is no heat sink. The case itself is a one big aluminum heat sink as it is milled from a block of metal. They have basically made the CPU heat sink into the shape of a notebook computer case. You can't do that but you CAN use an aluminum case that has no exterior paint and place it on a couple strips of wood so air can flow under it, in other words use the case as a heat sink so you don't need to run the fans so hard. Don't polish the case, Apple's sand-blasted texture is what you want if you can get it.