It's really, REALLY hard to beat tape in $/GB. It's also very good for long-term storage and archiving. However, it's relatively slow (no random access, either!), and for vast amounts of data, you need a lot of tapes (so either you'll be swapping tapes all day, or you need an expensive autoloader with a large capacity).
I'd lean towards tape if you need to make backups you'll store for a long time, or if you need to keep multiple sets of backups going back months (or years). External hard drives aren't a bad idea for nearline storage/backups. What actually might make sense would be to build a 'backup server' with a bunch of hard drives in it. Have your regular servers back up to its (network-mapped and redundantly RAIDed) drives, and then have the 'backup server' dump to tape every now and then for long-term storage. Sort of a low-cost SAN approach.