Imagine that you could only get to choose one to use for all of your computery needs.
What do you choose and why?
How much data? More than 6TB? Tape. That's not a question, it's a statement of reality unless you are willing to pay 10x tape costs due to niche circumstances. Tape is dirt cheap, reliable and shelf stable. Hard drive is a good option for short term retention, small backups and/or as an intermediary stage because it's cheap ish but fast. A 6TB HD is only $250.
But as a rule of thumb, you actually want disk to disk to tape. You want to back up your stuff to cheap slow hard drives, and then backup that to tape. Low hit on your production systems, quick recovery, all things considered cheap, and damn secure. Practice your recoveries regularly. A 26TB NAS in RAID5 or RAID6 is only like $2-$3k. Dirt cheap, in other words. A tape drive is $1k-$15k. Autoloaders with magazines and tape silos are expensive, but not necessary unless you have enough data that you have a full time sysadmin / backup person. Tape is infinitely superior to hard drive in every way except for speed.
If you go with hard drives, go with the Toaster. I'm not recommending that specific Toaster, btw, just showing you what I'd recommend. They make 1x, 2x, 4x, 6x and (rarely) 8x Toasters, so you could slap in a large number of multi TB hard drives for ease of backups.
http://www.amazon.com/Sabrent-External-Duplicator-Function-EC-HDD2/dp/B00IKC14OG/ For small amounts of data? Flash drives or SD cards. Cheap, durable, fast as hell.