So if I want to get my own domain name, website and email, where should I be looking for a reliable and cost-effective service? I don't need retail or cash handling or anything like that. Just a personal website.
Buy your domain name from a separate company than your web hosting. There's no technical complexity in doing so (usually), and if you ever need to ditch your hosting company due to issues, you're not totally hosed.
I assume you want a hosted provider, that takes care of all the background stuff, compuwebhosting.com is decent and I've had few issues with them.
For my virtual server provider, that I don't really recommend to your requirements, I use linode. They're friggin awesome.