I mainly ride a newer Jamis touring bike. It's a heavy steel framed bike but I love it for long rides/overnights, etc. Looks like an old school ten speed with the drop handle bars, except I got front and rear racks for panniers. It has full fenders so I don't get a skunk strip if I ride in the rain. Very upright seating position, once I get stretched out after 5 or so miles, I can ride this bike all day. I also have a trailer I pull with it, sometimes I go shopping with it. This bike was $950 before I started adding stuff and I upgraded the rear wheel set ($200) last spring.
I also have a mid 90's Gary Fischer hard tail mountain bike that I use as my beat around bike. I saved this one a few years ago from under a deck, gave the guy $100, threw on $100 worth of tires and new brake pads, paid the LBS $100 for complete tune-up (March special deal). I have rock tires (for crushed rock trails) on it for the warm months and carbide studded tires (piecked those up new on craigslist for $50) for the winter. It's a quick bike, but no top end, it is aggressive on the seating so it does get uncomfortable after a dozen or so miles. I love in for dicking around town. I have a rear rack on it for panniers.
I'd go to some bike stores, get fitted/measured, start checking out bikes that feel comfortable to you. Stay away from the big box discount stores. Check out the used bikes as KD mentioned.
I used toe clips, but I'm thinking about going clip less. So much more efficient that bare pedals.
My next bike is going to be a fat bike, they look like a lot of fun in the snow.