Being somewhat of a wine aficionado, I can tell you one thing. It doesn't matter what anyone else likes, or claims is a "great" wine. It only matters what you like, and that is only something that can be obtained by experimentation.
I've got a cubic meter crate of wine inbound from New Zealand (yes, I'm fully aware of the duties I'll have to pay upon arrival). Most of it is white wine, though that is a product of wine coming from the Waipara and Marlborough wine regions. The only consistently good reds from those areas are Pinot Noir's, but there's not nearly as many of those as white's like Sauv. Blanc, Riesling, Chardonnay, and such.
In general my tastes run towards dry reds like Cab Sauv, Shiraz/Syrah, Zinfandel (not to be confused with the syrupy White Zin), and particularly the aforementioned Pinot Noir. The New Zealand whites that I have coming are all much dryer in style than most people are used to with white's, but that is a big part of why I like those white wines so much.
But for you, I'd recommend starting with white wines that are somewhere between off-dry and sweet. Most people new to wine just can't enjoy the dry, tannic nature most reds. Then again, try some as you just might really like it. If not just stick to the somewhat sweeter wines and gradually over several years start trying progressively dryer wines and you'll find that you'll come to enjoy the heavy reds over time.
I'd stay away from the White Zin and similar type super-sweet wines (e.g. Arbor Mist) unless you like the idea of drinking an alcoholic liquid Jolly Rancher. If you do what a liquid Jolly Rancher, then go for those, and enjoy. They'll be completely safe from me, though.
Yes, there is such a things as good and inexpensive wine. However that depends on your definition of good, and your definition of inexpensive. What my MIL considers good and inexpensive wine is what I consider cooking wine. What I consider good and inexpensive she considered to be a special treat. Some examples I've found to be good and inexpensive are the Petite Sirah from Bogle Vineyards. Their Old Vine Zinfandel is also quite good, but I don't care at all for their Merlot, Cab Sauv, or Chardonnay (too sweet for me, but you may really like them). All are usually under $14 where I shop (some varietals from Bogle are under $10). Concannon is another vineyard that is on par with Bogle for price and quality.
Yes, there is a time and place for red and white. Generally now.
As far as the basic differences between reds and whites? Other than color, reds tend to be dryer and more tannic, while white will tend to be sweeter, and more acidic. Reds will tend to have flavors described as being like plums, tobacco, black pepper, currants, and such. Whites will tend towards flavors reminiscent of flowers, citrus, butter, and herbs. Reds tend to go better with red meat or tomato sauce type dishes, while whites tend to work better with chicken, fish, or cream sauce type dishes. However, none of that is gospel and I could rattle off countless exceptions to the rest of this paragraph.
Where do you live? If it's anywhere near Phoenix I'd be happy to introduce to a number of different wines.