When I say I hate driving on LA freeways or the Beltway or whatever, it's not because I'm uncomfortable driving there, it's because I absolutely hate traffic. Especially rush hour traffic. Other than being extra aware for idiots, the driving itself is pretty straightforward, and as long as you know your interchanges and exits, you're okay. It can be a little hairy if you're going someplace new and hit a complicated interchange, because you often won't know exactly where to go until the last second, and then of course none of the idiots on the road will let you over, so sometimes it requires being as aggro as them (even though I hate doing that).
LA also is not super bad outside of main rush hours, though that timeframe seems to be greatly expanding to the point where all the rush hours are going to merge together into Rushnado. I've been able to do very early morning airport runs there without ever dropping below 60MPH. On the other hand, when I used to have to attend USCG meetings, which were held at the port or airport, always around 10AM, it was either sit through absolutely horrendous rush hour traffic at the 101 and 405 interchanges, one of the worst traffic jams in the freakin' universe, or leave at 0400, miss most of the traffic, but then have to sit around somewhere for four hours till the meeting started. Then of course have to do the reverse on the way home for meetings that got out late in the afternoon.
Places where I would be uncomfortable driving would be more like the middle of NYC, where it's crowded and unfamiliar and there's a maze of streets to navigate. I've never driven in NYC, but I have in downtown DC and in San Francisco and hated them both. Same with downtown LA, even though I'm pretty familiar with it.