As has been mentioned here, in the US you have no legal expectations of privacy, including being photographed, when in a public area and anything that can be photographed from an area accessible by the public.
I've had a few minor incidents while out and about in public with cameras but since my preferred subject is generally landscapes it isn't usually an issue. I've only had one instance where where I thought things my get "intense". I was at the marina where I keep my boat with my medium format Mamiya RB67. It is a beast of a camera that is solid metal and glass and weighs about 4 pounds. I had taken a picture of some friends walking down the dock with their guests. One of their guests, a big juiced up looking gym rat type, stopped and told me I needed to delete the picture. When I told him it was film he told me I needed to give him the film. I told him he was welcome to try and take it from me. The very public and very loud dressing down he got from his hosts was epic.
I probably could have beaten him to death with that camera and then finished the roll of film.
