I would just bite my tongue and ignore these people.
Yeah, that's what I did.
The term "half-breed" doesn't bother me. What annoyed me was that, apparently, this person found his heritage to be a scandal, and the most salient fact about him. That sort of attitude can really get under my skin, and I could feel myself getting ready to chew her out in a way that would have really ruined the evening for all.
Actually Mulatto is just saying half black and half white. I think it is the use of the word "breed" that is offensive. It kind of makes it seem as if those of another race are a lower species and denigrates them to animal status.
I think that's the common point of view. But if you think about it, it's quite backwards. "Mulatto" came from the Spanish word for the mule, an animal that is half-horse and half-donkey.* When they began referring to mixed-race people as "mulatto," they were calling them mules. There's your animal status. Although this term seems to be acceptable today, it is truly very offensive.
Meanwhile, the perfectly good and honorable term, "Negro," (a Latin word meaning black) is often considered an insult. "Half-breed" should be no cause for offense, either. In fact, it is quite a compliment to say that a person has "good breeding," meaning that their family has high social status.
*Yes, you could say that mules are half-assed. But since I come from a state that was known for mule breeding...