This photo illustrates my biggest concern about launching super heavy at Boca Chica. If that thing goes boom there's going to be lot of broken glass and the very high probability of people being injured. Heaven forbid that thing veers off in that direction.
meh. SpaceX's launch site is [checks google maps] 4.99 miles from South Padre Island. Several of the launch complex's at Canaveral are closer to Port Canaveral than that, and the town of Port Canaveral is pretty packed. I think NASA has mostly moved to launch sites farther north these days but there are certainly launch pads built closer to people than 5 miles.
Vandenburg's launchpads are closer to Dependa housing then SpaceX is to SOuth Padre.