Couldn't altitude simply be calculated from lat/long?
To a degree. If you are standing on the ground. And if it hasn't changed since USGS did their last survey.
Lat/long doesn't help determine altitude if you're in a plane, or on top of a building, etc.