Regardless of whether he meant it as sexist or not, it was unprofessional, and pretty dumb.
From the quote, he clearly thought the legislator and their staffers were hostile to the military, and giving them free ammo to cause a fuss down the line was not a great move. In this case it was career limiting.
He should have followed SECDEF's advice: "Be polite, Be professional, But have a plan to kill everybody you meet." The Gen failed on points one and two.