I've been seeing this a lot, and I'm curious if you actually believe it if it's more of a snappy slogan? I know I've generally seen it from my anarchist friends but I didn't put you into that category.
I roll my eyes at it as well. It has an ounce of truth. Someone is demanding you turn over part of your income or assets against your will. It's overly simplified, but the older I get the less I respect the "social contract" justification and the more I see the situation as "Ultima Ratio Regum". No taxes means no government, and that's anarchism not libertarianism.
On the other hand, let's be honest. We haven't had a government accountable to the people in a long time. That's not going to change either. You can still vote, but it rarely makes a difference. Voting districts are rigged to ensure that you end up in a party majority area. Only if a substantial number of people move or shift parties does any significant change occur. Majority of the time, you get exactly one choice for any office per party. The system is entirely rigged against third parties.
Of course, if folks get too upset over something, the Establishment follows along because they're not suicidal. Short of extremist behavior (confiscating all guns, disenfranchising women, extreme taxes), people will go along because there's no alternative. Maybe moving out of the country, but there's not a huge number of better countries and there's no new frontier anymore.
Social contract is a pretty way of saying "what people born before you enforce upon you". Inertia matters, a lot. Short of violent revolution, our current system won't and can't change. The game is essentially rigged a certain way, and it's not realistically possible to change that. You don't really get more than a symbolic say in how much money is taken from you (backed by threat of force and violence) or how it is spent. If a person did that to you, it'd be theft. Since it's a government, it's taxes. The question is, is there anything where it is moral/ethical/whatever for a government to do where it is not moral/ethical/whatever for an individual to do? Obviously, most people say "yes".