There is no utopia.
I agree, that is, as long as human beings are in charge. The expectation of a perfect world is unrealistic in political science, although one should hope for the best system.
Also, I don't know exactly how everything would work in a market anarchy.
Good starting point; let's be honest about our ignorance. I am not an expert in Constitutional law, and haven't read much Locke or Montesque.
The police would be paid voluntarily by customers. Police firms that displeased the populace would lose customers. The populace would be at least as armed as the police and would outnumber them. The police might resemble current security firms like Brinks.
How did Medieval fealty and feudalism work?
I bring up feudalism because you spoke of private police or armies in the pay of individuals, and by extension, groups. I wonder why the larger group or those with the best or largest army could not make slaves of a neighboring group. This is what happened throughout history. There were governments involved, sure, but these were only powerful individuals with wealth and military forces, perhaps with the backing of religious ideas. I can't see how anarchy would last or even truly develop, as people naturally band together under good leadership for their own protection from real or perceived threats. Under the feudal system of Europe, which began in an age of horrific Viking raids, everyone pledged loyalty to someone more powerful than they and often recieved loyalty from others with less resources. Contracts were honored, which bound people to certain plots of land, and to give a certain share of crops, labor for road-maintenance, etc. At least this is what I learned from a survey course of Western History. I welcome any corrections or other perspectives.
Or, what if a large security force becomes respectable in the community and begins to run things. Soon you have mafia rule.
The populace would be at least as armed as the police and would outnumber them
The masses have always outnumbered those who opress them, so that doesn't help. Being equally or better armed is desirable, but I doubt it would happen. Why would you hire police and military that weren't better armed than those around them? And why wouldn't this populace surrender to weapons control as other peoples have done? Why would these people be more jealous of thier liberties than others?
Maybe each contract would mention in which court future arbitration would take place. Anyone that refused to attend court or abide by the ruling would be considered to have relinquished his own similar rights and would be fair game to all.
For instance, if someone took delivery of a good but didn't pay and didn't attend court, he would be publically stripped his rights to personal property. Anyone could take his property.
If one assaulted another and didn't show up for court, he would forfeit his right to personal safety. He could be then treated like a marauding grizzley or other uncivilized animal.
Boycott and shunning would be effective to make the offender have to move away.
OK, with contract law, I suppose that might work. They could agree on things ahead of time. With criminal law, though, I don't understand this. If you accuse me of stealing from you, what court decides? You insist on your father as presiding judge, with your mother and siblings as jury. When I refuse this court, your family declares me "fair game," and what recourse do I have? What if one is a wealthy elderly widow, with no friends or family, accused by a family with money and hired security? I suppose the community must decide who to side with. Then the widow must stand her ground and lose all property or be sucked dry by those who will offer her "protection."
I suppose these things could and have happened with governmental systems in place, but I doubt your market anarchy will protect anyone's rights better than our Constitutional republic. My view is this: Someone will always have power over others; Government legitimizes that power, regularizes it, and, especially under popular governments, brings it under control.