You do realize that this is exactly what the Soviet labor camps did, and yet the professional class of "thieves in law" flourished?
Now, I'm not particularly familiar with the "thieves in law", but according to Wiki, one of their tenets was to "refuse all work" while in the prisons.
That doesn't sound like forced hard labor to me.