It requires a man about two hours of work to remove a cubic meter of soft soil, or at least so the military engineering manuals I read say.
TNT's density is about 1.6 grams/cubic centimeter. So it would require about 300 cubic meters of soil to be removed at a minimum, which would take about 600 hours of work to build if the soil was soft. Say 1200 hours, to accommodate for the soil not necessarily being soft. A platoon of sappers, [about 50 men?] would require 24 hours of work to build it (or maybe 3 days of digging). Now this is just for the hole to stuff the TNT in. There'd probably be far more stuff to dig here, but it could be done.
Also, your calculation figures that there's only one man carrying the TNT. Again, imagine there being a platoon of sappers. They could split the work between them. I've read in the past about massive combat engineering jobs being carried out in stages by groups of people using only basic tools. So while I have doubts about this I'll withhold judgment until a more knowledgeable poster confirms or negates this story.