Cops with Molon Labe tatoos and enforcing laws at controversy with classical Constitutional interpretation are poster children for Stockholm Syndrome.
I would suggest that this includes the modern anti-free-migration stance.
The problem that most pro immigration control advocates have with free movement of people tends to be the impact on the welfare state. The next biggest complaint is the impact upon wages for unskilled and low skilled labor.
I've yet to encounter an immigration restriction advocate that could articulate any reason (actually based on immigration, not impact on welfare or labor supply/demand curves) why Ellis Island should have allowed a 98% blind admittance rate, but that current policies which come closer to a 98% rejection rate are appropriate.
It's a rare control advocate that can articulate concerns over crime, terrorism, or contagion of a deadly 3rd world disease. And they don't seem to care about the Ellis Island reference and its allowance for enormous organized crime utilization during Prohibition (which certainly rhymes strongly with Prohibition 2.0 against various illegal drugs, and our southern border).
Do you read what other people post on this site?
I'll start with the 2 you disallowed.
1) You can have a welfare state, unlimited franchise, OR open borders. Pick two. Otherwise you'll no longer have money/non-socialist/ non-dictatorship country. Import people who vote for socialists in their country and SHOCKINGLY, they vote for socialists who promise free stuff here.
2) Importing unskilled labor impacts those at the lowest end of the economic stratum far more, including those recently immigrated. This not only harms our citizens, who should be what our government is concerned about, this also makes the "voting for socialism" more likely, too.
These are not unimportant issues, but most libertarians like to pretend that they can just stand on "MUH PRINCIPLES!!!" based on some theoretically perfect world with no regard to the impact in this real world. (Kind of like socialists, in that regard.)
Now, for the ones you think are "justifiable" arguments, they are myriad as well.
1) Crime: Importing unskilled, unvetted, often illiterate aliens increases the crime rate*
2) "Multi-Culturalism"/Social Cohesion: Our country is currently either at or above, the historical
maximum of foreign born population. Any country requires a significant amount of cohesion, and having such a large percentage of assimilated residents will necessarily cause conflict.
3) Disease: We already have enough idiots attacking herd immunity through anti-vaxx opinions, we don't need to import more unvaccinated, possibly diseased individuals with zero vetting. (Honestly, this is the least of my concerns, but if you want to include it, fine by me.)
4) Terrorism: I'm sure you'll have no problem with the resulting government over-reach in order to track which of the completely unvetted and unknown immigrants are terrorists. Much easier to use the NSA, FBI, and the lot to track EVERYONE'S activities, rather than limit who comes into the country and thoroughly vet them.
There's a BRIEF run down on some of the arguments.
*Oh, but THE CATO STUDY! that completely agrees with what I just said. The study liked to tout how immigrants were more law abiding!!! (small print- than citizens of similar demographics in the country). So what it means, aside from conflating legal and illegal immigrants, as such studies are wont to do, they found that the immigrants had a higher crime rate than the average of the country, but since they were better than the native hispanic/white/black/asian/other population's crime rate, well, THEY ARE MORE LAW ABIDING! Of course, the study also makes me wonder what the rates of the citizen children of those immigrants will be. Just guessing it will be closer to the native hispanic/black/white/asian/other population.