Part of it is, I'm not really upset with the Idaho thing. they weren't J6'd, they were arrested, some of them were hit with misdemeanor charges based on the plan (which I admit I haven't read) and released.
Part of it is that I keep seeing the same names of individuals moving from group to group, so while "Patriot Front" hasn't been violent, "Vanguard America" and "Identity Evropa" certainly have, and they have significant overlap of membership. In the case of VA, almost 100%. PF sprang up from the VA discord servers after one of that groups members killed someone in Charlottesville, and they decided to "retire" that brand. I am unwilling to let them play the "we aren't violent that's the other BAD white supremacist's" game when it is largely the same individuals.
I also don't (or try not to) conflate actual violence, with unlawful, or distasteful behavior. Distasteful behavior, like the recent march in Boston, should be allowed by law enforcement, but condemned by the community. Unlawful behavior like the alleged plans for violence in Idaho, or the trespassing in the capitol, should be dealt with by Law Enforcement and the Justice system. An arrest and misdemeanor charges seem appropriate, and gives them their day in court if they want to argue they were targeted. Actual violence like Portland or Kenosha should be water cannoned off the streets, and arrested for felonies from the gutter, and those charges should be pressed. I am aware that we are having real problems following through on that last part, but still that's the should.