So you wouldn't take issue with an America quota on assembled motor vehicles, for example, since that wouldn't be a tariff? Maybe that automaker moves most of its facilities back to the US and now yea you've got one factory making 10,000 cars/year or something and that's under the quota, but you were effectively tariffed anyway because to maintain facilities would breach the quota that was set. Were you outright tariffed? No. Were you effectively tariffed? Absolutely.
That's why this argument you are putting forth is just semantics. It's not tariffs on duty free goods (except when it is). It's not tariffs on imported products (except when it is). There aren't any restrictions, except when there are. Nobody is moving facilities until they do.
Other countries do the same thing to protect their domestic industries, I'm not singling Canada out here, but I do take issue with the lack of honesty where folks pretend that we had these free trade schemes without any barriers right up until Trump got elected again. It's simply untrue.