While it might work and be actually safe, that does not mean it does not cost a lot to keep it safe for hundred of years (or more).
You normally don't hear about these kind of problems because of course the people responsible don't let the simple and stupid stuff happen. But this has to be prevented forever for all practical purposes, for all waste ever generated.
And so far most of the nuclear waste has been stored right were it was used. SMRs are supposed to be everywhere. So nuclear waste will be everywhere, ready to get stolen, or shot at or blown up.
So it is a relatively close call. To close to hand-wave away the possibility of terrorists or failed states abusing nuclear material.
Just hand-waving away future risks like that does sound intellectually dishonest and careless to me.