Not router level, but "Enhancer for YouTube" has "Hide shorts" in its appearance preferences. Available on Chrome, Firefox, and Edge.

If I was a concerned parent, I'd just install and hide the extension from the bookmarks bar.

The downside being that it doesn't affect native YouTube apps for mobile devices...