Please write a nodejs server that produces those tiny images, don't forgot to correctly set the caching headers...
For really simple ones, the size of the icon isn't a whole lot bigger than a CDN/object storage URL. And if you reuse it a couple times, it might even be a net bandwidth reduction.
99% of the inline <svg> usages I see are people using js includes to inline an SVG which is itself stored in a .jsx file. This is just as lazy and wasteful as it would be if Styled Components were built not to generate classnames and a single stylesheet, but instead with all the rules simply inlined in style attributes on every element.