Yes, retrieval from disk caching is not as fast as one would expect. Detailed article: https://simonhearne.com/2020/network-faster-than-cache/

Since the svg is not inline but rather loaded externally it is being cached aswell. (source https://stackoverflow.com/questions/37832422/how-can-we-cach...)

That's a valid point, for subsequent loads in this case the inline svg is so small that when compared to the image tag it's replacing the difference is pretty small.

But the other side of this is cache latency, this depends on the caching policy defined in the http header, for example some modes require validating caches with the server which incurs a round trip even if it doesn't require always reloading the resource. If it's fully offline caching then as a sibling comment pointed out, disk caching is not free either, under some threshold (which definitely applies here) inline is going to be the fastest way to get an svg rendering.

When optimizing for lighthouse pagespeed metrics, yes.