If you want to do this yourself, caddy http server [0] works really well and is incredibly easy to use / setup. They even have an example on their home page that shows this for /api and /blog
It's trickier than it seems, though, because you need to get the right combination of Host/X-Forwarded-Host right for a given hosting service. And you frequently need to rewrite HTML to fix links. Not rocket surgery, but usually I'd rather put my hours into dev and not tweaking a proxy.
You don't have to host all your stuff on one server at all.