For small personal projects, you don't usually buy a $5/month VPS just to use as a dedicated reverse proxy.
What needs to run on this that's a memory hog making 512MB too small? By my (very rough) calculations youd need 50-100MB for kernel + systemd + sshd + nginx base needs + tunnels home. That leaves the rest for per-request processing.
Each request starts needing enough RAM to parse the https headers into a request object, open a connection back to the origin, and buffer a little bit of traffic that comes in while that request is being processed/origin connection opens. After that you only need to maintian 2 connections plus some buffer space - Generously 50KB initially and 10KB ongoing. There's enough space for a thousand concurrent requests in the ram not used by the system. Proxying is fairly cheap - the app servers (at the origin) may need much much more, but that's not the point of the VPS being discussed.
Also worth noting that the cheap VPS is not a per-project cost - that is the reverse proxy that handles all HTTP traffic into your homelab.