> I use 32bit server images in VPS because the RAM usage is significantly lower. This is critical in VPS units with 512MB of RAM... I'm worried.

You could use Debian x32 instead. 32 bit pointers but amd64 instruction set: https://wiki.debian.org/X32Port