If you switch to the test-fp8 branch of AUTOMATIC1111 web UI, you can do 512 * 1024 on 4G GPU with SDXL.