Are you looking at a single worker process out of about 10? This is such a frequent mistake it's not even funny. Do something resembling a proper measurement: run `free -m` (or the alternative on your OS) before starting VSCode, and then right after it finishes loading, ideally on a completely bare system to reduce noise.
Here are my numbers for a freshly installed 1.53.2 with zero plugins and no configuration:
used
2882
used
3404
OK, it's "only" 522 MBs for a bare-bones text editor which requires "two dozen plugins" to make somewhat usable.