Almost all of that is Mesa shaders and GTK's CPU side font-cache for GL/Vulkan, compiled CSS state, FWIW.
If you run:
GSK_RENDERER=cairo ptyxis -s
You can verify that with 69,985 here RES and 52,428 of that SHR. With 5 tabs open it jumped to 71,208 here. Presumably for the encrypted scrollback pre-allocations.
You still may not choose to use it, but it should stay relatively similar the more tabs you open.
Also, it's not a core GNOME app. It's just an app I wrote for me that the distros seem to have liked for its design/platform integration.