I hadn't considered self-hosting the compiler, but having put it into works, I probably will.
This was the render the speed up version gave: https://paste.c-net.org/SpikingCarbs
One way to start could be to ignore performance of the data structure.
The first main job dicts are being used for is the `mem` dict mapping a key (variable name) to some value record.
A data structure that supports Store(K, V) & V = Get(K) could be something like an stack allocated array of (Key, Value) pairs, that you search through using linear search to implement Store & Get. It wouldn't be very fast, but you probably don't have too many items in a typical DSL program. You'd need to implement some kind of stack or so on - or perhaps you could get away with reserving some fixed capacity.
The first value of each instruction would then always be one of a fixed set of opcodes, variables their IDs, and numbers left as-is (and we've invented machine code :)). Then (K, V) is always fixed-size and laid out predictably in memory, so the linear-search approach sounds reasonable.
> rg var ray.dsl | wc -l
142
> rg func ray.dsl | wc -l
6
+28 for opcodes, bringing it to 176. So it works for this interesting program, the raytracer, but the compiler likely requires way more. Maybe not the 4 cells I've been using, but 2^16 = 65k would be enough buckets but unique names.