That's an interesting project, going to keep an eye on it! I have a hobby/DIY project involving sending several hundred UDP packets per second to various ESP32s across the house (driving LED strips live based on music playing on my PC). Much to my chagrin, the ESP32s tend to choke/get stuck quite frequently at higher packet volumes, and I've not found a good way to debug the issue further. See [1] below. While I haven't crossed the bridge of leaving the comfort of the Arduino IDE for flashing the ESP32s, trying out/beta-testing an open WiFi stack (once available) that I know I'll have a chance to thoroughly trace/debug would be very enticing.

[1] I did find out so far that my WiFi setup occasionally "clumps" packets, causing like 10 packets to hit a given ESP32 at an instant (instead of a few ms apart) - not great, but should not be disastrous. However, this seems to cause the ESP32 WiFi stack to just slow to a crawl: It responds to pings much slower (like, in 100+ ms range) (to my surprise it actually responds to PING requests out of the box in the first place...) and/or doesn't really process any more packets in general if I continue sending at the same rate as normal. But backing off on the packet stream usually gets it back on track, strangely enough. This also happens if I do nothing in the main loop except clear packets as they come in, so it's not in my code.