there's been a rising tide of academic HCI work in a similar space, wonder if there will be cross-pollination of ideas along these lines (many more papers i'm sure but some off the top of my head): https://arxiv.org/abs/2305.11473 https://arxiv.org/abs/2309.09128
I should really share the design doc from when we started putting this together. The core idea here was a) making a little hobby machine on the canvas and b) using LLMs to “power” the widgets. Somehow that ended up looping back to “ai workflows” but definitely not where we started. I’m only catching up on the space now!