no yet! that's a good direction!
306 karma · joined October 21, 2024
Would especially be handy when we are building some agents that produce charts that serve end users (they want faster and more reliable experiences!).
But! We have a new project coming in to work on annotation machanism and interactivity to make it supporting presentation tasks.
I added an issue to track this
Effectively Flint is designed to specify charts using another set of semantic parameters over geometric parameters, which happens to be something AI are pretty good at! (They can based on field names + sample values to infer the semantic types reliably than guessing geometrical params.
Btw, for others reading this comment, this is the XKCD we are talking about! https://xkcd.com/927/
Good news is that AI do make new language a bit more accessible than before! If your agents can use it well and you can steer easily, it will naturally be good adoption.
This is exactly why this is an intermediate language designed to get 95% stuff right easily (for expressiveness and reliability purpose), while 5% of more advanced case where the agents need to revise chart for other purpose can be done easily on top of the compiled low-level spec (low in terms of Vega-Lite etc, not SVG). We are not really designing a higher abstraction to replace existing ones.
In the past, the split is like 50% good at first run for some common stuff, all other stuff requires agent-loop or user involvement.
Our goal is to make it easy for most case, not everything needs a full multi-round trip agentic workflow to solve. :)
We are kinda all advanced users in fact, for a lot of users, they are easily get confused with the first time result if that is not as good, and the interactivity cost / multi-round isn't an option.
We use a fun elastic algorithm to decide dimension etc within the developer's constraint.
The "how it works" section explains a little bit of this. For example, for the heatmap example showing temporal data, for a "good-looking" chart, we need to (1) reconcile the conflict between banded discrete steps and continuous temporal axis, and it requires understanding and setting stepsize and time parser, (2) for the correlation color, we need to set domain etc under the color axis.
These are supposed to be handled automatically as system defaults, but the tricky part is that these decisions are "semantical", thus requires us to understand the data and design principles, thus existing languages won't stretch that far. And the actual good looking spec is actually over 40 lines of json spec with many low-level paramters, way beyond the simple 5 line encoding promised by GoG. Flint uses semantic type and a layout optimization algorithm to handle this, so 5 lines of encoding + data semantic types can derive rest parameters automatically.
Some examples in the gallery are more extreme: like the waterfall chart example is way over 100 lines of code, and sunbusrt, rose chart are even more since compositions are quite difficult in GoG.
Glad to have a discussion on this level! In fact, we wrote a paper about this, will be putting it online in a week or so!
A challenge with GoG is that it assumes configurations as second-class stuff, which makes it quite difficult for users to deal with things like changing formatter, scale, annotations. Flint kinda want to hide this aways (so Flint sets them on behalf of the agent or the user). But yeah, GoG is still the foundation for expressivenss.
For other parts, it's quite common in visualization and diagram etc libraries to have json, since they are easily portable in different rendering contexts.
But when building it in a tool that serve end users, we are starting to see that a 80% success rate in generating good looking charts can become a big issue. We experienced this when building some data analysis system. So the reliability, expressiveness, and costs (in terms of time and tokens) are hard to achieve all together with directly generating matplotlib, vega-lite etc.
So we essentially designed the langauge as a trade-off across the three, by moving some decisions to the compiler to reduce generation cost while maintain good expressivenss.