Comment by anishathalye | Hacker News Reader

Comment by anishathalye | Hacker News Reader

Hi HN! I've been frustrated with standard GUI-based design tools, because they don't match the way I think: like a programmer, in terms of relationships and abstractions. (Does this resonate with the community here?)

So I've been hacking on this new DSL for design that allows the designer to specify figures in terms of relationships, which are compiled down to constraints and solved using an SMT solver. This also leads to good support for abstraction, because constraints compose nicely.

The system is still a WIP, but it's usable enough that I made all the figures and diagrams in my latest paper and presentation with it, so I thought I'd write up a blog post about it and share some results.

This post is about how I think about design and how the programming language implements this philosophy. It also contains a number of case studies of designing real figures with the tool.

There are also some (probably useless?) but really amusing results, like what happens if you solve figures via gradient descent: https://www.anishathalye.com/#gradient-descent (the result is oddly satisfying).

I'd love to hear what you all think about the ideas in the post. Do you share any of my frustrations with existing design tools? How do you think when doing design -- does your model match mine, or is it something different? Do you currently use any design tools that you're really happy with, that I should know about?

A couple years ago, there was a constraint CSS library but the startup behind it went under.

https://gss.github.io/guides/ccss

Do you plan to add a GUI editor too? What do you think of Xcode/Figma's constraint system?

You can use something like https://diez.org/ to provide easy cross platform support. (Note it's only partially open source)

The browser and most "responsive UI" platforms like Flutter have a built-in constraint solver to process layouts. However often the language does not provide sufficiently powerful systems to express constraints in relation to each other, so for the backend you can probably take advantage of existing code when Z3 is too heavy a dependency. (I am not sure if this is possible for all layout solvers, you will probably have to prove it mathematically that the CSS (or whichever platform you are targeting) solver is equivalent or of higher abstraction level in relation to your design language so it can generate the appropriate code. Otherwise the purely declarative approach might not work and you will need JS/Dart/the native scripting language.)

This looks very nice, I've been a fan of constraint-based systems for some time.

I was surprised not to see any reference to at least some of all the previous work, such as the arc from ThingLab[1] to Cassowary[2] that led to commercial systems such as Apple's Autolayout[3]. Or the Garnet[4]/Amulet[5] systems, with lessons learned[6]. Or Feiner's Drawing with Constraints[7]

You also initially write that the alternative to constraints is to position manually/numerically, but that obviously isn't true, as you have link to the code to do it procedurally later. In the same vein, you initially claim that you can't have abstraction without constraints, but that's also obviously not true when you have procedural abstraction.

You do say that the constraint code is "nicer", and I would probably agree with you, it would probably help to focus on that difference.

[1] https://en.wikipedia.org/wiki/ThingLab

[2] https://en.wikipedia.org/wiki/Cassowary_(software)

[3] https://developer.apple.com/library/archive/documentation/Us...

[4] https://medium.com/@donhopkins/constraints-and-prototypes-in...

[5] https://www.cs.cmu.edu/~amulet/amulet-papers.html

[6] https://www.isi.edu/~szekely/contents/papers/2001/lessons-le...

[7] https://research.cs.wisc.edu/graphics/Papers/Gleicher/CMU/br...

Are you familiar with "metafont/metapost"?

Metafont was the original system for drawing fonts for TeX and used constraints to describe the character shapes. Metapost extended this to diagrams in general. They might be an interesting point of reference.

All I have to say is "bravo!". Geometric constraint solvers are vastly underused and any vector based drawing tool ought to have it.

I find it interesting that you tend towards SMT solvers instead of gradient descent solvers. Is it because you want to avoid having to specify approximate solution? CAD tools get away with this because the draughtsman is providing the rough starting point.

Also, as a further question? Is it open-source/free software? I couldn't find any links.

Thanks :)

I'm currently using SMT for a number of reasons. One issue is that I've found the constraint is often non-convex, and this sometimes causes issues. And yeah, as you guessed, the initialization is also a hard part. With some figures, I've found that gradient descent can find the right solution with a good initialization (or even with just trying a handful of random initializations), but it doesn't work every time, and I wanted to avoid relying on the user to provide a good starting point.

The right thing moving forward might be some hybrid of SMT and gradient descent, especially if an interactive GUI editor comes into the picture. SMT could provide the initial solution, and gradient descent could help when the user manipulates the figure using the mouse.

I remember wishing for something like this when I started working in Illustrator after using things like rainmeter and hand-writing SVG code for ages. Of course, I don't have the patience or dedication to implement that sort of thing.

I'll definitely give it a go if I find the time. I love the idea.

If you could make it work with LaTex you could probably find some happy users in IEEE / ACM type communities that would use it for drawing figures in research publications. However, it probably won't take on unless it's made open-source (most researchers are on tight budgets already!)

There once was a "smart" structured drawing program called Aldus Intellidraw

Basalt as it exists today already integrates pretty well with LaTeX -- I have papers written in LaTeX that have Basalt figures. The figures are rendered into SVGs, converted to EPS using Inkscape's CLI, and placed in the paper with \includegraphics.

The integration works pretty well, and it's even possible to make the fonts and sizes match (e.g. see Figure 1 in https://pdos.csail.mit.edu/papers/notary:sosp19.pdf).

This is something I very recently was wondering existed. I was thinking of creating a vector logo for a project, and was curious if vector software was capable of defining relationships like you have done here. It would allow for easily determining what ratios between elements look most visually appealing. I'll definitely take a look at this and let you know if I have any feedback.

Is there actually no other existing tools that have a similar functionality?

I have done a similar tool but based on geometric relations instead of constraints, you can see links in my profile (it's only on mac app store at the moment)

Also Haskell Diagrams.

https://archives.haskell.org/projects.haskell.org/diagrams/

I am curious what sort of technologies and math are used in that kind of program.

Gradient descent looks exactly like the effect you would get using a verlet-based physics constraint solver. I wonder if there is a deeper similarity between them beyond being iterative?

I love seeing this, and I was dying to try it out, only to find that there appears to be no such option? :(

Reply on news.ycombinator.com