Hacker News Reader

Comment by tasty_freeze | Hacker News Reader

Are these generated by doing normal text to (pixel) image generation, then doing a raster to svg conversion? Or are LLM models actually capable of generating SVG directly?

I can imagine that working for simple things: make a red circle with a blue line under it, because there are many simple examples on the web.

But generating complex SVGs directly from text seems like it wouldn't work at all. The examples on their home page are well into the "too complex to generate by piecing together bits of svg examples seen during training."

View on HN

Definitely raster to svg conversion. The giveaway (besides complex SVG generation being too difficult, as you say) is the splotchiness of the images, since the vectorization doesn't work super well with gradients.

You can prompt ChatGPT to generate SVGs. It unsurprisingly doesn't work well.

I once asked it to create a SVG of Earth and it create a green ellipse on top of a blue circle.

Maybe it knows earth is flat after all?

With multimodal abilities, I wonder if it could eventually check a raster of the SVG it created and then make iterative improvements that get a better result than vectorizing a raster from stable diffusion.

Yeah unfortunately differentiable SVG rendering is still something that is very difficult to accomplish with diffusion models. I looked into this last year and found VectorFusion [0][1], which appears to be using a similar technique to this app - generate the raster image and then apply a vectorization tracer. This leads to the blobbiness you see, given the way the diffusion model generates the image and the tracer applies to the output.

There have been a few attempts (i.e. this recent one [2][3]) to attempt to fine-tune the parameters of the SVG with image segmentation, before comparing rendered outputs with a CLIP model. This is promising but the search space for vector images is just so huge that so far you really need to start with an existing image as a basis rather than starting from scratch. Interesting area of research!

0. https://ajayj.com/vectorfusion/ 1. https://arxiv.org/abs/2211.11319 2. https://intchous.github.io/SVGCustomization/ 3. https://arxiv.org/abs/2309.12302

Yes, if you look at the paths it's rather obvious.

Reply on news.ycombinator.com