I am a PhD student and currently writing a blog post on how Deep Learning can be used in combination with vector graphics.
Do you have any ideas / remarks on this? What are the general options of applying Deep Learning to vector graphics (as input and/or output)?
If you are planning on training a generative model on vector graphics data, in a sequence-to-sequence style (like sketch-rnn), you might find it difficult for an encoder to capture all of the spatial elements in a coherent manner. One way is to also (as the other commenter pointed out) rasterize the input vector image and feed it into a convnet to extract features, and get the decoder to also use those features. This has been attempted in this paper [1] where they extended sketch-rnn to have a convnet encoder and showed better results.
But if you want to just have a lot of fun, try to train a plain vanilla "char-rnn" model just on SVG text files and see what it generates. The results might look more interesting than you would have initially imagined. Kyle McDonald has tried this before on a dataset of raw twitter SVG files [2].
Good luck with your blog post, and please share when it is out!
[1] Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories https://arxiv.org/abs/1709.04121
[2] Emoji generated with char-rnn and the Twitter twemoji svg files. https://www.flickr.com/photos/kylemcdonald/24308833950
I think you are totally right. As far as I understand, CNNs rely on the correlation of neighbouring pixels. And so directly applying them to vector graphics won't work.
However, vectorisation has massive downsides as well.
I will try to summarise it in some article. This had nerd-sniped my brain and so I need to write it down. Even if I have no solution for the problem yet.
Happy to send you a link to the draft once I have it so you can comment.