Comixify: Transform video into a comics
comixify.ii.pw.edu.pl
comixify.ii.pw.edu.pl
From this video[1] (one of their given demos), it generated this result[2]. From this other video[3] (another of their demos), it generated this result[4]. All used the default settings.
Was frankly disappointed. They mention “neural style transfer” and whatnot, but the results looks only slightly better than what you’d get by applying default photoshop filters from a decade ago, probably longer. ImageMagick can also achieve something similar[5].
The extracted frames, including their sizes and positioning, don’t look like anything special. The final layout was the same for both tries, meaning they’re probably not doing anything to detect/convey a better sense of the story. In that sense, the result is as much a comic as any other random sequence of photographs (comics don’t need to be drawings) divided into squares.
It’s a neat demo, but I’m failing to understand why it is part of a larger academic paper. Seems like the whole effect could have been achieved over a decade ago with a bash script glueing ffmpeg and imagemagick.
[1]: https://youtu.be/CvvAftMZYKM
[2]: https://imgur.com/a/rbrQBbx
[3]: https://youtu.be/pvAhRcUofDk
[4]: https://imgur.com/a/kZm2M5w
[5]: http://www.fmwconcepts.com/imagemagick/cartoon/index.php
Thank you for your interest in our demo.
Regarding the points you raise in your comment:
1. Layout
You are right that we use static layout for all comics we generate. Changing that has a high priority on our todo list, we are currently working on more dynamic layout generation that takes into account what we know about the story and which frames are more important than others.
2. Style transfer
The method we use gives the most pleasant results of all obtained with various style transfer methods applied to comics - see our previous work on comparison between them here: https://arxiv.org/abs/1809.01726.
3. Story sense
(>probably not doing anything to detect/convey a better sense of the story<) We use Video Summarization model in our pipeline, however as mentioned above, we use static layouts and therefore this is not always apparent.
Overall, this demo is only a starting point. We are working on adding automatic speech recognition and text generation to introduce dialogs into our comics. And we have many other improvements planned. Stay tuned and if you have any other comments feel free to send us an email to any of the addresses from the paper: 243678@pw.edu.pl, 280814@pw.edu.pl, 261406@pw.edu.pl, t.trzcinski@ii.pw.edu.pl.
BBC iPlayer for Kindle
google cache: https://webcache.googleusercontent.com/search?q=cache:c8dVMl...
edit: my mistake, it has the full PDF there