Image Analogies using Neural Networks | Hacker News Reader

Image Analogies using Neural Networks

github.com

186 pointssyrusakbary28 comments

It's really neat stuff. The 3rd example is especially stunning.

For more projects like this, you can also check out the neural style transfer implementation in Keras: https://github.com/fchollet/keras/blob/master/examples/neura...

This is the script that OP's project was adapted from.

See also http://ostagram.ru for some very striking neural network art mashups.

the third "analogy" leaves me speechless. i wonder how disruptive this could become to the graphic design industry.

https://raw.githubusercontent.com/awentzonline/image-analogi...

unless someone goes and creates a easy to use standalone package, very little.

this guy had the right idea http://ostagram.ru/static_pages/lenta?last_days=30

but the service costs an arm and a leg to run on current cuda clouds

first to get to a standalone python or ios version takes all :D

For GPU, https://www.balticservers.com/gpu-servers

I used them for other things, and they delivered servers same day.

AWS GPU spot instances are better for something like this, and are probably cheaper.

the aws smallest instance comes at 450$/mo (about 300/mo paid upfront), the one he suggest comes at 200$/mo but the gpu seems cheaper. still one would need some sort of payout to keep it running, it's not exactly cheap and you need to shell 2k upfront

aws spot instances can cost from $0.30 down to $0.10/hour, and if you have a fluctuating load with image generation, that can be a good price :)

But they could terminate your instance at any time...

Usually, training step is the one that takes the longest and that can take a whole day. Evaluation step can be done on CPU usually almost instantly. Not sure what the utility of spot instances is in deep learning. Not to mention code complexity and Dev-Ops investments to run on very transient hardware.

Finally, you can't haggle with AWS, but you can basically name your price with private dedicated server providers like that.

the net comes pretrained. I haven't dug into the code much, but from the surface it seems it works by using the net as a scoring system for feature similarity, and then does a gradient descent on that (in very broad terms, I saw the style layer being used multiple times at multiple scales, probably to capture high and level patterns independently)

That's why I said "spot instances", which are 10 times cheaper. And you actually get one hour for free if they interrupt you (as far as I remember).

This is amazing, why is there no western version of that service?

The "original" cloud version is http://deepart.io

As for an easy standalone version, it wouldn't have huge potential, considering the inevitable GPU / computing power requirements.

eh most computer now have some good muscle. also it's an iterative process so you can show a meaningful progress bar to user even if it takes an hour or so people wouldn't complain. and for many it would take minutes - one just need to set the expectations straight.

"By increasing the server memory to 27GB I manage to create 1024px images on CPU - seven hours per image" - http://spiceprogram.org/artcorn/

And the GPU version requires amounts of video memory that generally do not exist for anything else than research or GPGPU purposes. With 4GB of VRAM you might get a 920x690 image processed in under 1 hour, using recent optimizations added to the neural-style repo.

So a standalone iOS version seems pretty useless. A standalone Windows version might appeal to people with gaming rigs but without the technical knowledge or determination to install neural-style in its current form. It certainly has potential, just not as huge as the cloud version.

If you were a serious graphic design professional might not you already have an i7 or a xeon rig with SLI already set up, and be accustomed to leaving a render running overnight?

This makes me a little sad, though : "University of Tübingen has a pending patent application for the Neural Art technology."

I wonder what google would have to say since it's built from deep dream experience. Also it was recently improved with newer tech which may very well be non infringing

Man I love all the Kandinsky ones. My favorite kind of art.

I'd love to see the result of rotoscoping with this technique.

Looking at more examples, I'm not yet convinced this technique isn't straight-up magic: http://www.mrl.nyu.edu/projects/image-analogies/flightsim.ht...

Looks like this could open a new era of videogames with synthetic textures, procedural maps and sandbox worlds.

I would absolutely love to see papers that apply techniques like this to video. I've got an interest in filmmaking and would have a great deal of fun processing video with something like this.

Naively I imagine you could pipe every frame through an algorithm like this, but it likely won't have a smooth evolution over time. Maybe that isn't a problem?

Not quite the same thing, but here's the first animation I've seen made with it: https://twitter.com/memotv/status/706817101386936320

This made me think of the PatchMatch algorithm, see e.g. [1].

It is almost as if this is a generalization of that algorithm. It would be interesting to know if that is the case (for example, it would be interesting to know if the examples from the patchmatch papers can be reproduced with this image analogies algorithm).

[1] https://vimeo.com/5024379

That's pretty cool. The patch matching in neural image analogies is based on this: http://arxiv.org/pdf/1601.04589v1.pdf

The MRF loss is patch based, and adapted from CNNMRF [1]. Since the precursors to PatchMatch (as mentioned in the PatchMatch paper) were MRF + belief propagation based, I am pretty sure it could be done with some tweaking.

These analogies seem quite similar to the "user constraints" PatchMatch allows to be set, though an explicit "be straight" constraint might be much more difficult to optimize.

[1] https://github.com/chuanli11/CNNMRF

Can someone who's more familiar with this stuff give me an idea of how to go about trying it?

Is it possible to just spin up an Amazon EC2 instance and make it work?

If so, how long would it take, and how much would it cost, to produce one hybrid image? (Let's say something small, like 640x480 pixels.)

We're not talking hours of processing time per image, right?

I've tried colorizing photos with this code (result here : https://twitter.com/matsiyatzy/status/706270684209680384) using EC2 instances. It's pretty fast, around 10-15 minutes, though you'll definitely want GPU-instances with CuDNN drivers installed (which can be a bit tedious to set up).

Reply on news.ycombinator.com