Hacker News Reader

Comment by gwern | Hacker News Reader

Yes, a generative model of HTML+CSS is definitely a direction I'm nodding towards. (I discussed it briefly in the previous section about reinforcement learning in general.) I'm still hazy on the full architecture: you need an RNN to generate the CSS, you probably want to feed in a particular HTML page to target the CSS onto, the users' browsers generate the reward signal, and you can create images of the HTML+CSS combo as rendered in a web browser and bring in a convolutional network somehow... There doesn't seem to be anything quite like this in the literature that I've come across.

(Actually, there's a surprising dearth of reinforcement learning in general. Very few blog posts or demos or introductory materials. It makes it hard to understand what is new about DQN or how the whole system works on a concrete coding level.)

View on HN

Reinforcement learning seems like overkill if you're just trying to match a single target. RL agents want to maximize reward accumulated over time, while you'd be happy to find a single good state. You could frame this as stochastic optimization (MCMC or simulated annealing) over the CSS string, with some combination of RNN and convnet serving as proposal.

The best paper award at CVPR this year (http://www.cv-foundation.org/openaccess/content_cvpr_2015/ht...) has an architecture somewhat like this: instead of a web browser they're using a 3D rendering engine, and searching for pose parameters that cause their rendered image to match an observed image. This is just Bayesian inference using MCMC, where they train a deep net to function as a data-driven proposal distribution.

If your goal were to actually get a working system, you'd probably want to do inference directly on a parameter vector encoding all the relevant quantities - heights, widths, font sizes, colors, etc. of the various boxes, etc. - that you could programmatically ground out into a CSS file. Trying to do inference over the raw text is making things artificially hard since you have to put so much work into even just getting correct syntax. Though maybe that's part of the fun. :-)

I think the optimal way to do this would be to use an RNN that can walk up and down the DOM. So it starts at the HTML tag, and chooses which path to go up and down, visiting each node. Then iterate through every style attribute and predict what value it should have.

This would create a simple way of generating CSS styles for a document, without dealing with the complicated issues of RNNs having limited memory and producing correct syntax.

Then you can use these predictions as a prior probability over what the CSS styles should be. Then you can use some kind of bayesian optimization to find the optimal settings in the least number of experiments.

The "reward signal" aka objective function seems to be the challenging part here. In the parent post's suggestion, all you'd end up with is a neural network that could /maybe/ reproduce a picture (assuming CSS was capable and the network has the approximately learnable properties necessary). It'd be more interesting to have some "quality" measure that actually meant something to evaluate outputs.

of course you need a fitness function; why not pixel by pixel comparison of the rendered output?

If the original image were created in a vector-based program -- in other words, something where the x,y,width,height parameters are known and stored -- you can load the generated HTML+CSS in a known reference browser and enumerate the x,y,width,height of the matching elements in the DOM. If your fitness function is a golf score, then something like:

  sum( abs(X_expected - X_actual) * 
       abs(Width_expected - Width_actual) + 
       abs(Y_expected - Y_actual) * 
       abs(Height_expected - Height_actual) )

mapped over all elements ought to do the trick. When you hit 0, it's a perfect reproduction.

yeah, but you need to render in a headless browser. This might take 0.1 seconds per webpage which is extremely slow when trying to use in the context of reinforcement learning.

I thought about trying to do MCMC over a beam search through the rnn output, but ran out of time and patience.

My point was that this is boring. A fitness function that evaluated some notion of how "pretty" a page was would be cooler than being able to regenerate a screenshot's CSS (in a likely very complex form).

I don't think that would get you a convex cost function. Having a correct example of the CSS should work better.

Check out Genetify for some related approaches: http://ejohn.org/blog/genetic-ab-testing-with-javascript/ https://github.com/gregdingle/genetify

Reply on news.ycombinator.com