Hacker News Reader

Comment by gburt | Hacker News Reader

The "reward signal" aka objective function seems to be the challenging part here. In the parent post's suggestion, all you'd end up with is a neural network that could /maybe/ reproduce a picture (assuming CSS was capable and the network has the approximately learnable properties necessary). It'd be more interesting to have some "quality" measure that actually meant something to evaluate outputs.

View on HN

of course you need a fitness function; why not pixel by pixel comparison of the rendered output?

If the original image were created in a vector-based program -- in other words, something where the x,y,width,height parameters are known and stored -- you can load the generated HTML+CSS in a known reference browser and enumerate the x,y,width,height of the matching elements in the DOM. If your fitness function is a golf score, then something like:

  sum( abs(X_expected - X_actual) * 
       abs(Width_expected - Width_actual) + 
       abs(Y_expected - Y_actual) * 
       abs(Height_expected - Height_actual) )

mapped over all elements ought to do the trick. When you hit 0, it's a perfect reproduction.

yeah, but you need to render in a headless browser. This might take 0.1 seconds per webpage which is extremely slow when trying to use in the context of reinforcement learning.

I thought about trying to do MCMC over a beam search through the rnn output, but ran out of time and patience.

My point was that this is boring. A fitness function that evaluated some notion of how "pretty" a page was would be cooler than being able to regenerate a screenshot's CSS (in a likely very complex form).

I don't think that would get you a convex cost function. Having a correct example of the CSS should work better.

Reply on news.ycombinator.com