Hacker News Reader

I'm a bit curious to see how close the solutions were.

When it couldn't solve it within the constraints, was it 'broadly correct' with some bugs? Was it on the right track or completely off?

View on HN

The code they generated and the outputs (including trace back errors) is all included, you can view them on the post itself or from hf space:

https://huggingface.co/spaces/jerpint/advent24-llm

Huh, don't know how I missed that. I was hoping for them to have done some analysis themselves.

Glancing through the code for some of the 'easier' problems Claude (his best performer), it seems to have broadly correct (if a little strange and overwrought) code.

But my phone is not a good platform for me to do this kind of reading.

I should specify that unfortunately I didn’t store the raw output from the LLM, just the parsed code snippets they produced, but the code to reproduce it is all there

Reply on news.ycombinator.com