Hacker News Reader

I built my own arena for benchmarking local LLMs on a suite of small projects intended to grade the models on workflows I am likely to use them for.

It looks like you're mostly 1-shotting apps and reviewing the results based on # interventions etc

Like it!!

neat! I will check it out. I'm not really using local LLMs as dont have a big enough machine for these to run with a decent tps