Hacker News Reader

Comment by mdp2021 | Hacker News Reader

mdp2021·

By the way: I am intrigued by that «I built the scraper by pasting example HTML into Claude» - Simon picked Claude for this specific task.

Is anyone maintaining benchmarked or informal (anecdotal, empirical) lists of what are the strong abilities (and maybe weak ones) per LLM?

View on HN

I've been defaulting to Claude 3.7 Sonnet for a while now as it produces Python and JavaScript that fit my personal tastes, but to be honest I'd expect any of the leading models (GPT-4o/4.5/o1/o3, Gemini 1.5 or 2.0 or 2.5, probably the larger Llamas) to be able to do this well.

Dumping a chunk of HTML or JSON directly into a model and saying "write something that converts this to format X" tends to just work in my experience.

Been using OpenAI for most my api use in production and just started with Anhropic. Having to grind up customer tiers once again realy makes it difficult to experiment as 'Tier 1' hit it's limit after 15 mins. And no, I do not want to 'contact sales' or wait weeks when I'm merely checking out initial viability. Just let me buy and use my available credits.

i assume you know the leaderboard, which at least shows the strongest ones in a specific area:

https://lmarena.ai/?leaderboard=

Thank you, I did not remember the "Category" selector ("Hard Prompt", "Instruction-Following", "Math"...).

Descriptive blog page at https://blog.lmarena.ai/blog/2024/arena-category/

I don't know if anyone's doing that but Claude's superiority at coding (currently, amongst non-reasoning models at least) is widely known.

Agreed. Using Claude Sonnet 3.5-7 with Aider produces significantly better output than any other models I have tried.

Reply on news.ycombinator.com