Hacker News Reader

Comment by janvdberg | Hacker News Reader

The really interesting thing here is that the creator of this language (betaveros) is using this specific language in the Advent of Code contest where he is currently number one on the leaderboard: https://github.com/betaveros/advent-of-code-2022

Which is an interesting take on the concept of domain specific language.

View on HN

I think it takes a lot more than a language, no matter how efficient to get top 10 in AoC - its more about being able to parse the problem exceptionally quickly, or rather recognise practically from memory the class of problem that the day is asking for and be halfway through coding it before you have even finalised the specifics.

I managed to get 47th on one of the days once... and that was after solving it in under a minute.

> and that was after solving it in under a minute

That depends a lot on the day. Towards the end it takes longer. Today's 47th place solved the problem in 11:54

Consistently being able to grasp the problem very fast, and then write a solution is what impress me the most.

If you check the results for the events the last couple of years, you see the same set of persons being up at the top 10-20. Basically solving hundreds of problems. Problems that span a quite a large set of CS, math problem domains.

I thought the leaderboard was dominated by automated GPT-chat bots.

Did they get banned or couldn’t they keep up with the puzzles?

GPT only got a good placement on Day 3 Part 1. It seems to not have been very useful in any other days (except for people maybe using Copilot).

After about day two, ChatGPT didn't produce anything useful anymore iirc; at least when asked to write the solution in Haskell.

Interestingly, it still produced internally consistent solutions with valid Haskell code, they just didn't have much to do with the given problem. And I feel like that sums up one of the main issues with ChatGPT quite well: if you scrolled through StackOverflow and saw a really polished solution for something, you'd assume it was mostly correct, because for people, 'visible effort' put into code usually correlates with correctness/ourselves. But ChatGPT always produces really polished solutions really confidently, even if they are complete nonsense.

I’ve had that signaling discussion here a few times. With a person, usually they’ll give you some indication (explicitly or not) that they’re uncertain of the correct answer, or they’ll have a relevant track record to help you evaluate their honesty.

No such luck with ChatGPT.

People are confidentally wrong all the time, and ChatGPT does have a relevant track record to help evaluate whether it’s apparent confidence is justified (at that track record, as is true for many people, is “highly mixed, so you shouldn’t implicitly trust it”.)

People were asking chatgpt multiple times, and then taking the most common working answer. Guess that's kind of a method to estimate it's confidence.

> if you scrolled through StackOverflow and saw a really polished solution for something, you’d assume it was mostly correct

IME, that should, at best, be a very tentative assumption for StackOverflow.

Heck, for me it is also a very tentative assumption when I see a very polished solution in official documentation from lots of providers (AWS, I’m looking at you.) Though in that case I assume it was correct at some point in time, for some (possibly not publicly released) version of the software/service.

Yup, in other domains ChatGPT will confidently give answers that sound vaguely plausible but miss out key things a human would definitely highlight, or just get the specifics wrong.

There's a video of Harstem I think, following build orders ChatGPT came up with for Starcraft II. ChatGPT doesn't say "I have no idea how to play this game" instead it confidently offers advice that is... not good. Harstem wins at least one game anyway, but you know, not from the great advice just from being a good player.

or a way to get a k or g implimenation.

Reply on news.ycombinator.com