Hacker News Reader

Comment by tayo42 | Hacker News Reader

Ai can work through all of those questions in number 2 and design around it.

I was just doing that yesterday designing a game demo lol

I have also tried developing games with them. I find them very weak at these things.

Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.

They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.

My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.

That's for code,I'm not talking about that.

Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.

You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.

The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.

> The point is that AI is aware of constraints and can manage a round it.

Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.

I meticulously described the netcode structure for my tactics game and it's simple enough to just send over on web sockets and fable was still giving me an endless hydra of sync issues

Are you an AI?

Your right to push back on that...

> Your right to push back on that...

Human test passed. AIs make giant flashy mistakes. They don't fuck up basic grammar.

I have found it intriguingly difficult in my own tests to make LLMs deliberately fuck up grammar in any way similar to how humans do. I suppose this is actually an architectural limitation.

Out of curiosity, I just tried this:

  write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency

and the result was a pretty good simulation. I tried less insulting prompts first, this did not produce a good simulation.

Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.

Redditor is too easy.

Conveying the errors typical in a Raise The Colours group on Facebook, that's the hard stuff.

I'll take your word for it, I think I'm in zero groups on Facebook, I only even look at the feed when I see a notification in Messenger; and I barely use Messenger, too.

> They don't fuck up basic grammar.

Sometimes they do. From Gemini 3, some AppleScript:

  if something then
    if something then
      if something then
        stuff
      end if
    end if
  </if>

Each time I asked Gemini 3 to fix the syntax error, it acknowledged it had screwed up the basic syntax, and then emitted a new version of the code. With another XML closing tag somewhere. Then it was embarrassed, apologetic, etc. to see it had made the same mistake repeatedly. Eventually it output several attempts per turn as it kept checking its own work in-turn, seeing that it had made the same kind of mistake and tried again, this time definitely fixing it, proudly and confidently presenting me with the correct, final, definitive version. With another 'end if' or 'end tell' replaced by an XML closing tag that it couldn't see until the next turn.

After about 10 turns the problem went away.

Reply on news.ycombinator.com