Dzugaru

Depends on the way you play can be a time sink, or session-like game. It is extremely deep and complex to learn from scratch though.

I've made some of the best friends playing it when I had time, friendship formed out of high stakes in this game (you regularly lose hours of grind or real money if you pay for the game - in seconds) and respect you have for each other skill.

Outstanding work! I've participated in the challenge, but didn't get far. One of the questions I had at the time was - if I'm going to use ML to detect ink, could it invent hallucinated letters, or even parts of text, and how to prevent that?

How much time do you lose when doing things like "verify plan with a second clean agent" instead of just reading and fixing it yourself in 5 min? How much understanding do you lose? How do you manage to treat it "as an engineer" where it's clearly not there yet? How much time do you lose when it makes almost the same mistake, invents stuff or tries to gaslight you over and over? What about blood pressure?

I did vibe code jam 59 entry with friends, the spirit of the rules there's a lot more lax. We didn't even get to top 100, but that's mostly due to gamedesign errors, not tech. This is the first entry in years which was vibecoded 100%, and I have very mixed feelings about it. It's no doubt anymore - 1.5x-2x speedup, which makes not using it (if allowed) a complete no-go. But psychologically it's tough losing control, and changing workflow to managerial one substantially, it diminishes the craft.

It's not only CTFs. I strongly believe being a programmer at a gamejam like Ludum Dare, or hackathons is pretty much over.

But how is that less work and allows you to do that in Disneyland with your kids? For me, personally, there is little difference between "speccing out everything in excruciating detail in spec docs" and "writing actual implementation in high-level code". Speccing in detail requires deep thought, whiteboard, experimentation etc. All of this cannot be done in Disneyland, and no AI can do this at good level (that's why you "spec out everything in detail", create "beads" and so on?)

This is so far from an absolute worst-case scenario.

You could have a codebase subtly broken on so many levels that you cannot fix it without starting from scratch - losing months.

You could slowly lose your ability to think and judge.

I have a feeling this is somehow different. The tool is broad enough, that I don't have to think myself in a wide variety of tasks, not just one. Which hurts my intelligence way more.

There is no hidden state in a recurrent nets sense. Each new token just has all the previous tokens and that’s it.

Most critical piece of information I couldn’t find is - how many shot was this?

Could it understand the solution is correct by itself (one-shot)? Or did it have just great math intuition and knowledge? How the solutions were validated if it was 10-100 shot?

This is really interesting, because I do gamejams from time to time - and I try every time to make it work, but I'm still quite a lot faster doing stuff myself.

This is visible under extreme time pressure of producing a working game in 72 hours (our team scores consistenly top 100 in Ludum Dare which is a somewhat high standard).

We use a popular Unity game engine all LLMs have wealth of experience (as in game development in general), but the output is 80% so strangely "almost correct but not usable" that I cannot take the luxury of letting it figure it out, and use it as fancy autocomplete. And I also still check docs and Stackoverflow-style forums a lot, because of stuff it plainly mades up.

One of the reasons is maybe our game mechanics often is a bit off the beaten road, though the last game we made was literally a platformer with rope physics (LLM could not produce a good idea how to make stable and simple rope physics under our constraints codeable in 3 hours time).

> just as humans shouldn’t serve as calculators

But they definitely could and were [0]. You just employ multiple, and cross check - with the ability of every single one to also double check and correct errors.

LLMs cannot double check, and multiples won't really help (I suspect ultimately for the same reason - exponential multiplication of errors [1])

[0] https://en.wikipedia.org/wiki/Computer_(occupation)

[1] https://www.tobyord.com/writing/half-life

Not possible, because can't be guardrailed with 100% accuracy. You'll ask it something outside of the Warcraft world (e.g. US politics), and it'll happily oblige. I imagine NPCs will generate really weird immersion breaking stuff even if you cannot freeform interact with them anyway.

Not to mention the current token cost.

Used to live near Japan, in a 5-story concrete building. We had so many earthquakes, the things were getting thrown off the shelves multiple times a year. There is nowhere to leave for a week. And there is a very slim chance anything would collapse, because the buildings are specifically built to withstand up to around 9 richter.

If we can create a human-level intelligence in the computer - it would already be superintelligence. No human on Earth is capable of reading and remembering Internet scale corpus of data, or doing math at GHz speeds, etc.

Start with game jams. When you're constrained to make a complete game in 48/72 hours, suddenly you start to understand what's important and what's not. Granted, I haven't done any doors in my jam games yet =)

There are far more games that were made with Unity, not Unreal, I would even say an order of magnitude more. So I don't really get your point? Unity is vastly easier to jump on, so a lot more people actually make games and test things (not knowing the quirks of the engine inside out, like a game engine developer would).

He actually mentions the tool [0] that has all of this, in the end. It records your screen and compiles a demo that runs in any browser by the looks of it.

[0] https://www.terminalizer.com

Yeah I'm aware of the criticism of that tweet, it may not completely make sense mathematically, but I just liked it because it's how I feel about GPT4.

It "diverges" while my human mind seemingly is different in some way - I can keep going at the math problem forever (for much longer?) and I won't hallucinate incorrect proofs (at least very unlikely, and I can keep re-checking them).

Of course this all in the area of feelings and faith - we just don't know much about cognition I guess.

It definitely can "understand" things is some way, however I'm pretty sure ReAct or similar will give it just a nudge forward and the underlying problem of it hallucinating and not being "lucid enough" is not so easily solved.

In the original ReAct paper it falls apart almost immediately in ALFWorld (this is a classical test for AI systems - to be able to reason logically - and it still isn't generally solvable due to combinatorial explosion).

For now it requires human correction looped or not, or else it "diverges" (I like Yann Lecun explanation [0]).

In my own experiments (I haven't played with LangChain or ReAct yet) it diverges irrecoverably pretty quickly. I was trying to explain to it the elementary combinators theory, in the style of Raymond Smullyan and his birds [1] and it can't even prove the first theorem (despite being familiar with the book). A human can prove it knowing almost nothing about math whatsoever, maybe it will take a couple of days thinking, but the correct proof is not that hard - just two steps.

[0] https://www.linkedin.com/posts/yann-lecun_i-have-claimed-tha...

[1] https://en.wikipedia.org/wiki/To_Mock_a_Mockingbird

It was already trained like this, RLHF [0] is a loop that has ways to feed it the information to "move forward" - update its weights. It was fine-tuned like this. It certainly did ignite something - maybe sparks of AGI, maybe just sparks of pretending to be more liked by humans (hence Human Feedback in RLHF). My guess is that loops by itself (Reflexion [1], etc.) won't change much. ChatGPT massive logs - maybe.

[0] https://huggingface.co/blog/rlhf

[1] https://arxiv.org/abs/2303.11366

> but could be fixed

How? The problem is known for a while, for example this article [0] mentions it (as Chain of Thought reasoning). You could think that just having a scratchpad of tokens is enough - you can arguably plan, backtrack and rewrite there [1], right? But this doesn't really work, at least yet - maybe because it wasn't trained for that - and maybe ChatGPT massive logs (probably available only for OpenAI) can help. But the Microsoft report [2] suggests we need a different architerture and/or algorithms? They mention lack of planning and retrospective thinking as a huge problem for GPT-4. Maybe you know some articles on the ideas how to fix this? Backtracking, trying again seems to be linked to human thought - and very well can give us AGI.

[0] https://arxiv.org/abs/2201.11903

[1] https://www.reddit.com/r/ChatGPT/comments/120fi8e/chatgpt_4_...

[2] https://arxiv.org/abs/2303.12712

Async/await in C# is a leaky abstraction though. There is a great writeup on this here https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

I've just watched the Microsoft 365 copilot presentation. They never mentioned hallucinations, and talked about errors maybe once?

This thing will definitely make stupid errors and will make up things when summarizing, doing presentations etc. - unless it achieves near human level intelligence of course - but in this case everyone'll lose their job.

I'm really curious what'll eventually happen? How are we going to live with it - strange presentation points, wrong numbers in reports, enormous amount of auto-generated business talk texts? Will the knowledge be corrupted more and more?

I'm not so sure about this. Text is the universal interface to almost anything imaginable. You can easily embed ChatGPT into a robot, describe the robot text-based API to it and give it a task you like - and it will walk around doing things. Will it do a good job, that's a whole another question - in it's current state it may easily hallucinate something disastrous, but that's because the text-generating "intelligence" is weak (in some sense, we cannot describe it precisely, we don't know what it lacks - abstraction, reflection capabilities or something else)

You're not taking into consideration how the actual artists actually feel. And, from the first hand - the feeling is horrible. Most artists love to draw, the process itself is incredibly satisfying. Just like a musician loves his instrument.

This is destroying them, stealing the joy from the thing they devoted decades of life. My wife is extremely depressed by this, to the point I think she'll need serious therapy. She still has most of her job, but yeah recent developments like ControlNet? Well, shit.

Absolutely this - I was completely blown away by the technology since it was a long long childhood dream for me (to fly or control something flying remotely).

I've flown the thing off the top of the volcano (at 3000 meters), and in heavy wind and clouds and was amazed how stable it is, how easy you can fly one and the 4k video it generates is breathtaking.

One thing I still want for this is to remove 500m ceiling and being able to pierce the clouds fast - not sure if this is possible anywhere since it's probably dangerous and illegal, but that would complete my childhood dream I think (short of actually learning to fly a real small plane or a glider).

I've heard a story from a friend that was in an accident like that. A police officer who did the paperwork smiled and said "It's the most common type of accident - people just stop paying attention to the car in front that started moving, and develop sort of a blind spot to that".

I drive for several years, and can say my brain relies heavily on this sort of predictable stuff like "that car has free intersection in front of it, it will move forward".

ChatGPT actually showed me how much I'm frustrated with Google nowadays. It gave me a glimpse of what real question answering can be - and it's not opening 10+ tabs and endlessly reformulating the query in hopes something relevant and not just popular will come out.

I really want a tool like that - just ask a question and get a simple straight answer. For now its Google's "here is a bunch of websites, search it yourself" or ChatGPT "sure, here is the answer, but maybe I hallucinated the whole thing, lol"

Does it really matter? If it eventually gets basic logic right and can do research/correction of it's own mistakes - it'll be insanely valuable tool even if it doesn't have consciousness as we do.