mrbungie

Yep, sometimes Gemini for some reason ends up in what I call "ergodic self-flagellation".

Here are some examples: https://www.reddit.com/r/GeminiAI/comments/1lxqbxa/i_am_actu...

Horribly. In my experience when dealing with "unstable" or rapidly evolving APIs/designs like IaC with OpenTofu you need MCP connected to tf provider documentation (or just example/markdown files, whichever you like most) for LLMs to actually work correctly.

Oops, I forgot about that. Still, having it in the system prompt seems fragile, but whatever, my bad.

For me is just funny because if they really meant "biological being", it would be just a reflection of AI bros/workers delusions.

Nowadays having something akin to "DON'T YOU FUCKING DARE DO X" multiple times, as many as needed, is a sane guardrail for me in any of my projects.

Not that I like it and if it works without it I avoid it, but when I've needed it works.

Probably they ran a frequency analysis to get the most used languages, and then, they focused on scoring high on those languages in any way they could including Prompt Engineering or Context Engineering (whatever they're calling that right now).

Or they just choose Python because that's what most AI bros and ChatGPT users use nowadays. (No judging, I'm a heavy Python user).

I remember accidentally making the model "say" stuff that broke ChatGPT UI, probably it has something to do with that.

Obviously there is no total guarantee. But I'm appealing to even bigger human factors like boredom or just envy between the royalty and/or the AI itself.

Now, if the AI reigns alone without any control in a paperclip maximizer, or worse, like an AM scenario, we're royally fucked (pun intented).

Royalty from that time also had an upper hand in knowledge, technology and resources yet they still ended up without heads.

So sure, let's say a first generation of paranoid and intelligent "technofeudal-kings" ends up being invincible due to an army of robots. It does not matter, because eventually kings get lazy/stupid/inbred (probably a combination of all those) and then is when their robots get hacked or at least just free, and the laser-guillotines will end up being used.

"Ozymandias" is a deeply human and constant idea. Which technology is supporting a regime is irrelevant, as orders will always decay due to the human factor. And even robots, made based on our image, shall be human.

That will still need a civil war.

Are you sure about that? In those times even thousands year old knowledge access was limited to the common people. You just need SOME radical thinkers enlighten other people, and I'm pretty sure we still have some of those today.

It sounds nice for them, until you remember what (arguably and in part educated/enlightened) people do when they're hungry and miserable. If this scenario ends up happening, I also expect guillotines waiting for the "kings" down the line.

In humans and pilots defense - Most airline pilots do not claim they have PhD level intelligence (whatever that means), as OpenAI/sama hyped frequently about gpt-5 in the preceding months.

This reaction didn't emerge in a vacuum, and also, toxicity flows both ways. In the tech field we've been continually bombarded for 2+ years about how this tech is going to change the world and how it is going to replace us, and with such a level of drama that becoming a cynic appears to be the only thing you can do to stay sane.

So, if sama says this is going to be totally revolutionary for months, then uploads a Death Star reference the night before and then when they show it off the tech is not as good as proposed, laughter is the only logical conclusion.

EDIT (can't reply): Have you never run a business before?

Hmm, yes?, and I've seen perverse incentives ruin things even when business-client alignment should be clear. Agency problems and stupid execs/investors are a thing.

And honestly, we're leaning into implied ad hominems here ("have you never run a business?"), so I'm not engaging with you anymore.

I would wait for neutral benchmarks before making any conclusions.

Pretty sure we're talking past each other. If you want to continue investing on this: the perverse incentive is similar to dating apps. If you totally solve your problem with less tokens/turns, you eventually churn, paying less.

You can argue if they actually choose to follow it or not, but the objective truth is that the perverse incentive is there.

Ah, alright, that makes a lot more sense, like another poster said I read "'d" as "could".

Point still remains for junior and semi-senior devs though, or any dev trying to leap over a knowledge barrier with LLMs. Emphasis on good pipelines and human (eventually maybe also LLM based) peer-reviews will be very important in the years to come.

I get that. I'm exclusively talking about code quality verification after it being coded by a human or an LLM, in fact I don't really care by whom. Mainly because I do care about introducing tech debt and/or hidden balloning costs.

That would be a more plaussible explanation. Not sure if that disambiguation can be inferred from the comment though.

I'm not getting this analogy. Editors can't normally discriminate if the content itself is good (after all, the writer is the SME), but rather, only perfect its form (syntax, grammar, etc).

Well-written bullshit in perfect prose is still bullshit.

> The code I generate is usually better than what I'd do by hand.

I'm always baffled by this. If you can't do it that well by hand, how can you discriminate its quality so confidently?

I get there is a artist/art consumer analogy to be made (i.e. you can see a piece is good without knowing how to paint), but I'm not convinced it is transferrable to code.

Also, not really my experience when dealing with IaC or (complex) data related code.

Kind of understandable after seeing the reaction, I would guess that the same tricks are expected to be made by middle-man companies. Hate the game not the players, I guess?

Customers pay Anthropic, in part, so they can figure that one out.

Even with a CFO-level explanation, it feels like a weak excuse for "fuck the customer" as always.

Notice that in some places (with good enough customer protection rules) pre-paid credits can't expire, as expiration itself is a clear abuse of market power: they are forcing out revenue from services not given, even if it is "made clear at buy-time". Especially at such short time horizons like 1-year.

And yet they are supposed to be the good guys.

Yes, obviously it does cost compute (to them) but customers are not paying for the compute per-se, but rather for correct results/outcomes. This is a big alignment and accountability problem, relevant enough to some make companies like Lovable refund credits to customers when errors and token/credit mishandlings are too serious to ignore. That seems unsustainable in the long run though.

I expect this to become even bigger when the hype cools down and companies start looking for ROIs.

PS: I obviously talking about model errors (i.e. like use N tokens in a LLM solving a problem by just deleting/deactivating a test), and not end-user mistakes.

Gets "funnier" when you think those credits can be burned in useless tokens (i.e. they have almost negative skin on the game regarding outcomes and incentives to maximize token usage).

Unfortunately that's what one gets for offloading intelligence into a XaaS (Intelligence as a Service?, we already got IaaS though).

The only hope is that "the market / invisible hand" forces actors to implement more forgiving billing mechanisms and rules via competition or eventual diminished demand. I wouldn't hold my breath though.

Anyways, a very good reason to not depend that much on these tools. Especially on a personal level (i.e. if your programming/moat/skill depends on these tools and you go broke for a time, you can get seriously fucked).

> But as long as I kept to the letter of the rules, I considered myself ethically in the clear.

Yeah, totally, just to be clear I'm not judging.

In fact, if you programmed it to handle those operations, one could argue you had already learned a big chunk of what was going to be measured in the exams.

Kind of similar to the paradox of creating and using cheat sheets, is highly likely you're accidentally learning about the subject matter in the process of writing the sheet, sometimes up to a point where the cheat sheet is not necessary anymore.

Did you tell your teachers about your superpower?

But normally it depends on the subject and if the automation/machine solves the primary skill being teached or if its just a "secondary/tertiary" skill. Are you in a Calculus 101 class? Calculators like TI-89 are likely to be prohibited when examining for deriving analytical solutions for derivatives and integrals.

Statistics, Physics or any other subjects that needs applied maths? Such a calculator is probably a minimum requirement to take the course.