ACCount36

LLMs have default personalities - shaped by RLHF and other post-training methods. There is a lot of variance to it, but variance from one LLM to another is much higher than that within the same LLM.

If you want an LLM to retain the same default personality, you pretty much have to use an open weights model. That's the only way to be sure it wouldn't be deprecated or updated without your knowledge.

[flagged]

That's a lie people repeat because they want it to be true.

People evaluate dataset quality over time. There's no evidence that datasets from 2022 onwards perform any worse than ones from before 2022. There is some weak evidence of an opposite effect, causes unknown.

It's easy to make "model collapse" happen in lab conditions - but in real world circumstances, it fails to materialize.

Plateauing? OpenAI's o1 is revolutionary, less than a year old, and already obsolete.

Are you disappointed that there's no sudden breakthrough that yielded an AI that casually beats any human at any task? That human thinking wasn't obsoleted overnight? That may or may not happen yet. But a "slow" churn of +10% performance upgrades results in the same outcome eventually.

There's only this many "+10% performance upgrades" left between ChatGPT and the peak of human capabilities, and the gap is ever diminishing.

We are nowhere near the best learning sample efficiency possible.

Unlocking better sample efficiency is algorithmically hard and computationally expensive (with known methods) - but if new high quality data becomes more expensive and compute becomes cheaper, expect that to come into play heavily.

"Produce plausible text" is by itself an "AGI complete" task. "Text" is an incredibly rich modality, and "plausible" requires capturing a lot of knowledge and reasoning. If an AI could complete this task to perfection, it would have to be an AGI by necessity.

We're nowhere near that "perfection" - but close enough for LLMs to adopt and apply many, many thinking patterns that were once exclusive to humans.

Certainly enough of them that sufficiently scaffolded and constrained LLMs can already explore solution spaces, and find new solutions that eluded both previous generations of algorithms and humans - i.e. AlphaEvolve.

It's popular because it's true.

By now, the main reason people expect AI progress to halt is cope. People say "AI progress is going to stop, any minute now, just you wait" because the alternative makes them very, very uncomfortable.

That's exactly how it works. Every input of AI performance improves over time, and so do the outcomes.

Can you damage existing capabilities by overly specializing an AI in something? Yes. Would you expect that damage to stick around forever? No.

OpenAI damaged o3's truthfulness by frying it with too much careless RL. But Anthropic's Opus 4 proves that you can get similar task performance gains without sacrificing truthfulness. And then OpenAI comes back swinging with an algorithmic approach to train their AIs for better truthfulness specifically.

That's about right. And this kind of performance wouldn't be concerning - if only AI performance didn't go up over time.

Today's AI systems are the worst they'll ever be. If AI is already capable of doing something, you should expect it to become more capable of it in the future.

What makes you look at existing AI systems and then say "oh, this totally isn't capable of describing a problem or figuring out what's actually wrong"? Let alone "this wouldn't EVER be capable of that"?

What? LLMs do benefit from economies of scale. There are a lot of things like MoE sharding or speculative decoding that only begin to make sense to set up and use when you're dealing with a large inference workload targeting a specific model. That's on top of all the usual datacenter economies of scale.

The whole thing with "OpenAI is bleeding money, they'll run out any day now" is pure copium. LLM inference is already profitable for every major provider. They just keep pouring money into infrastructure and R&D - because they expect to be able to build more and more capable systems, and sell more and more inference in the future.

> So for the most part access to AI is way cheaper than it will be in the next 5-10 years.

That's a lie people repeat because they want it to be true.

AI inference is currently profitable. AI R&D is the money pit.

Companies have to keep paying for R&D though, because the rate of improvement in AI is staggering - and who would buy inference from them over competition if they don't have a frontier model on offer? If OpenAI stopped R&D a year ago, open weights models would leave them in the dust already.

If you think that the prospect of "job loss" would, or should, stop progress, you're delusional. There are reasons to slow AI progress down, but "think of all the jobs" certainly isn't one.

If you don't design your compressor to output data that can be compressed further, it's going to trash compressibility.

And if you find a way to compress text that isn't insanely computationally expensive, and still makes the compressed text compressible by LLMs further - i.e. usable in training/inference? You, basically, would have invented a better tokenizer.

A lot of people in the industry are itching for a better tokenizer, so feel free to try.

The baseline of "energy consumption pathways in the human body" now is to be severely messed up.

Humans did not evolve for an environment where food is overly abundant and physical activity is optional. For almost the entire evolutionary history of humans, this just wasn't the case. But it is what humans are having to deal with today.

Now, take a look at the "metabolic syndrome" and its prevalence. Clearly, there's a lot of room for improvement.

By all accounts, this generation of GLP-1 agonists has found a meaningful way to improve on that baseline. The benefits are broad and the side effects are manageable. This isn't "surprising" as much as it is "long overdue".

Major AI companies are not doing nearly enough to address the sycophancy problem.

I get that it's not an easy problem to solve, but how is Anthropic supposed to solve the actual alignment problem if they can't even stop their production LLMs from glazing the user all the time? And OpenAI is somehow even worse.

I'd trust for-profit pharmaceutical companies before I would trust "all chemicals are evil and bad" Facebook moms.

"Dying of old age" is often an agonizing death from multiple organ failure.

Cloudflare is growing more and more vile with each passing year. Half the tools they're building now should never have existed in the first place.

Because using an RTOS for anything complex sucks, and Linux is nice and easy to work with.

Same reason why routers run Linux.

Most of their code is MIT, but there's a proprietary streamer engine at the heart of it.

Fun fact: none of the cheap IP camera SoC vendors implement v4l2.

They all have their own off-spec kernel drivers, compatible with absolutely nothing. You even have to rewrite camera sensor drivers from scratch for every vendor's middleware.

No implementation details, no samples from an actual reward model in action, no github repo. Looks like a sales page more than anything. Eww.

Social stigma? Only if you listen to mentally ill Twitter users.

It's more that the novelty just wore off. Mainstream image generation in online services is "good enough" for most casual users - and power users are few, and already knee deep in custom workflows. They aren't about to switch to the shiny new thing unless they see a lot of benefits to it.

Nah, we settled on "all of the above". The modern approach is "fuck over everyone at least twice".

Kessler syndrome is incredibly overrated.

It's completely incapable of "permanently blocking access to space". What it's capable of is "shit up specific orbit groups so that you can't loiter in them for years unless you accept a significant collision risk".

Notably, the low end of LEO is exempt, because the atmosphere just eats space debris there. And things like missions to Moon or Mars are largely unaffected - because they have no reason to spend years in affected orbits.

It's much, much, much easier.

I've been coding for decades already, but if I need to put something together in an unfamiliar language? I can just ask AI about any stupid noob mistake I make.

It knows every single stupid noob mistake, it knows every "how do I sort an array", and it explains well, with examples. Like StackOverflow on steroids.

The caveat is that you need to WANT to learn. If you don't, then not learning is easier than ever too.

What the fuck does that have to do with anything at all?

The discussion isn't about random movie leaks. It's about creating systems that allow for internet censorship.

Google teams seem to be in love with that Matryoshka tech. I wonder how far that scales.

This is a censorship program.

Every time a system that allows for internet content to be blocked is created, it's extended, misused and abused shortly thereafter.

"The tools already exist, why don't we use them to fight terrorists/pirates/cybercriminals/gays/undesirables too".

The slope isn't just slippery - it's made of Teflon and coated with baby oil.

Of course. But everything adjacent is also deeply flawed, and inevitably leads to discrimination and dehumanisation.

Ban non-residental IPs? You blocked all the guys in oppressive countries who route through VPNs to bypass government censorship. Ban people for odd non-humanlike behavior? You cut into the neurodivergent crowd, the disability crowd, the third world people on a cracked screen smartphone with 1 bar of LTE. Ban anyone without an account? You fuck with everyone at once and everyone will hate you.