ACCount37

It's a load of bollocks, and always was.

Modern robotics is, at its core, not a hardware problem. It's an AI problem. We have plenty of headroom in the hardware - what we don't have is an AI good enough to utilize it. We don't know the practical limits of current hardware because we can't make a robot AI that would make the hardware a meaningful bottleneck.

Today's robots don't fail at tasks because they have poor fingers. They fail because they don't know how to perform those tasks. If you put an effort into solving that? You get demos like: Gemini Robotics 2 tying a garbage bag. Take one long look at that and think of manual dexterity.

Human body is crude and suboptimal in a thousands different ways, and all of it is salvaged by advanced intelligence.

It's like the good old joke about a bear-proof trash can.

There is now a considerable overlap between the capabilities of the dumbest users and the smartest AIs.

Most censorship isn't "removal of knowledge" but "installation of behavior that prevents some knowledge from being revealed or applied in certain ways".

This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.

People in the US buy cars.

Can a household robot provide as much utility as a car? If "yes", the market gets pretty big.

That's how it works for humans. And, sure, LLMs are severely human-adjacent. But they aren't human.

An LLM can just re-ingest the entire codebase every time it wants to make a change - open it up, find the relevant parts, derive how a well structured, maintainable change should look like from them, then make that change. The way a compiler can just re-ingest the entire codebase every time it wants to make a binary.

The intermediates are a cache - a resource optimization, not an outcome optimization. Is it wasteful not to have a cache? Maybe. Can you get away with not having it? Yes.

And, the more maintainable a codebase is, the easier it is for an LLM to "re-ingest" it from scratch. Or for a human to get onboarded. There's some overlap there.

Why would anyone ever want to solve 99 Cloudflare captchas a day if you can get a bot to do it instead?

Why would "memory" be relevant to that? I honestly don't see it be much more relevant to "write maintainable code" than it is to "play chess".

That's the entire point of the "maintainable" part: someone with no "memory" of what was written where and why should be able to walk into the codebase, understand how it works, and be able to make a sensible change to it without setting things on fire.

Someone with "memory" should be able to do the same faster, but having that memory should not be a requirement.

Whether that "someone" is a freshly onboarded mid-level SWE or a mid-tier coding LLM is largely irrelevant.

At this point, I wouldn't be too surprised if LLMs just get better at "write understandable, maintainable code" than most developers are.

For now, LLMs are still bad at it, but they're already at "competitive with humans" tier of bad. I expect them to get better.

"Drop a preprint like it's a mic" is ML's new "publish".

Increasing raw model scale is one of the most consistent and reliable ways of increasing its intelligence.

In a way, training an AI is: using an algorithm to find, discover and refine other algorithms computationally. When we scale, we pour more raw inputs into finding the right algorithms for a given objective. Is it surprising, then, that we find a better algorithm for it?

As a very stupid analogy - for a given task, a small model's internal algorithm might, under its capacity and training signal constraints, top out at slightly above "bubble sort". While a larger one could dig deeper and get closer to "quicksort" internally. A naive, dirty algorithm got replaced by a more sophisticated algorithm that performs better.

Keep in mind: intelligence is very much not a binary. There's no "threshold" at which a model goes from "this is dumb statistics" to "this is actual intelligence". A 1B LLM and a 10T LLM both have some amount of intelligence. It's just that one would have intelligence that's so weak, underdeveloped, and overindexed on statistical regularities that it's very easy to dismiss it altogether. And the other might have enough of it to snipe unresolved conjectures with novel counterexamples in math. Makes it considerably harder to dismiss it outright. While the curve between the still two looks less like an abrupt jump and more like little increments building up to an avalanche.

The jump in something advanced and specific like "math abilities" can look quite sharp - but under it, there are far more generic capabilities that back it. They build up slowly to eventually enable that jump. A more advanced model makes less reasoning mistakes and recovers from reasoning mistakes more gracefully - two very generic capabilities - but once those capabilities improve enough, a whole new type of logic problem might fall to it.

I wish the users weren't so fucking stupid with the "they made the model worse" stuff.

Then that 1 out of 10 case where the model was actually made worse (whether intentionally or by mistake) would stand out instead of being swallowed by the noise floor.

You should. Feel like a conspiracy theorist when saying things like this.

Users are not reliable or consistent model evaluators. Users adapt to models - the moment they get a model that performs better, their expectations rise, their tasks get harder and their prompts get shorter.

"They made the model worse" is PEBKAC in 9 cases out of 10.

This is the "baby's first AI risk" tier of AI danger. There is no known upper limit to how powerful those systems get, and there might not be one.

Don't think "a smart guy". Think "project Manhattan and CIA put together, all in one server rack".

We're lucky to have "they can hack traditional systems" as an early warning shot. Clearly, it's wasted on many.

I would be delighted if AI companies got together and thoroughly dismantled DMCA.

That atrocity of a law was a blight upon digital freedom since the day it came to exist. DRM should never have been given any legal protection - and I would push for numerous forms of DRM to be outlawed instead.

Sounds like a way to distract from the real environmental issues with yet another "plastic straws" debacle. A fake issue that makes for some nice headlines in the media - but when you run the numbers, it obviously doesn't matter and never did.

Distract from the real issues like, you know. Clean energy projects in the US being throttled on federal level due to lobbying by the oil companies - lobbying that borders on regulatory capture. At the time when the oil prices are rising globally - and the demand for energy in the US is rising too.

Unfortunately, you can't stop an environmentalist from throwing his weight behind the cause that's being promoted the loudest. So influence groups can just push one meaningless "headline cause" after another, and keep them distracted from anything that matters in perpetuity.

Niche text. It's not impossible that there was only ever under a thousand of them printed and released into circulation.

A digital copy would exist somewhere, of course. But for us, that only matters if we can buy or download it. And for AI companies, that only matters if they can get a digital copy DRM-free and licensed permissively enough.

They are paying a fair amount. In the ballpark of $5 per book.

You know, piracy online is nice and simple - but it's kind of hard to get physical media without paying what the previous owner considers "a fair amount" to part with it.

Scanning books by taking them apart into singular pages and scanning those pages is faster and cheaper. AI training is a numbers game, so they want faster and cheaper.

What happens to the pages after? No one needs them anymore, so they get mulched and recycled.

That would be the dominant scanning method even if copyright wasn't a thing. But then again - if copyright wasn't a thing, there would be much less need to scan any physical media.

The reason why OpenAI can't just go on Amazon, buy a "digital edition" of a 2018 book and use that is that it would violate the license in ten ways, and then the DMCA laws that forbid breaking DRM on top of it.

The publishers sued AI companies for training on shadow library data, hoping to negotiate content deals for big $$$ down the line. Instead, they got analog hole'd.

Turns out that buying an old book for $5 and destructively scanning it for $25 is way cheaper than paying extortion fees to the copyright-mongers.

What I don't buy is it being "rare, precious books". First, they're not after ancient texts - they're after the books that there's still copyright on. Second, when it comes to books, "old" doesn't mean "valuable" - plenty of libraries destroy old books because there's no demand for them, and storage costs you. This is how those scanning companies get books for so cheap.

Can we not do the "AI is drinking all the water" bullshit at least on HN?

I couldn't. Amusing that archive.ph's own crawler, evidently, went through just fine.

VPN exits often "come from datacenters". And there are entire countries where the web can be unusable without.

If you don't have LaLiga in your country, that doesn't mean everyone is so lucky. Blanket IP range bans, in this day and age? Basically a proclamation of incompetence.

And then Anthropic had to go around, knock at every door and ask every neighbor for whether they have any spare compute to sell.

> Are these companies slowly gaining rot in those codebases?

Yes!

And that "yes" holds regardless of whether they use AI or not.

Let's not pretend tech debt accumulation is somehow an AI problem. Some of the world's biggest companies routinely ship code that reeks of years of rot and decay.

I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.

Symbian pioneered this bullshit. Having to sign apps with Nokia keys just to install them was a thing since the days of S60. And it all went downhill from there.

Pun employed intentionally.

Starship landed in the ocean intact and only slightly on fire!

No, that's exactly what happens. Most of the time.

There was a couple of times when SpaceX accidentally soft landed a Falcon 9 first stage into the ocean, and had it just float around. They're yet to do that trick with Starship though.

EDIT: Of fucking course.