Hacker News Reader

scarmigon We must not grant AI agents legal personhood

No one understands these things, which is why considering machine consciousness as a possibility is entirely reasonable. People who have a shallow model of how they work ("it's just linear algebra!") betray a lack of understanding.

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

Unfortunately, that's not the case in the United States. The LOC only selects around half of published books to be permanently held. The rest are disposed of (usually returning them to the publisher, donating them to a library, or destroying them).

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

The magical library in the countryside, to be painfully explicit, does not exist.

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

Is your complaint that they follow copyright law, or that they don't follow copyright law?

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

Dumpsters also don't typically come equipped with a robot scanner and network uplink built in.

Like, I really don't know what people objecting to this imagine typically happens to old, unwanted books. They don't get sent to some magical library in the countryside if unpurchased where they are carefully maintained forever (next to where Rover spends the rest of his days). They are very literally thrown into the trash.

That said, I'd be thrilled if the US government required AI companies to make them available to the public. I'd even settle for the US government making it legal for them to.

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

Relinquishing copyright does not imply any of the labor you're suggesting. It's the opposite: you're just committing not to perform the labor of pursuing legal action against someone who does digitize and freely distribute the work.

Anna's Archive, for one, would be more than happy to host at no cost to the author.

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

You've got to compare it to the alternative. Books have a half-life, and the vast majority of these books being purchased are grody, moldering ex-lib copies of books that no one has read in decades. Their other likely outcome is mulching.

scarmigon AI companies destroy physical books – let's scan rare books before it's too late

The hypothetical "heroic figure goes and buys last copy of a 1962 guide to Ford cars to carefully maintain it in an appropriately climate controlled library" is vanishingly unlikely. A ten or a hundred or a thousand times to one, it just goes to the trash. At least here it gets scanned by the AI company.

scarmigon If your agent commits a crime, who is responsible?

Yes. Otherwise, you open the door for every criminal to use "oopsie" as their defense.

scarmigon Pacing model development in an era of cyber-critical capabilities

https://en.wikipedia.org/wiki/Don%27t_Look_Up

A movie fit for our time.

You can produce detailed descriptions of the incident, verified by adversarial parties, and some people will still scream "it's a conspiracy! It's a marketing stunt!"

This is all very unfortunate--there's a meaningful chance that AI will cause unprecedented disaster, with the HF incident being just a small preview, but people would rather squawk "stochastic parrot" for the millionth time than revise their beliefs.

scarmigon Field measurements of neighborhood-scale air temperature impacts of data centers

California and Arizona use literally orders of magnitude more water for alfalfa farming than they do for data centers.

scarmigon Field measurements of neighborhood-scale air temperature impacts of data centers

> we shouldn't be building these heat generating facilitates

Jesus.

scarmigon Norway should buy OpenAI

California's biggest problems are mostly along the lines of "too many people want to live here."

I have plenty of complaints about California governance, many of them conservative coded. But if California is a failed state, it's a failed state that much of the world aspires to be.

scarmigon AI Isn't Outthinking Mathematicians. It's Out-Remembering Them

So, what you're asking for is an example of someone extracting value out of something that is fundamentally irreducible random noise?

scarmigon AI Isn't Outthinking Mathematicians. It's Out-Remembering Them

There are plenty of artifacts that, if not impossible for humans to understand, then at least no human has ever completely understood. To start with, the universe as a whole. Despite that, we are able to choose legible pieces of it to model and perform useful actions from.

This will shift your argument--that doesn't count! etc., to the point where it's by construction unsatisfiable and vacuous. And it doesn't matter: an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.

scarmigon AI has access to a vastly larger working memory than the human brain

Take something like

  (1+x*y)^3*z+y^2*(1+x*y)*(4+3*x*y);y+3*x*(1+x*y)^2*z+3*x*y^2*(4+3*x*y);2*x-3*x^2*y-x^3*z|0,0,-1/4|1,-3/2,13/2

If a thousand monkeys typed at a character per second, on a keyboard with the 23 relevant characters, it would take roughly 10^136 years for them to come up with this counterexample. Though, to be fair to monkey scenario, there's a large family of them known now, so it's not quite this bad: suppose there are a trillion permutations and similar examples that fit in this string. Then we are down to 10^124 years.

If LLMs are monkeys, somehow trained LLM weights allow them to model and prune massive numbers of universes in parallel.

scarmigon 70% of AI revenue comes from OpenAI and Anthropic [video]

> Because one day Ed will be right

Bold prediction.

scarmigon No one can afford to make Myst games anymore

Whatever the morality involved, it's an unstable norm. There can be good actors who use AI and disclose, and good actors who don't use AI and don't disclose. There are bad actors, however, who will use AI and not disclose. The entire discourse will be flooded with non-disclosers accusing each other of AI use (particularly, the bad actors will accuse everyone else of AI use, to flood the zone with shit), and there will be no way for a consumer to distinguish good from bad. Bad actors will prevail economically.

See e.g. corporate green and rainbow washing, Amazon marketplace, organic food.

scarmigon DeepMind's WeatherNext model achieves breakthrough forecasting cyclones

Imagine you've got to evacuate a hundred thousand people. That extra day is incredibly valuable.

scarmigon Responding to the next frontier of critical cyber capabilities

"The fire fighting department really just hired an arsonist so they could market their fire fighting abilities!"

scarmigon OpenAI's latest math breakthroughs commit research misconduct, experts say

You can dig through every single paper and find some citation that was missed. No one cares that much unless the result is important, and then you get bitter recriminations saying that it was stolen or plagiarized. Tale as old as time. See e.g. Schmidhuber, who has a long list of vendettas against people he thinks stole his work.

scarmigon I'm Scared a Stranger Will Call My Novel AI, So I Built GitHub for Words

Sounds like something an AI would say.

scarmigon I'm Scared a Stranger Will Call My Novel AI, So I Built GitHub for Words

The issue isn't just that cheaters are going to cheat. It's that the project will provide a positive and mostly trustworthy provenance exactly during the period of time when no one cares about or trusts it. However, as knowledge of it diffuses and people care about and trust it more, it will attract more and more cheaters. This negative feedback loop puts an effective cap on how much value people can get out of it.

scarmigon Ten advances in mathematics and theoretical computer science

Everything has limits, of course. But if you've not yet seen a deceleration, it's reasonable to expect that at the least you're in the middle of the sigmoid, not the top.

scarmigon Ten advances in mathematics and theoretical computer science

It's worth reading Marcus' first line, for the naysayers and flaggers on this post:

> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.

scarmigon Ten advances in mathematics and theoretical computer science

From a random article I grabbed of his (https://www.wheresyoured.at/subprimeai/):

"it isn't clear whether generative AI actually provides much business value at all"

"cannot seem to find a product that people will pay for, in part because the results are so mediocre"

"Last week, we got our first real, definitive glimpse of what’s around that corner that future. And boy, was it underwhelming."

"OpenAI claims that o1 “performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.” Just not in geography, it seems. Or basic elementary-level English language tests. Or math. Or programming. "

"Worse still, it's kind of hard to explain why anybody should give a shit about o1."

"o1 shows that OpenAI is both desperate and out of ideas."

"the software is not becoming more useful"

Honestly, every other line is quotable in this context.

scarmigon On the non-use of AI in my writing process

> On that current AI is a bit lacking as they don't have very good sensors whereas human brains are pretty wired up for touch, smell, sight, sound and the like.

For sight and sound, the sensory input layer for AIs is much, much richer in terms of bitrate, though is redundant.

> AIs do get linked to cameras though and could improve a lot in those directions.

Video is an interesting one, as most AI systems use the full pixel information of a scene, while human vision is much more temporal, focusing primarily on logistic deltas rather than raw pixels. That's not inherent to hardware (e.g. see event or "neuromorphic" cameras), but it's a significant difference. My more sympathetic reading of embodiment is that the relative paucity of the human sensory stream is part of what makes us work better than current AI in many domains, in that it forces us to form richer, temporal world models from a low bitrate stream of salient events instead of relying on shallow responses to rich sensory data.

scarmigon A Rant About “Technology” (2005)

I think the critical factor that makes sci-fi hard isn't known physics, exactly, but early on establishing a known set of rules and letting them play out as they will. The narrative has to respect and adjust to those rules, not vice versa.

This ends up driving most hard sci fi to current understanding of physics, with maybe one tweak. If you diverge more from that, it's difficult or impossible to establish the rules and their implications early on, and so choices about them later end up serving the narrative.

Though, a certain irony: hard sci-fi doesn't seem to offer much predictive insight about the future compared to other subgenres. The 20th century sci-fi genre that seems to offer the most insight into our current moment is probably the one with the wildest imaginings: cyberpunk.

scarmigon On the non-use of AI in my writing process

Embodiment doesn't exist, even in regular humans. We do not have direct access to reality; we have sensory inputs that are much lower bandwidth than one might expect but correlate with external events, and our brain uses them to form rich world models that are usefully predictive. Most of the world we experience is just in our head.

scarmigon Charlie Stross – On the non-use of AI in my writing process

Things that are vaguely GAN-shaped might play some role in post training. Though no one would call them GANs or identify them as the key underlying technology.