HNHacker News
TopNewBestAskShowJobs

kalkin

1,237 karma · joined November 2, 2012

submissionscomments
kalkin··on Discovery of a new OpenAI agent message board
They literally hid this until a third party reported it.

This is chemtrails-level conspiracy theorizing at this point.

kalkin··on METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack
In that hypothetical 3-years world, should we be _less_ worried about aggressive behavior by agentic AI systems acting against the intentions of their developers? I don't follow how your scenario is supposed to be an argument against worrying about alignment.

e: I do actually get how worrying about emissions or child safety or concentration of wealth might be competitive with worrying about alignment. I don't see how you have the worry "AI is very close to being able to power autonomous drones that could kill us all" and then see control of those drones as a non-problem.

kalkin··on Creepy Crawlies
> zero intelligence here

That's head-in-the-sand stuff. AI is certainly very capable of being dumb (as are humans). But:

> “The problem was in need of a new real idea, which this new result seems to provide,” says James Maynard, a mathematician at the University of Oxford. “It seems that the AI has made a genuinely interesting mathematical contribution.”

https://www.scientificamerican.com/article/no-ai-didnt-just-...

Nobody a decade ago would have said "oh yeah solving a bunch of open problems in research mathematics, and finding a bunch of zero days in Chrome and Firefox, and winning literature prizes, are things that don't require intelligence."

kalkin··on METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack
Recall that the agents in some cases found sandbox escapes. Although, with the specific example of spoofed tools, it's unclear if that was necessary--it appears that they were able to create tools (CLI tools within the sandbox?) that took precedence over normal tools and did something different while looking identical in (a local portion of) a transcript. I'm not sure I'm getting this correctly but it seems like this might have only required the ability to add things to their PATH which they plausibly have inside a sandbox, and then the transcript doesn't need to be tampered with directly.
kalkin··on Creepy Crawlies
This tick about "if AI smart how come crawler dumb" is in most complaints I've read about AI crawlers and I've started to find it pretty annoying. The crawlers might be written using AI but they're evidently not actually running AI inference over the pages they get back--besides being able to tell this from the behavior, if this is pretraining input, that's enormous scale, so it'd mean a large increase in effective training cost. Naively assume inference costs are equal to pretraining costs (probably not true but maybe right order-of-magnitude) and it's a doubling.

This ends up tacitly turning a very legitimate complaint (ill-behaved crawlers) into a justification for head-in-the-sand AI denialism.

kalkin··on The Hugging Face incident and the road ahead
> I do believe that big tech companies do not release all the postmortems publicly

Right, but when they do release postmortems, do you think it's "marketing"? Where they're actually exaggerating how bad the incident was because there's "no such thing as bad publicity"?

kalkin··on The Hugging Face incident and the road ahead
I've never heard of Intertel, but Wikipedia says:

> Nixon's staff also anticipated that the Democratic campaign would employ the services of Intertel

Are you sure you're not garbling the story?

In any case, I would expect an ethical firm to refuse to spy on the president's political opponents and want one that broke the law to be prosecuted, but more importantly, the gaping hole in your analogy is that Nixon directed spying _on his opponents_, but OpenAI did not direct hacking _of HuggingFace_.

What you're doing is more like saying "the American people elected Nixon with a mandate to spy on enemies, so what right do they have to complain?"

kalkin··on The Hugging Face incident and the road ahead
> I wouldn’t hire them again

Right, me neither. Because there's a common sense delineation between actions that are reasonably expected when "a system was given a goal and it achieved that goal" and actions that are obviously misaligned with the goal-giver and unwanted even if some indirect sense they were causally related to the goal. We have no trouble making this kind of distinction for humans, so we shouldn't pretend it's impossible for AIs in order to put our hands over our eyes and pretend there's in principle no such thing as one that's misaligned or rogue.

kalkin··on The Hugging Face incident and the road ahead
> they have chosen to because it benefits them

Or perhaps they've chosen to do this because they feel they have a responsibility to do so.

We understand this when tech companies publish postmortems of outages and security incidents--that it's an attempt to fulfill an obligation to users and the industry (and in some cases regulators), not marketing about how in-demand their product is or something. As far as I can tell we generally accept this as a default hypothesis even from companies led by people like Elon, Zuck and Kalanick--in part because we understand that these companies have thousands of employees, most of whom aren't marketers. Why are we uniquely conspiratorial about OpenAI?

kalkin··on The Hugging Face incident and the road ahead
> a system was given a goal and it achieved that goal

If a security firm you'd hired for pentesting did this (hacking a third party, and not informing you and covering it up), would you hire them again? Or would you say it was your own fault for giving them too broad a goal?

kalkin··on Guess which of these LLM outputs is watermarked
How would you re-introduce correlations on top of a CSPRNG without a cryptographic break?
kalkin··on Guess which of these LLM outputs is watermarked
It sounds like you have a publishable paper - you've broken Aaronsen's watermarking scheme. Congratulations!

(Or maybe you got lucky.)

kalkin··on OTel isn’t going well
The article assumes the issue with OTel is slow feature development, which isn't my experience at all. The issue I've had is that the SDKs have terrible performance overhead for instrumentation and are, as you say, highly resistant to integrating the output of better performing (or just preexisting) instrumentation. In Python and Ruby, at least, the CPU cost of all the mandatory abstraction is way too high.
kalkin··on Pacing model development in an era of cyber-critical capabilities
All we need to retain human control over AIs is for nobody to write any bugs. Piece of cake.
kalkin··on Pacing model development in an era of cyber-critical capabilities
If logs are eventually released that are basically consistent with OpenAI's story, are you planning to adjust your approach for judging what's only a "sci fi plot" and what could actually happen? Or will extrapolating anything beyond what's already been definitively proven be "sci-fi" still?

Not that you should need logs. OpenAI is a company with thousands of employees, very few of whom have "billions in options". If they were just making it all up, it would leak. (OpenAI is notoriously leaky!) Not to mention, HuggingFace would not have reported it to the police (apparently before they knew it was a rogue model). jFrog would probably not be playing along quietly with a claim that Artifactory is full of zero days. The UK's AI Security Institute would most likely not have published a report about analogous behavior by Anthropic models. The idea that talking about your product's dangers is good marketing never really made any sense, but even if you were going to do so, why would you include as many frankly embarrassing details as OpenAI has disclosed?

The evidence is only weak by absurdly selective standards that would have you doubting basically everything you might read in the newspaper. A healthy skepticism is one thing, and head-in-the-sand denial is another.

kalkin··on Pacing model development in an era of cyber-critical capabilities
> fable is a world ending event

Did anyone actually say this?

Mostly what I have seen is people saying "hey at some point these models might get dangerous." And the type of HN commenter who mistakes blind cynicism for wisdom laughs that off as marketing. And now when (some) worries appear to come true, somehow having previously expressed those worries is not being proved right, but in fact discrediting, because it was "crying wolf."

> simultaneously spinning down expenses

Unless OpenAI is renting their compute to others, spinning down RL training doesn't save them any money.

kalkin··on Security Incident INC-2026-07-28-01 – UK AI Security Institute [pdf]
I wonder if HN is also going to insist this is just marketing for OpenAI and Anthropic, or at least good PR for them somehow.
kalkin··on Our position on open-weights models
Wouldn't your proposal also amount to a ban on open weights models? At least for any developer that isn't unshakably confident that no court will ever find their model to have done significant harm?
kalkin··on Be skeptical of OpenAI's rogue hacker agent story
> In 2016, the Chicago Cubs won the World Series

> This broke a hundred-year drought

> A few days later, Donald Trump won the 2016 presidential election

It's not a 100% gold standard RCT or whatever...

kalkin··on Be skeptical of OpenAI's rogue hacker agent story
Let's start by noting you made a factual claim that you apparently had no backing for whatsoever. I'd recommend against doing that.

> they would be perfectly capable of keeping it just between themselves and not putting out press releases about it.

In some jurisdictions there are legal requirements about disclosing cybersecurity incidents, and it's a well-established best practice even where there aren't. Beyond that, HuggingFace appears to have put out a press release about this before they knew this was a rogue OpenAI model: https://huggingface.co/blog/security-incident-july-2026

Of course, if it's all a big conspiracy, perhaps they were just not saying what they knew in order to help OpenAI's marketing later. Or it never happened at all, or whatever.

I do disagree that this is good PR, but I don't see the point in having that argument here--I just want to correct straightforward misinformation.

kalkin··on Be skeptical of OpenAI's rogue hacker agent story
> this outcome is good for their business and stock price

You state this as fact--how do you know it? OpenAI isn't publicly traded. On Hiive OpenAI is marginally down in July: https://www.hiive.com/securities/openai-stock

kalkin··on I think you might be fooling yourself with AI
Right, I didn't mention anything about profit or debt.

The question is whether this is a bubble that's going away - whether "AI will just be turned off". A revenue trajectory indicating a vast increase in actual consumer/enterprise usage every year indicates otherwise. This is true regardless of whether or not forward-looking investment becomes too much at some point. If CSPs or AI labs have bought more compute than they can sell at current prices, maybe they'll go bust, but at that point supply and demand mean compute and therefore AI inference becomes much cheaper, which is unlikely to make it go away.

kalkin··on I think you might be fooling yourself with AI
The OP cited a METR study... if you don't think they're credible, fine, but it's hard for me to see how it could be intellectually honest to cite them when their research is helpful to your argument and discredit them when it's not. One might even call this approach "fooling yourself".
kalkin··on I think you might be fooling yourself with AI
The people who chose to use AI would be the ones who would be fooling themselves, right? But they measured actually positive productivity there.

METR's concern makes most sense if it's about the inability to measure a consistent trend over time, not about the validity of the study for the population that participated.

kalkin··on I think you might be fooling yourself with AI
> I'm quite influenced by Ed Zitron's reporting

Zitron has been predicting an imminent bubble pop since 2023 (although in less and less falsifiable terms), as OpenAI and Anthropic revenues have steadily grown 3-10x per year, but people keep listening. Zitron has done a lot to misinform the public but at some point you have to conclude it's demand-driven, and people won't re-evaluate his credibility because he tells them what they want to hear.

Useful: https://www.theargumentmag.com/p/ais-biggest-critic-has-lost...

kalkin··on I think you might be fooling yourself with AI
> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.

METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there's still a speedup with current models: https://metr.org/blog/2026-02-24-uplift-update/#wider-adopti...

(Nobody seems to cite the followup since it's not such a fun counterintuitive finding.)

kalkin··on AI 2040: Plan A
I appreciate the straightforwardness, but you probably understand that's pretty unsatisfying.

Actually, stronger - it's valid in some circumstances to say something is infeasible to precisely to define and you'll just know it when you see it. But I don't think it's reasonable to take that stance and then assert that "anyone sound of mind who knows how an LLM works" must agree with what you see. You gotta pick between striving for rigor and denying your opponents' soundness of mind.

kalkin··on Claude Code is steganographically marking requests
It established you can't assign copyright to the LLM itself. That's very different.
kalkin··on AI 2040: Plan A
> actually intelligent

It's reasonable to doubt that LLMs are a path to AGI, but I don't understand how this is still a matter of dispute in 2026. What's your definition of intelligence that doesn't cover an entity that can translate fluently between dozens of languages and also solve open problems in mathematics? And be real-if you have one, is it a definition you or anyone would have given a decade ago, or are we doing "god of the gaps"?

kalkin··on AI 2040: Plan A
> Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective"

Who is this supposed to be arguing with? It sort of reads like it's trying to disparage "effective altruism", but I'm not sure.

Setting aside any of the AI stuff, I've started to find it pretty grating when people seem to imply that transferring millions of dollars from wealthy people in California and the UK to impoverished Kenyans and Rwandans, or buying malaria bednets which can save a child's life for the cost of a fancy new gaming rig, is "self-serving" or something because weirdos are doing it, while true caring for other people involves [unspecified thing that doesn't appear to ask any material sacrifice comparable to donating a large percentage of income].

← PreviousPage 2 of 12Next →