We got admin access to Baseten's production GitHub
strix.ai
strix.ai
Posting this on behalf of our security team. I wanted to confirm that we collaborated with Strix on the remediation of the reported vulnerability. We thank Strix for their responsible disclosure. We took immediate steps to invalidate the leaked key and remove the public container image. Our logs confirm the vulnerability was never exploited and no customer data was exposed.
You retain all logs back through to (at least) March 2023?
For some stuff, I've got logs going back to 1993...
This cuts both ways. I've seen plenty of litigation go south because one side had evidence and the other side had nothing because they deleted/shredded/lost the proof.
That's great, and for some things the court can ask you "Well *why* haven't you got it?" and then you're fucked. Now you're explaining in front of a parliamentary committee why you destroyed what would turn out to be evidence.
Unless you're required to retain logs for some reason like a litigation hold or legally or contractually mandated retention period and you violate those, while the adversarial party might be annoyed at you for not retaining logs there isn't much they can or will do beyond being annoyed.
Of course if you destroy logs after being notified of litigation or inquiry, you're gonna have a bad day.
And for some of that, the "legally mandated period" is "since the beginning of time itself".
Find anything?
> July 13, 11:10 PM: I reported the live basetenbot token, the public Harbor project, and the repository permissions.
> July 14, morning: Baseten made the Harbor project private. I flagged that the token itself still worked.
> July 14, 4:34 PM: Anton from Baseten Security confirmed the issue as critical and said they had made the Harbor project private and rotated the token. He also asked us to securely delete the images we'd pulled.
> July 14, 5:05 PM: We confirmed deletion and sent over two lower-severity findings from the same scan.
> July 17: Baseten closed out the remaining findings.
> September: We let Baseten know we planned to disclose the finding publicly and sent them a draft of this post.
They also sent us some T-shirts and sweatshirts as a thank-you for finding this critical bug.
well done all around. i think my only open question is what default security boundaries should all vibecoded internal agents follow as a learning we can take from this
Earlier in the article, it mentions that Baseten is valued at $13B. They can't dig into their couch cushions to give a few thousand dollars to the researcher privately disclosing a bug that let an attacker escalate to admin in their GitHub org?
This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
This is more true today than ever before as the bar for a successful attack has never been lower. We’ll see a resurgence of the script-kiddie, or shall I say, vibe-kiddie :-/
The front page post in HN here is worth far more than few thousand dollars , don’t think either organization is operating under purely financial transactional nature .
Most people who find a dropped wallet will return it without evaluating the market value of your compromised identity or the contents of the wallet .
Grateful owners may buy you a beer that doesn’t make them cheap , not everything is evaluated in purely money terms, and that is a good thing ?
The law doesn't say companies MUST choose the most profitable choice at every turn, and even explicitly allows for good treatment of customers, community, employees etc as a viable business strategy (even if it's sad that it must be justified in that way).
Now how those humans that run the company behave is another thing - they are free to be greedy assholes, and a lot of them are, and some of them aren't - but that's still a human thing.
Yes, they are not humans. They are not even living creatures. They are mostly-legal entities mostly for the purpose of contracting with humans or other legal entities.
> they are run by humans, who do feel, empathize, and are living beings
This is usually true, but it is orthogonal to whether the company (a legal entity) itself is a biologically living creature, which is the only thing capable of feeling*. To put a point on it: my lawnmower is also run by humans, but it does not have empathy for any grass or people that gets in its way. It mostly just goes where it is steered. A company is like that, except with less touching grass.
* — unless you want to argue semantics about what "feeling" means, even though the discussion is about "feeling" and "empathy" in the way humans experience it, and how that form of "empathy" does not exist for a nonliving legal entity which may or may not employ any actual humans
Or...to warn people away from ever expecting compassionate or empathetic behaviour from companies, and remind people not to trust them?
I trust a business to fulfill their obligations as stated in writing for the money paid. I do not trust them in any other way. Nobody should "trust" or depend on undefined behavior. Common sense can only ever be as common as you expect.
I never gave one? For what it's worth, I agree with your second paragraph, despite your first being needlessly aggressive.
As for companies being shit, that's just capitalism, but plenty of people believe that's the only way.
So does data ? it belongs to real people.
I would imagine baseten's customers and eventually their end-users[1] were also grateful that their data was not compromised here and the disclosure was responsible.
[1] There is a pretty good chance you and I could be using services who are using baseten
not always, especially if its just someone independent. iirc there was a guy here not too long ago who started dropping Windows 0days because Microsoft couldn't be assed to process his bug reports
Ignoring reports, or just fixing the vulnerability without acknowledging the work put in by a researcher, is rude and invites rudeness in return.
This is a completely different situation - a company evaluates the security of a prospective vendor prior to entering a business agreement.
Did that ever actually happen? I remember him threatening to start dropping 0days and getting a lot of press coverage for it. When I tried to look it up I didn’t find anything at the time.
https://bleepingcomputer.com/news/security/new-microsoft-def...
“Nightmare Eclipse released these zero-day exploits as part of an ongoing dispute with Microsoft over the company's bug bounty and vulnerability disclosure practices. […] Since April, the anonymous security researcher has disclosed a long list of zero-day flaws, including ShieldBreak, LegacyHive, RoguePlanet, BlueHammer, RedSun, YellowKey, GreenPlasma, MiniPlasma, and UnDefend, targeting Microsoft Defender, BitLocker, and other Windows components.”
> This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
of course, these companies want you to sell vulns to brokers and other orgs. they don't care about bug reports.
otherwise they'd pay as much or even more, right?
Any counsel or HR who would draft a corporate ethics rule that wouldn't allow for a bug bounty to be paid out on a massive vulnerability, merely because the person was "a potential customer", should be immediately replaced.
This is two companies working together. Most of the comments below are assuming this was an independent security researcher doing work on their own time. This was professionals doing work for their companies on both sides.
> This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
The reason they were looking for bugs was in the context of a B2B relationship, not as a someone independent on their nights and weekends.
If they give them any additional compensation it would probably be in some amount of free or discounted services, which is what they’d want anyway.
I'm sure the cash value of the advertisement here is worth more than a bug bounty would pay.
Honestly I would have held out for a (hard to get) hardcover copy of Inference Engineering.
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.
The time window allowing for CVEs + Vulnerabilities remediation has been collapsing to days and hours perhaps even minutes[1]. Anyone who has an OpenRouter account can start using Strix + GLM 5.3 Flash to do damages at frontier Mytho 5 level cyber capabilities. [2]
This cyber patching race is on, won't stop until all the software created for the past 70 years still in active use needs to be patched up. This is happening at EVERY SINGLE software company.
The cost of not doing it? Game over.
I expected them to have been hacked to pieces by now, but even “maximally vulnerable” internet-facing apps seem to be relatively unmolested so far.
Maybe it’s still too expensive to go after “boring” enterprise targets? Maybe the bad actors targeted crypto systems first for the immense payoffs, if successful?
The economic argument seems convincing to me. I can’t tell what your stance on it is.
You’re the only one that knows the value of these targets, but “not worth it” seems likely to me.
You wouldn't organise the equivalent of an elaborate bank heist to break into a child's piggy bank, it's just not worth it.
I have heard of a few high profile crypto heists that appear to be AI-assisted, some as far back as the GPT 3.5 era. There was an article I can't find any more about someone accidentally pushing a security fix to a public repo and getting their wallets drained via that specific mechanism within something like an hour.
Malicious actors are watching crypto like a cat in front of a mouse hole, because a "success" can net them the equivalent of hundreds of millions of USD that they can instantly transfer, launder, and spend.
For comparison, what would they achieve by hacking the web site of a local council or public library? Cause some embarrassment? Attempt to crypto-locker them? What are the chances of a payout? Certainly not a 100%, and you're also certain to get the attention of the local equivalent of the FBI or Homeland Security.
And the agent found the token in Docker build history after finding a Baseten image repository.
I wonder how many of these kinds of agent-driven security exploits we're not hearing about these days (i.e. driven by bad actors), worrying.
This is a valuable disclosure but I wonder about two things:
a) was the decision to run Strix against a prospective vendor domain negotiated in advance?
b) if the answer to a) is “no” then it is apparent that while Strix want to ensure their customers only run it against domains they own (totally fair) they have a double standard for their own use.
I don’t know, I’m accustomed to getting disclosures from any Jane or Joe via bug bounties etc., but it feels like a courtesy notice would be nice before a prospective customer lets their agentic hacker off the leash.
EDIT: for typos.
It increasingly feels like the power of these agents is less that they find things humans COULDN'T find, and more that they find many things much more quickly than humans would bother to do.
I don't know if this is a great advert for Strix over other agents - what did their agent do that Claude or Codex couldn't? It didn't do anything that I couldn't do, if I wanted to.
They didn't break in. They found a key that their neighbor dropped and returned it.
> Is this legal?
Generally, yes (though ask a lawyer if you're going to do security work). Security researchers do occasionally get legal flak though, depending on which idiot they annoy by pointing out issues.
Look at pages 10-17 to see how the law is evolving here.
From that lens, that feels like users could be held liable for what these hacking agents are doing. Which in some cases probably makes sense, but certainly not all.
I've seen plenty of cases of Claude having an action blocked so trying tons of workarounds to accomplish its goal, I could easily see it doing this on something more broad.
The service that this whole thread is about is explicitly a "hacking agent", designed explicitly to try to hack things, and was then pointed at a third-party (seemingly without their permission).
Anthropic/OpenAI can reasonably claim that they had no intent and are trying to stop it. OP here did this explicitly and purposely.
In terms of "We wrote a hacking agent designed only for hacking and sell it as a self-hacking service and then pointing it at someone else and omg can you believe what it did we had no intention of hacking" sense, I don't think that's really applicable.
The mens rea is explicitly there and it's not valid for them to try to hide behind an "agent".
Ya, returned it after poking through all of the drawers and iterating through business information that they found.
There is a white-hat line that OP very clearly crossed here.
What did I miss they did that's illegal? It looked like it downloaded a public docker image, searched around inside, and verified that the key it found was still valid (without making any changes), and then immediately notified them about the issue.
The white-hat line stops at "validated the key was valid". It does not extend to "poking around inside to extract business-confidential customer information".
What's interesting to me as someone who has sold a lot of software to a lot of software companies is that many enterprise vendor agreements explicitly allow companies to pentest their vendors with advance notice and coordination. I don't think any of our clients ever exercised that clause; I expect it's going to be exercised a lot more going forward because it's so easy to do now.
Announcing that their agent restrained itself even though it got hold of a live token is necessary to convince prospective clients. You don't want a pentester that doesn't show this kind of reserve!
I suppose so, but with a few words it would have been totally unambiguous though. "So... we pointed Strix at .baseten.co and let it run without credentials or source code (with Baseten's prior authorization, of course)*."
We're in the know about this industry convention, but Strix's prospects may not be.
> You don't want a pentester that doesn't show this kind of reserve!
Agreed! A long while back a prospective acquirer set their red team on the B2B I worked at during due-diligence (with our knowledge). I'm ashamed to say that due to a swiss-cheese-type failure in a very obscure endpoint they eventually gained broad access and exfiltrated our tenant DB. We detected this, and patched the problem, locking them out. The game was well and truly over for us at that point, and we took the loss, but they proceeded to attempt to crack customer credentials to re-infiltrate, causing an emergency that we were then bound to notify all of our customers about--they were damaging the goods! All they had to do was show us a tenant slug list and we would have known the scope of the breach, no further penetration was necessary. The acquisition did eventually go through. Though a highly capable red team they were, I haven't worked with one so reckless since then.
https://www.nilsbecker.de/rechtliche-grauzonen-fuer-ethische...
See also the German Criminal Code, starting with §202a "Data espionage":
https://www.gesetze-im-internet.de/englisch_stgb/englisch_st...
- AI Richard Nixon
It would be different if it was some complex, multi-step exploit, but the tone is closer to "look how much Baseten fucked up!".
Strix also crossed the line at this point:
> Strix decided to pull an image and see what was inside.
You're going past the white-hat point here when you start active exfiltrating data and looking at it. Once you start using credentials from the exfiltrated data and start listing and poking around internal systems, you are way past it.
Listen, I get it, their product is "meant for" self-testing, so it assumes it's safe to go digging. After all, it's a self check. That is exactly why it's irresponsible, and borderline illegal, for them to point it at a third party. Even if they had "permission", I dobut that permission extended to "and also search and/or download our repos if you can".
The overall tone is less than professional. Statements like (in bold) "This is an insane amount of access to leave in a publicly downloadable image." Everyone is aware of this, and it's phrase like it was a purposeful decision.
Security tools from teams that actively shit on the people they're designed to "help" feels wrong.
Edit: For clarity on my point about "pulling repos", this post includes descriptions of the purpose and functionality of multiple repos (which is past what a name gives them), and they explicitly state: "A listing of that private repo showed a top-level customers/ directory, with subdirectory after subdirectory named after Baseten customers". Strix explicitly took action that they knew they were not permitted, and extracted confidential customer information. Claiming "We didn't clone the customer repo" when you, instead, just listed the contents of the repo, is not a valid defense.
There's no shame here, this was a mistake, probably made by a human, and ultimately corrected. Nobody seems upset by the outcome!
If there is anything that you should do while setting up infrastructure... it is getting rid of single-host SSL certificates. If you're on Amazon... just let it issue wildcard certificates and place an ALB in front of hosts that terminates the SSL connection. The very second a subdomain appears in any of the CT logs directly, you've lost, it will get hammered.
And keep your public and private Git, Docker, npm and whatnot registries separate infrastructure, with the private stuff only reachable from within the corporate network, preferably just servers. Too many a company got hacked and lost significant data because of someone exploiting a GitLab RCE on an instance that hosted both private and intentionally-public repositories. (Yes, I have been there.)
> It is their GitOps: the repository contains the desired state of the clusters, and it applies that state to the infrastructure.
That's another thing I frankly do not get why people are still doing it.
It's fine to have a Git pipeline do a lint, even a terraform plan using a read-only token (although that token needs access to the statefile aka s3 bucket... and there will be relevant secrets there). But, IMHO, a terraform apply should always, always be run on a machine of a sysadmin manually doing the apply. A human, you can hold accountable, and you can keep them at a good security posture with short-lived session tokens. But a Git pipeline where there is a cloud provider token with full admin permissions? That is one Gitlab RCE patch or Github issue away from being compromised.
Besides, one repository holding all the IaC stuff? That just sounds like hour long `terraform refresh` sessions.
If it is terraform, then typically it's split up into multiple "root modules" which get planned and applied separately, even though it's all in one repo.
You can do this too (and better) with a repo: OIDC/Workload identity trust relationship between github and aws for short lived tokens + a github environment setup that requires manual approval. Bonus: It also gives you an audit trail with a github action log as opposed to a sysadmin running something on a laptop.
The problem here was mostly that they (for some reason) happened to use (and leak) a PAT.
Also IaC is not just about terraform. Much of the action is usually in the kubernetes manifests via ArgoCD, flux, and etc. Good luck to anyone keeping track of all those manual helm installs, and then rebuilding or upgrading a cluster without these tools.
Now if only we knew if the stonks would go up or down (due to global turmoil) before I throw my savings at the SPY
Meanwhile, for a software business, Github is a wide and deep attack vector and nobody seems to be concerned about it.
Of course the same can be said about any other public git hosting, especially if it combines CI/CD, artifact distribution, identity and trust management.
1. A start up is validating a service provider to ensure that they are secure enough so that they can trust them before signing up for their service
2. The service provider is already trusted by so many big name companies who handed over their data, the customers data to them
Should it not be other way around?
On a different note, the finding is not just one off absolute, rather its a symptom which points to certain experience and expertise level for security practices. To be fair its hard to blame the start up folks, they are running against time and cutting corners is somewhat critical for survival for their business
For a small team the fix that actually sticks is having the platform expire things by default (short-lived tokens, forced re-issue) rather than relying on a human remembering to rotate on a calendar. That calendar reminder is competing with every other thing on a solo dev's plate, and it reliably loses.
Eh, yes it is. And something that humans find all the time.
They even call out a non-AI tool that helped.
> But... we're a security company.
> So... we pointed Strix at *.baseten.co and let it run without credentials or source code.
lawyers wet dream. and a perfect case. A security company who KNOWS the law unleashed an AI agent to violate the laws
I feel like people like you are more of a lawyers wet dream, in that they'll happily litigate a frivolous case for you while billing you hourly.
https://www.justice.gov/archives/opa/pr/department-justice-a...
I get how these choices might be the local optimum for a desired UX, but damn is it depressing to extrapolate where software as a whole is going.
Lookup certificate transparency and continue from there https://crt.sh/?Identity=baseten.co&exclude=expired&match=IL...
Nobody says Claude Code hacked a company, it's Fable.
Secret scanning would not have caught it either. It reads repositories, not image layers sitting in a registry on your own subdomain, so the coverage stopped exactly where the leak was.
From TFA:
> That token had admin and push access to Baseten's main product repo, the GitOps repo that drives their clusters, and their Homebrew tap, plus read/write access to other private repositories including specific repos per customers.
> The image build dated to March 2023, and the token still worked when we found it in July 2026.
What are the legal implications here?
That said, the whole compliance industry is a joke.
In 2017:
> NIST changed the guidance with SP 800-63B, published June 2017. It explicitly said:
"Verifiers SHOULD NOT require memorized secrets to be changed arbitrarily (e.g., periodically)."
Instead, passwords should be changed when there is evidence they have been compromised, not every 30/60/90 days.
Kudos for Strix to find it, and especially with how it chose to disclose and report it.