Researcher finds flaw in a16z website that exposed some company data | Hacker News Reader

Researcher finds flaw in a16z website that exposed some company data

kibty.town

552 pointsudev4096225 comments

When we released our open-source project[1], this hacker (Eva) pentested our project pretty extensively and was very professional in their disclosures. They didn't even ask for a bounty since we didn't have a program back then!

Eva is an incredibly gifted hacker and a responsible one, a16z should treat them better.

[1]: https://github.com/heyPuter/puter/

I made a similar mistake actually.

We used a nodejs cms called apostrophecms that had an admin panel called global settings.

We used that for managing api keys to our auth server.

We only found out a few months in that it was outputted in the html source code. They did this so it was available to JS, of course it was in their docs. So not blaming them. We glossed over it.

Annoyingly we paid a reasonable amount of money for a pen test with one of the big consultancy companies but they also didn’t see it.

I ended up finding it and checking the logs seems like it wasn’t abused but it was shocking and a big leak

> it was in their docs. So not blaming them. We glossed over it.

You should be blaming them. You can't excuse dangerous behaviour by documenting it. I feel like this lesson should be known by now.

If the panel setting was specifically for API keys, then yes, that's on apostrophecms.

If it's just some kind of generic settings with name/value pairs, then it might make sense to expose those to the browser, and make that very clear up front.

Yeah you can define extra global settings extending the existing fields, so we used that for our multi tenancy solution. And is available on the node side of things as well as on the frontend.

We always need to do our due diligence when using someone else's project. It's an open source project, available for free.

If they weren't very clear in the docs is one thing, but it doesn't appear so. Anyway, we won't combat these types of shenanigans by assuming others did everything up to snuff. We gotta be more careful ourselves.

Why were you using a web-based content management system for secret management?

In apostrophecms you can easily create setting and content types with custom defined fields. There are quite a few good ideas in there.

Those are mostly used on the node side of things, but often for convenience also shared to the front end.

I think I'd be looking for at least a refund on that pen test. I've never come across one that was anymore than a box ticking exercise.

I've absolutely been involved (conducting, coordinating, and receiving) some high value pen tests over the years.

One problem is there is no hard definition of what is considered a "pen test". I've seen very highly reputable vendors claim essentially out of the box nessus scans as pen tests, automated burpsuite scans as pen tests.

In my own personal definition of a pen test: security practitioners may use those tools amongst others, but they generally leverage them as recon and then try to uncover pathways in from those vulns, in addition to abusing application logic and misconfiguration.

Second problem: paid pen tests have limited scope and time constraints. If the application surface is sufficiently large, that engagement may simply not be big enough to conduct a thorough test. Contrast this with Bug Bounty hunters (and attackers): they have unbounded time and resources. They can literally keep testing until they find something.. and best part, there are so many of them!

So these public bug disclosures are hard to compare to a private/paid for test. You could argue, the app owners didn't pay enough for a comprehensive test.. but the downside is: just because you paid more, doesn't mean the pen tester did a better job :( While they are high noise, I tend to think bug bounty programs are the best fit for the problem space. You end up with much deeper coverage, and a very positive ROI (even factoring in your engineers to triage the bounty reports).

Security is just box checking. Most IT work is. The deployed stack has limited set of parameters to learn and test for.

Leetcode is popular hiring criteria for a reason; that kind of code checks the “KISS/don’t be clever” and DRY rediscovering known algorithms boxes

Except in a few fields, most startups are pretty vanilla config ops and secops tasks.

Recent popularity among the working class has inflated the egos of run of the mill office workers. “Programmers are lazy” has long been waved around like a badge of honor.

Rather than Silicon Valley I’d like to see a Mad Men take on IT. Start in 06-ish with a bunch of entitled first world craft beer drunkards wasting nights on syntax art, framework wars, rise of cloud. End with Covid, launch LLM AI and a bunch of code school burnouts being laid off.

This actually sounds like a great idea for a show and one I'd watch with great interest.

Edit: don’t wanna blame apostrophe cms here, it was our multi tenant setup and misunderstanding of apostrophe that lead to this situation

Hello, I'm really sorry you had this unexpected exposure using ApostropheCMS. As you've mentioned, this data sharing was noted in the documentation but can still prove surprising.

A note for future researchers: the currently supported major version of Apostrophe no longer behaves in this way. Any data injection to the logged-out front-end would be a choice made at the developer level, specifically to avoid this sort of surprise.

That said, there are still use cases for including API keys as part of the configuration and 'content' of certain types of widgets.

For context, I am the head of design at Apostrophe and also play an engineering role.

Yeah, I didn't want to dunk on ApostropheCMS, this was our responsibility for not understanding the tech. I made another comment hoping to make that clear.

Overall it's a great & in current headless craze a unique product. V3 looks very good, but we never got that in production.

When I create a new service and add LetsEncrypt cert to server via ACME. I immediately see logs filled with junk, obviously bots searching for shitty defaults that devs might leave open. I have even seen requests for the process env file lol.

How was such vuln not found and abused in this case? a16z is very lucky or maybe it was abused and not disclosed. Researcher or bored person with a kind heart/white hat hacker mindset is the first to reach out.

a16z should be fined heavily unfortunately there is no legal framework for this type of negligence

> How was such vuln not found and abused in this case?

Maybe it was..

There might have been more value in leaving this one open than just screwing with them.

To be fair, their main site doesn't seem super interesting. Couple of those credentials, such as OKTA seem bad though.

> a16z did not give me any bug bounty on this because of the fact i publicly reached out instead of trying to reach out privately. the only reason i did it this way was because there was no available contact on their main site and the email i could find engineering@a16z.com bounced my emails

That's a clever lifehack to save your company money, by not having any way to privately contact engineering all bug bounties will have to be reported publicly which means you don't need to pay anything.

But it also teaches security researchers to sell that info next time instead of reporting.

Seriously, if anyone from a16z is reading this, all you're doing is incentivizing the next exploit to be sold and used against you.

All sorts of cleverness going on there. I'll bet they saved a ton of money on development by lowballing people on fiverr or whatever they did, and indirectly they'll also save a ton on bookkeeping when a russian ransomware group effortlessly takes them for everything they have.

Even more bookkeeping will be saved with lost business opportunities.

With all the money a16z funnelling into the Trump campaign as it is, yada yada, I’m too lazy to make the rest of the line.

Counterpoint: OP is a security researcher and couldn’t find a single human email address at one of the most well-known VC firms on the planet? LinkedIn? Twitter? Facebook friends? Come on. They’re not hard to reach if one really wants to.

(Note: I still think A16Z should have paid them.)

Exactly, if he even just browsed their website a bit he'd have stumbled across loads of email addresses that could have been a useful point of contact.

It’s more fun getting attention by doing it publicly and being the victim (security researchers love hitting the 'nobody respects us' button) than putting basic effort in.

A single email bouncing is frustrating of course, but he then posted that an easily found vulnerability existed on Twitter, while a16z:

- has a contact page page https://a16z.com/connect/ with 4x emails to their offices at the bottom (despite claims the main site had no other emails)

- links to their Twitter where DMs are open https://x.com/a16z same with instagram, FB, and linkedin, all open

it would be easy to just email all of them at once and waiting a couple days to see if it gets escalated.

Why should it be an onus on the researcher to find this information? It should be plainly provided in the first place.

Someone shouldn’t have to jump through hoops to help the company secure its resources. That is not how this works.

Trying more than one email is not jumping through hoops when it's one of the worst possible vulnerabilities hitting all of their databases/platforms. Being a research means being an adult and having a basic level of responsibility. Just like being a gun owner, it's a powerful tool that needs to be treated with utmost respect.

A lot of pentesters are just kids who are angry at the world and the poor state of security, which I get, but it's not a huge barrier to try a bit more. He would have been rewarded if he did.

A researcher should not have to “try different emails”. Period. There should be a clearly disclosed email provided by the company to report such issues. Very obviously plastered. Or just use the standard abuse@, security@, infosec@, etc.

It is by far in the company’s best interests for this to happen because the alternative is public disclosure or disclosure to black hats instead.

Anything more is jumping through hoops. It should not be the researcher’s responsibility or burden to go out of their way to help a company that hasn’t done the bare minimum to welcome white hats helping them secure their own systems.

Yes of course company's should do that, but in the real world a lot of companies don't think to do that, especially a marketing site for a VC firm.

Any dev knows what it's like having a million responsibilities, a lot of things get put on TODO lists that never get completed. Them being owned by a wealthy company doesnt mean they have a huge dev team running 247 to handle this stuff. Which is probably why such a obvious failure even happened...

Security researchers get high and mighty extremely quickly, which is immature IMO.

WTF is this thinking?

>Any dev knows what it's like having a million responsibilities,

Any airplane mechanic has a million responsibilities, and if they are not followed people fucking die. Maybe software devs should step up and take a little responsibility for their lack of action that can have consequences for their users.

Security researchers owe you nothing. If you make the path of least resistance selling sploits to blackhat groups the world will be a worse place.

The security researcher in this case worked for free to find a hole in their security, reached out via a provided email address, had that bounce, so then chose to reach out via a different messaging system to let them know that there was an issue. ALL OF THIS WAS UNPAID. They have 0 or less responsibility to this firm. The researcher was doing them a huge favor.

> Security researchers get high and mighty extremely quickly, which is immature IMO.

Immature would have been not trying to responsibly disclose this, or disclosing the hole before it was patched.

Alright then: you go to Andreessen Horowitz's website[1] and see if you can find a SINGLE email address in any of the normal places a business would list the (not-social-media) contact information. Because they did their damnedest to make sure you won't find any.

[1] https://a16z.com/

I already linked to them in my comment below

Click nav

click “how to connect with us” -> https://a16z.com/connect/

See 4 emails at the bottom for each office

See 4 links to social media pages where every single one has DMs open

Wait at least a couple business days to see if anyone replies, if no one does or it’s not being taken seriously then you can announce it publicly on social media you found something but can’t reach them

> Huge effort, I know

Okay. There’s 4 front office emails and 4 social media accounts, both presumably manned by non-technical folks.

So now you have to go back and forth just to get routed to the right place. Which may not even happen if this is the first time that employee handled a security incident.

You’re making it sound like sending the email or DM is the end of the work. That is usually far from the case.

Emailing an office manager with a company security issue would be incredibly irresponsible. They're in charge of managing the physical office and are about as "outside" as you can get in a company while still being employed by that company.

I don't think the onus should be on the researcher, and I think A16Z should have paid them. But if they actually wanted to get in touch, I'm just saying they could have.

If they're putting the effort into vuln scanning the site, they can also put in the effort to get in touch like a professional. You could just as easily say "why should the onus be on the researcher to find vulnerabilities when it's A16Z's job to secure their own site". The researcher is in this to find holes and make a few bucks (which is fine!). The job is complete when you get in touch.

> You could just as easily say "why should the onus be on the researcher to find vulnerabilities when it's A16Z's job to secure their own site". The researcher is in this to find holes and make a few bucks (which is fine!). The job is complete when you get in touch.

Presumably, the company wants to be as secure as possible. It’s in their best interest to make this process as painless as possible. A security researcher has many options for what to do with a found exploit, some far less moral than others. The company has very few, relatively. They are the ones that are limited and therefore should be doing everything in their power to ensure the best outcome, a responsible disclosure that is fixed as quickly as possible.

The best way to ensure they do this is to provide an obvious, easy to find avenue for these things. This includes reasonable, well-displayed emails (or using something like a standard abuse@, etc) and a bug bounty.

Simply put, the company is the one that should be going out of their way or else they will just have researchers either disclosing it publicly or selling the exploit for likely far more money than a bug bounty.

I understand where you're coming from, but you're using "should" a lot. Companies should do a lot of things! They should make their sites secure. They should have a formal bug bounty program. They should have security@ and engineering@ and lots of other emails easily visible. We agree.

But many don't. And a lot of things in the business world are not as they should be. And in this real world of imperfection, others sometimes need to put in effort (and be paid for that effort) to make up for the failings of companies. This is one of those cases of imperfection.

Of course I’m using “should” a lot. Because “should” clearly didn’t happen.

That doesn’t change anything. Just because a company has shitty security reporting practices doesn’t suddenly mean the onus is on the researcher to do the company’s job.

> If they're putting the effort into vuln scanning the site, they can also put in the effort to get in touch like a professional.

They did. They emailed, and when that was bounced, they used a different medium to reach out. Twitter is a place that many companies actively engage with the public.

> The job is complete when you get in touch.

They got in touch. If A16Z aren't going to respond to people via email, but they do on twitter, they don't get to decide that twitter isn't a viable communication platform.

They said they got in contact via Twitter, but a16z didn’t like that.

The issue wasn’t using Twitter. It was publicly broadcasting the existence of a security vulnerability.

This is especially egregious given that A16Z’s DMs are open.

This what you expect from VCs. I always prefer to report these incidents to GDPR authorities if user data is leaked. Then they pay the fines and some get a criminal record. Money is something VCs “print” and manipulate.

>Implying the Eu will actually do anything at all whatsoever upon reporting a gdpr issue

>Money is something VCs “print” and manipulate.

You wot m8

It is the member state authority, although EU GDPR is a Directive, is up to the member state. It doesn’t just apply to the EU, it can be UK ICO.

I have literally seen EU institutions fragrantly break GDPR

The company doesn't need a "hack" to not pay money. If they don't have a published bug bounty program then they owe nothing.

They also have contact email addresses listed at the bottom of https://a16z.com/connect, which the researcher conveniently missed.

They were looking for clout, not responsible disclosure.

Let's imagine your backpack is open.

It's polite to say thanks if someone informs you that you accidentally left your backpack open.

But in no way you are supposed to give them anything.

Even further, some people take precious things from your backpack (trying to exploit the issue) and then come back to you asking for money; claiming they are nice people. This is non-sense.

Terrible analogy. This is more like someone returning your wallet full of cash, on live TV. You aren't legally obligated to give them anything, but it sure is a dick move not to and good luck getting your wallet back next time you drop it if you don't.

Why will giving someone a cash reward mean you have a better chance of getting your wallet back in the future?

Because the next person will know there's a good chance you'll give them a cash reward, and that will tip the "immorally take all the cash" vs "return it and hope for a reward" balance more in favour of it being returned.

I would have thought that was completely obvious so maybe that's not what you were asking?

(On the other hand this is HN...)

It’s just that the analogy breaks down a bit. It’s fair to say a dropped wallet in a city is a one-shot game—it’s reasonable to expect neither the participants nor their acquaintances will ever encounter each other again; whereas a security vulnerability is closer to a repeated one—it’s a fairly small world. (Some kind of neighbourly behaviour would work better here, but then again, it’s more difficult to find a universal experience of that kind.) I didn’t misunderstand this, but perhaps GP did?..

The places you're most likely to get your wallet back in the world are the places you're also less likely to get a reward. The reward for returning a wallet is knowing you're doing your part to make the place you live in a nice place to live.

Doing free work for A16Z or any of the awful companies ruining our world is not helping make anything better.

I think A16Z and the companies they’ve funded have done a great deal of good for the world. The very web browser you’re typed your angry comment into is a technology pioneered by one of its two founders.

Being anti-VC is essential being against technological and economic progress.

I like netscape & its decedents.

Not everything that happens is progress, the world can often do without 'disruption'

You're using the wrong line of thought on the analogy here.

The value of the wallet is not the cash you'd directly lose inside of it. The value is getting your ID and cards back without them being copied by someone else, along with any other identifying information.

The value of having and up front and easy to use bug bounty system is it's easier to use then selling it off to some blackhats (hopefully). Those blackhats may otherwise scrape all your s3 buckets or somehow otherwise run up a zillion dollars of charges over a holiday with your keys.

Being cheap gets expensive.

>You aren't legally obligated to give them anything,

Acktchually, depending on where you live, you might be.

Also the wallet had "please return me, cash reward" written on it. (Bug bounty advertised)

... Did they actually steal anything or take advantage, or just touch the bag to make sure it wasn't fake? Seems more of the latter, and your analogy falls flat when the bag carrier contains other people's pii.

There are pleny of people here saying the equivalent that "not paying will only encourage people to take things from your backpack instead".

It's not the same. Figuring out a bagpack is open takes no effort. Finding a backdoor takes a lot of effort.

Not when you find it on first "inspect element". That really is the equivalent of looking through someone's window and seeing their bank information and credits cards just lying in full view of anyone who'd look in.

So you’d rather researchers reach out to black hats with this information instead? Because that’s what this line of thinking leads to.

It’s in everyone’s, especially the company’s, best interests to have a bug bounty and easily accessible security hotline. Expecting researchers to jump through hoops like contacting their offices’ front desks to get to security is absurd.

> So you’d rather researchers reach out to black hats with this information instead?

That is pretty much what they did. Posting publicly about the vulnerability most certainly meant that every hacker in the world tried (and probably succeeded) at reproducing it, all before the company had enough time to act.

So you’d rather this happen? That is the question I asked.

Because this is explicitly what happens when a company doesn’t have a good process for accepting and responding to exploits.

The onus should entirely be on the company to invite researchers to find and report exploits in a responsible way. They are the ones at risk of losing millions of dollars over an exploit.

They didn't post publicly about the vulnerability; they reached out via twitter to tell them that they had one, without giving any details about it whatsoever.

Telling everyone that there's a vulnerability is usually as bad as providing detailed steps. No one was looking, and now you've pointed them in the right direction.

> No one was looking

It's a16z, not Grandpappy's Model Railroad Museum Showcase ("Come see a photo of the tiniest steam wagon in Sheboygan!").

what do you want them to do? nothing? we've already established that they tried to make contact.

How about - go to the company's contact page, look at the email address there, and use that?

Lol what a reach

As far as I can tell, their tweet was just:

> someone from @a16z get in touch, now. its bad. security related.

https://x.com/xyz3va/status/1807330215955177937

If your email bounces, I think reaching out over social media is reasonable for a fast response.

I did the same thing with OP years ago, I tried to contact in every way possible the dev team of the largest telecom company in my country.

All channels were ignored, so I have to resort to contacting our government agencies. Luckily, one agency replied to me and had one of the devs contacted me. For this hassle I was only paid $50.

You have no idea the effort we go to report this things. So I quit bug hunting after that.

I mean, a16z should be very grateful this got reported by an honest hunter regardless of the means it was reported.

I stumbled upon a big vulnerability in an unnamed Czech ministry's web apps around January. It's now July and after trying the appropriate support email, the official "snail mail but digital", and calling various people's office landlines (thankfully they publish those in the org chart), it might get fixed this month.

If there is a next time, maybe I'll try convincing the cybersecurity bureau to take my vulnerability reports instead.

Am I blind? I don't seem to find the email address at all on that page

Only thing I can find are office mails, which looks more like a trashbin than mail which would respond. Also not where I'd look for a contact mail.

They seem to only want you to connect via social media (which is a poor choice for primary contact IMO).

i think you're missing the fact that that indeed is not a security email, and the engineering/security email i found bounced.

i had no ill intentions. stop pretending i did.

It’s hard to assume good intentions when you find the site via a set of searches that begin with ‘crypto bullshit’.

> They also have contact email addresses listed at the bottom of https://a16z.com/connect, which the researcher conveniently missed.

They have those now. Do we know they did when the researcher tried to reach out?

Edit: I decided to take a look at it myself. It does seem that that was available on June 3rd of this year [0]. (You'll have to look at the source since the archive doesn't do their animations.) It seems to be available on previous snapshots as well [1].

[0]: https://web.archive.org/web/20240603210532/https://a16z.com/... [1]: https://web.archive.org/web/20240000000000*/https://a16z.com...

[0]: https://web.archive.org/web/20240603210532/https://a16z.com/...

> not responsible disclosure.

The researcher found an email address, tried it, it bounced, then reached out over Twitter with:

> someone from @a16z get in touch, now. its bad. security related.

https://x.com/xyz3va/status/1807330215955177937

That doesn't seem irresponsible to me. Sure they could have searched the bottom of a connect page for the office emails to try, but I don't see any significant issue with what they did instead.

"an" email address, not the one on their contact page.

The email the researcher found (engineering) seems more appropriate than the office info emails (menlopark-info, ...) at the bottom of the Connect page (an actual "contact" page used to exist, but is now 404 with no redirect). I don't see anything irresponsible about trying engineering then reaching out over social media.

Why broadcast the tweet publicly instead of sending it as a DM to A16Z then?

It’s obviously not safe to publicly announce the existence of a security vulnerability, and there was no barrier to alerting them privately via the same platform.

Are their DMs open though? Can’t DM someone who isn’t following you.

> It’s obviously not safe to publicly announce the existence of a security vulnerability

Publicly showing the vulnerability would have been unsafe, but I don't think there's much harm in asking to get in touch about an unspecified security issue (not even saying that it's a vulnerability in their website). Andreessen Horowitz is a massive firm, not some tiny website flying under the radar.

> and there was no barrier to alerting them privately via the same platform

DM would have to get picked up by their social media person next time they check Twitter, whereas a directed tweet can additionally leverage networks and be escalated by people with contacts - possibly someone could give the up-to-date engineering contact email, for instance.

Either way would have been fine, really. I feel we're going over the actions of an individual researcher with a fine-comb, searching for any hint that there was an arguably better course of action, when there are multiple huge obvious mistakes from a16z.

> I feel we're going over the actions of an individual researcher with a fine-comb, searching for any hint that there was an arguably better course of action, when there are multiple huge obvious mistakes from a16z.

You're going over things "with a fine-comb". I just wrote two sentences that made a single point.

The extent to which attempted fault-finding of someone's behavior is unwarranted is not determined by the number of words. I could complain "Why break my door when the window was open!?" to the firefighter carrying me out of a burning building in nine words.

I'm generally sympathetic to what you're saying, but I also detest a16z and Horowitz personally for being the epitome of "software guy decides he's expert at everything now" and his role in the crypto bubble.

Should the hacker have tried more? Sure, maybe. Do I really care? Definitely not

Do it enough times and you’ll be known for not paying any bounties, which makes people less likely to report issues they find.

A post to HN with a query for how to get in touch with a16z engineering probably would have been fruitful.

when companies say they are “hacked”, it’s now a corporate term for “we were negligent in securing important credentials, but please shift blame to this no-name entity we called a ‘hacker’”

If you accidentally leave your front door wide open and somebody steals all your stuff, you'll also say that you were robbed.

There might be a legal distinction between "breaking and entering", "burglary", "trespassing" etc, and in a legal sense, whether the front door was open might have some impact on whether the act was illegal or not and what the consequences are, but in colloquial usage, you've still been robbed.

More like complaining when your teenager takes a break from mowing on trash day and leaves the mower next to the trash and someone takes it.

If you put all your stuff on your front porch with a sign “please take what you want” and it’s all gone the next day - then you can’t say you were robbed.

I think this is a more apt analogy to what az16 did here

There's no analog for the sign. You just put it in because without it your scenario still feels like theft (because it is) and you end up arguing against your own point.

That is fair enough, I guess it’s not a great analogy overall.

But IMHO it’s hard to feel to bad for someone (az16 in this case) who handles their arguably most valuable goods in such a manner and gets robbed.

More like if they kept their wallets in an open basket on the porch.

It's not an invitation to take it, it's just really stupid.

Yes that would have been a much better analogy.

Using those credentials is still a violation of the he CFAA, no reasonable person would think they were invited to access the systems protected by those credentials.

Yea, I'm sure the Russian/China/NK/Iran hackers are deeply afraid of the CFAA, you got them shaking dude (and vice versa when someone in the US hacks one of their sites).

The particular problem here is we think of the crime on the web in a civil/criminal manner... "People should just follow the law or be punished for a crime". This is not the internet. Regardless of what you think about the internet, it is an international war zone. If you leave the hatch of a tank open and a drone blows it up, that was you being stupid. If you leave an ammunition truck unguarded and the enemy takes it, again, that is you being stupid.

History will look back and say WWIII started on the web, but as of now it seems a huge number of people are in denial about it.

None of this at all applies to this thread. It’s true, but also irrelevant to this discussion being had.

All of this applies to this thread.

Do you cultivate vines with fruit, or do you cultivate brambles and eat thorns?

Remember white hats don't need to exist. Black hats will exist by the very nature they are parasitic and thrive where exploits exist. We can either have a community that warns you that "Hey, the stuff on your porch is going to get stolen" or we can have a community that calls their buddy when they see some stuff fresh for the taking.

A huge portion these discussions under this article are people arguing the minutia of a puddle in the lawn while a 10 meter high tsunami is rushing their way.

IMO these sorts of analogies to houses and porches don’t really work because there are just different cultural norms between websites and porches.

If there were a convention of leaving stuff on your porch to donate it, and a general assumption that when people left stuff on their porch it was up for grabs, somebody started storing their groceries there, and they were taken… they would just be stupid and not sympathetic.

If somebody just moved to a neighborhood where this was tradition and didn’t know about it, they would rightly be a little bit annoyed when the groceries they stored on their porch were taken, but really they only have themselves to blame for not understanding the local conventions.

If somebody opens up a storage company and then just put all the customers’ stuff on one of these porches, they are just dangerously, unethically incompetent. Even if there isn’t a convention of taking stuff from porches, actually. Because there are also armed gangs (nation-states) that go check out people’s porches for secrets.

If I leave other people’s stuff that I promised to take care of on the street and it gets stolen, I would be to blame.

blame isn't mutually exclusive. you can still blame the person that stole it too!

> might have some impact on whether the act was illegal or not

Only the burglary, trespassing, or B&E parts. Theft is still theft even if you leave your doors unlocked and/or open.

Well, other legal distinctions aside, robbery is taking things by threat of force.

If someone doesn't know they've been a victim of larceny until later, it wasn't a robbery.

Good analogy, from a personal perspective.

In this case, a person was yelling through the front door "Your door is wide open!" and no-one was listening.

For a 42B AUM company, at a time where running an IT operation means "use CrowdStrike so that you pass audits", leaving the front door open all night should get you fired, regardless of whether you blame hackers or not.

A website is not a house. It is nothing like a house. There is no front door. There is no lock. There is no expectation of privacy. There are only things you can access and things you cannot. There is nothing inappropriate about trying to open the bathroom window from the outside.

If I wanted to try to use such a weak analogy, the analogy to hacked is not robbed. You were only robbed if content was removed and exclusively held by someone else, which in the security world we call a ransom.

You can see how quickly this breaks down.

Pretty shitty to not even give a token amount bounty for such a broad hole

The next time someone finds their keys, they're going to find this article and commit them to a public github repo instead...

You don't want to push secrets in their raw form on GitHub, secret scanning would disable keys from supported providers.

Yea, they aren't going up on GH, they are going up on sketchy-site . ru

that's the point

they are busy writing a giant "architecture of generative AI" whitepaper. give them a pause, they are dreaming a future agentic world of half-assed chatbots.

while the world burns with botched software updates.

world is already burning with effects of climate change.

botched software updates on a Friday is just the chef’s kiss

> engineering@a16z.com bounced my emails

No surprise there.

If you could actually access their Salesforce instance, that would be very nerve wracking for founders, since usually Salesforce, etc, logs emails which may continue unannounced fundraising plans or M&A plans that haven’t been shared externally by portfolio company founders.

It would also be pretty damaging if it includes their LPs.

Collecting the keys from a public source-code of a web page is legal (and can be safely reported).

Using these keys to access unauthorized systems is a crime.

This is a major difference.

Oh no CRIME! Thank goodness that something being a crime stops people from committing them.

Thank goodness the internet isn't an international operation filled with nation state level actors and questionable companies running data gathering operations from places they cannot be touched.

Always assume your data has been stolen by an assailant in a place that's only reachable by launching nukes at them. Also assume there is some competitor on the other side of the world now using your data against you.

Please stop treating data theft like Barney Fife level candy store theft. A huge portion of the time even if you know the name of the exact person who did it, there isn't going to be shit you can do about it.

Parent comment never suggested it was legal. They said it would be bad if this info was in their SalesForce and they leaked the key, which they did.

How can it possibly be a crime? They literally gave the keys to everyone who accessed their website

You (unintentionally) drop your house key in front of your door. Now we can all freely enter your house! It can't be trespassing with the key sitting right there, can it?

Totally agree, and if you think like that, then a SQL injection is just an undocumented public entry-point ¯\_(ツ)_/¯

The fact that this VC firm didn't provide bug bounty for such a gaping hole does not instill trust.

Yes, if they can’t do web development what does that say about their ability to deploy capital?

If my endodontist can't rebuild a car engine, what does that say about his ability to perform a root canal?

Turns out, not much.

Not a great analogy. Its more like if your endodontist hired a secretary who leaves the medical records unlocked, do you really trust them to be up to date with modern dental sensibilities when the rest of their office is ran so carelessly?

Unsurprising given that the founders are Trumpists now: https://siliconangle.com/2024/07/17/co-founders-andreessen-h...

Billionaires like tax cuts for billionaires; go figure.

According to the article, the decision to back him was due to the 2025 tax plan to tax unrealized gains, which I hadn't heard of, but I'm not surprised that he wouldn't be a fan of that, given that his entire business is built on investing in companies, and that these investments on the part of founders and investors are unrealized. It does seem like it would de-incentivize much of the startup and venture capital economy.

I'm not smart enough to understand finance and so forth. So can't comment on that 2025 tax plan.

I do know that "Bidenomics", aka the torrent of federal money (CHIPS Act, Inflation Reduction Act, EPAs new "Green Bank", Dept of Defense's retooling, etc), has been a huge boon for startups.

I would have thought a group of savvy entrepreneurs like a18z would join the renewable energy and domestic manufacturing bonanza.

But like I said, I don't understand finance. So I'm sure a17z have their reasons to sit this one out.

I wouldn't be surprised if they would have been on board for most of the Biden era economic policies. I think it may have just been the possible industry repercussions from the coming 2025 tax plan that made A&H anxious, given that it could disincentivize the venture capital growth market.

They believe Trump will be best for tech. Ironic, since Trump has pledged roll back our transition to renewable energy, curtail the use of EVs, etc.

Maybe marca and ben meant Trump would be better for VCs and tech investors. Which would be true.

The HN mods changed the title to a less embarrassing one. Not surprised

Oh, my comment must have been too critical of a16z as well. I see it has been moved from top to way bottom without a score change.

That's certainly one way to offer a response!

Sincere question: how do you actually make this mistake while having the skills to build a web app of this complexity level? All the frontend and full stack frameworks that I’m familiar with try pretty hard to stop you.

Don’t mistake complexity for intelligence.

I’ve seen people make exactly this mistake with Next.js. IMO React server components is a fantastic tool for losing track of what’s exposed client side and what isn’t.

Next.js makes you prefix env vars with NEXT_PUBLIC_ if you want them to be available client side, and Vercel has warning flags around it when you paste in those keys.

It's obviously not foolproof, but it's a good effort.

That’s env vars, but not actual variables - it’s really easy (if you are not actively context aware) to f.ex. pass a ”user” object from a server context into a client component and expose passwords etc to the client side.

That's a fair point! It definitely feels easier to make that mistake, and anything where context and discipline is required is a good candidate for making some horrifying blunders :)

If you add `import “server-only”` to the file, it will fail to compile if you to use it on the client. React also has more fine grained options where you can “taint” objects (yes that’s the real name).

Yeah, the problem is that these mitigations require the developer to be context aware, ”server-only” only saves you in the positive case where you correctly tagged your sensitive code as such. The default case is to expose anything without asking. I have also seen developers simply marking everything as ”use client” because then things ”just work” and the compiler stops complaining about useState in a server context etc.

It only takes a single mistake.

A little tired because you didn't sleep well, or worried about a relative in the hospital, or you stubbed your toe that morning and it's distracting... and whoops.

Whoops I accidentally exposed all API keys ever to the public.

No really this is unacceptable for a professional, it’s even bad for an amateur.

If your processes are so insecure that a little tired breaks your whole company you done goofed.

Yes, the answer must be additional processes and procedures. That way, you’ll never make a mistake! /s

Also bizarre to frame this as “unacceptable behavior”, as if whoever is involved was in some way aware of their mistake and/or would say “this is acceptable behavior!” when confronted with it or something.

GP framed leaking all your keys at something that happens when you are tired or distracted.

This is unacceptable behaviour for a professional in my eyes.

Humans are gonna human, if you have an environment where you fail to account for this, this will happen. Reminds me of a dev dropping a production database, or the aws engineer who incorrectly entered a command and brought down s3: many things have gone wrong to even be at this point, blaming a human for behaving like a human in an inhospitable environment is silly. Effort is almost always better spent building a system which is safer to operate for the people involved.

That’s why I recommend in my original comment as well: get a better process.

The person I replied to understood it as “piling on more and more agile bs” but IMO that was just bad faith so I ignored it.

You need both - processes that are lightweight but solid where it matters - operators who give a shit

Perhaps some processes should be put into place to make exposing the entire company into a multi-step failure?

Perhaps some already exist.

But if they have five security processes that each has a 99% chance of catching a bug, that's still a 1-in-10,000 chance that something will slip through. And I'd wager that a16z has more than 10,000 "components" that goes through those processes.

I've considered tracing outgoing responses from nginx/traefik/whatever to watch for known API keys. The difficulty would be identifying the keys amongst the noise.

Ever had a bug in code you wrote?

Not of this kind

That you’re aware of.

I come from security background and have been following best practices since 1997 so I’m pretty sure I have not made a blunder of this sort

> how do you actually make this mistake while having the skills to build a web app of this complexity level?

By not building this yourself and instead outsourcing the work to India, to people that work for 4.00$/h

And I'm not blaming the person that has to work for this little cash for delivering shoddy work like this.

my guess is internal tool that wasn't expected to be exposed publicly.

additionally, i didn't realize there are tools to automatically discover unreferenced subdomains like this. i would have just assumed security by obscurity

Presumably it's from certificate transparency logs. That's one reason I do not use TLS for my personal hosting.

Let's Encrypt allows issuing wildcards which is what quite a number of folks use for self-hosted services

If one person learns this lesson it's good. If it's on the public Internet, best to expect it will be found. Stick it behind an auth wall of some sort.

I've put internal sites behind AWS ALB's plugged into an OIDC provider[1] (Google), which works well.

1: https://docs.aws.amazon.com/elasticloadbalancing/latest/appl...

Maybe they should have installed CrowdStrike

Cant get hacked if youre bluescreened.

> a16z did not give me any bug bounty on this because of the fact i publicly reached out instead of trying to reach out privately.

I just don't understand this petty attitude. This almost guarantees next time somebody that finds vulnerability with a16z or any of its companies to seek black market rewards that will do far more damage.

This is just like when KakaoTalk refused to payout bug bounty because you had to be a Korean citizen which ended up causing more vulnerabilities to be discovered in the wild.

Companies and billionaires reading this, please don't be petty like Andreesen. Guy went from a leader to a borderline security fraud artist. You don't want to be earning more ire from the public in the current political climate. It's dangerous.

Why does this read like a 9 year old TikToker wrote it? This reads like some little script kiddie who runs fuzzing tools (and can't make any of their own) ranting online unprofessionally.

I agree that the bounty outcome is unfair.

Hopefully Martin Casado or one of the other awesome open source folks from a16z will take a look at this and make the person whole!

The person is whole as nothing was taken from them. If you choose to do free work you are not owed anything.

From the techcrunch article:

> “On June 30th, a16z addressed a misconfiguration in a web app that is used for the specific use case of updating publicly available information on our website such as company logos and social media profiles. The issue was resolved quickly and no sensitive data was compromised,”

What the fuck is this? They are blatantly lying here. There was a lot of sensitive data compromised. Anyone who inspected the site could have had access to everyones emails.

How do you know that? Both quotes seem to explain why what you're saying isn't true.

If anyone could view any of those secrets and access emails, then sensitive data was exposed. They can't just decide it wasn't exposed because no one else told them about this.

Couldn't it be the case that the secrets were not useful for accessing sensitive emails? Their response made it sound like the secrets were limited to a specific, limit-used app.

I'm just going off what the hacker said.

> the compromised list of services:

> their database (containing PII)

> their AWS

> their salesforce (never checked, account may be limited)

> mailgun (arbitrary emails from a16z domains, and also could read older emails)

> ... and probably more

Neko!

https://en.wikipedia.org/wiki/Neko_%28software%29

The Wikipedia article is missing the implementation in the article. Too bad they don't pay bounties.

It's really hard to generate "all due respect" for a16z.

Question to the community. I managed to expose all customer data of a well-funded D2C brand and when I reached out to them I did not ask for bounty before I shared the fix/the security hole. I only got a 200 USD gift card for their shop :D

What is best practice here? Do you first tell the company that they have a security issue, ask for bounty and then help? Is that unethical? Blackmail?

> i publicly reached out

Means what exactly? What information did your public reach-out include?

EDIT:

Ah, I think it's a tweet that said:

> someone from @a16z get in touch, now. its bad. security related.

Lol, ok. I guess they don't want anyone to know they had a security vuln. I wonder if they make you sign an NDA too when you get the bounty.

Stuff like this is what gives the entire security and white hat community a bad name.

1. "Surprise pentests" are illegal in the US and pretty much every jurisdiction in the world. If you are actively breaking into websites without a prior agreement, you are not doing anyone a favor. Save your efforts for companies that actually want you.

2. If the company doesn't have a published bug bounty program, they don't owe you anything. Yes they can still be nice and pay you, but they definitely won't if you disclose the vulnerability to the rest of the world without giving them a heads up and enough time to fix it.

3. "Oh I couldn't find an email address" is the worst excuse in the world. I found one after exactly 5 seconds of Googling (at the bottom of https://a16z.com/connect). And even otherwise there's Twitter, Instagram, LinkedIn and a hundred other ways to reach someone at the company if you really want to.

This is classic case of clout chasing over responsible disclosure.

> If you are actively breaking into websites

They viewed the source code. Despite what the governor of Missouri[1] thinks, that's not hacking.

[1]: https://www.theverge.com/2021/12/31/22861188/missouri-govern...

> They viewed the source code.

No.

"i like to do this thing where i search twitter, looking for companies, and then try giving them a quick pentest"

"the compromised list of services: their database (containing PII), their AWS, their salesforce (never checked, account may be limited), mailgun (arbitrary emails from a16z domains, and also could read older emails) ... and probably more"

By their own admission, this is a "pentest", and they were able to access a16z's "database" and ascertain that it contains PII. Amongst other services used by a16z.

I'm not the one to judge whether they crossed any legal (or moral) lines though.

That's what security.txt is there for. They don't even have a robots.txt file.

Too much javascript for everything (front & back) seems easy but for new developers it kind of blurs the lines between what should be on the server vs the client.

https://archive.is/Wk6OO

even web3 could protect a16z ugh, thats very bad

Crypto bullshit - a16z pipeline is a great reflection of a16z as a firm.

    >a16z did not give me any bug bounty on this because of the fact i publicly reached out instead of trying to reach out privately. the only reason i did it this way was because:
    >    there was no available contact on their main site
    >    the email i could find engineering@a16z.com bounced my emails

The age-old practice of screwing over security researchers over any possible technicality is still alive and well. Brings tears to my eyes.

Just a heads up, another comment was posted here that shows right on their website's contact page a list of e-mails for contacting them.

It only gets worse when the company that published their environment variables sues the security researchers for finding it. It happens.

Any legal basis to challenge this practice ? If a company claims that they pay bug bounties but use flimsy reasons like this to chicken out of seemingly genuine cases like these

I'm guessing no, and even if their was they could make the litigation costs very high.

The sad thing here is what has to happen is the data needs sold off to blackhats to the point that entire countries get pissed and start putting near draconian level regulations and fines against companies like this to get them to stop this insecure bullshit.

It's pretty shocking how many commenters are blaming the individual for not "trying harder" to find contact information. It's pretty clear a16z didn't want to pay anything or appreciate the disclosure at all.

Finding random email addresses and sending them a notice would have gone no where other than spam folders. I get dozens of "disclosures" every week from mostly script kiddies that think my DKIM setting is somehow going to be the end of my business. My brain automatically ignores emails like it.

I’m surprised there is almost no discussion about the severity of reputational damage caused by an extremely amateur bug not expected of a prominent VC firm

Yes... In my mind, there are three kinds of security bugs.

1. Caused by pure ignorance and completely avoidable (this bug).

2. Caused by subtle configurations, workflows, programming (mostly avoidable, secret scanning, security linters, code reviews, general intelligence, etc). This is where 99% of security bugs are.

3. Caused by a malicious actor aligning planets with a single intent to maximize their cause. You'll never stop these people (three letter agencies, state actors).

edit:

A must watch talk https://vimeo.com/95066828

Probably because a16z reputation has already been quite tarnished in recent years. This is par for the course. People will still take their massive bags of money and name brand boost but "these are smart, technical, 'making the world a better place' visionaries" as opposed to wealth chasing bankers, has already run the gamut.

See crypto, Clubhouse, "it's time to build [not in my Atherton neighborhood]", e/acc Nick Land manifesto, Trump '24 support, etc.

I like lower case tweets and texts but lower case in articles like this is just ridiculous (and trying too hard to be cool)

how do I disable the cat following my cursor animation on your website? how insanely distracting

uBlock Origin -> Dashboard -> My Filters -> add the line:

    ||www.kibty.town/files/js/oneko.js^$important

Wait, do hackers feel entitled to money for finding security holes, even if there was never any signal of such reward?

Ha my actual question was downvoted. I guess people are as entitled as they say.

Actually, I think entitlement is the wrong word. Maybe more like "window washing panhandler who's upset because you don't give them money for their service"

Isn't it fairly easy to get an address like marca's? I'm sure anyone who is responsible for the place would make the connection to IT security.

I'm surprised he didn't try harder to contact someone in the company privately.

Surely any contact would have sufficed to at least try to get an introduction to their security team?

If you browse their website there are loads of email addresses for various offices and divisions.

Reply on news.ycombinator.com