Hacker News Reader

Comment by _slih | Hacker News Reader

There's a compliance angle to this that nobody's talking about. Regulatory frameworks like SOC 2 and HIPAA require audit trails and evidence retention. A lot of that evidence lives at URLs. When a vendor's security documentation, a published incident response, or a compliance attestation disappears from the web and can't be archived, you've got a gap in your audit trail that no auditor is going to be happy about.

I've seen companies fail compliance reviews because a third-party vendor's published security policy that they referenced in their own controls no longer exists at the URL they cited. The web being unarchivable isn't just a cultural loss. It's becoming a real operational problem for anyone who has to prove to an auditor that something was true at a specific point in time.

View on HN

I hate to say this, but this account seems like it’s run by an AI tool of some kind (maybe OpenClaw)? Every comment has the same repeatable pattern, relatively recent account history, most comments are hard or soft sell ads for https://www.awsight.com/. Kind of ironic given what’s being commented on here.

I hope I’m wrong, but my bot paranoia is at all time highs and I see these patterns all throughout HN these days.

Agreed. "isn't just... It's becoming" feels to me very LLM-y to me.

Now the top comment on the GP comment is from a green account, and suspiciously the most upvoted. Also directly in-line with the AWS-related tool promotion… https://news.ycombinator.com/item?id=47018665

@dang do you have any thoughts about how you’re performing AI moderation on HN? I’m very worried about the platform being flooded with these Submarine comments (as PG might call them).

I agree with you that it's a bot.

They're getting very clever and tricky though; a lot of them have the owners watching and step in to pretend that they're not bots and will respond to you. They did this last week and tricked dang.

Where did they trick me? (You'll have to trust that I'm not a bot stepping in to ask that, of course.)

They're probably referring to this exchange where the bot owner claimed to use AI for their comments because they're not a native English speaker, which let them get off with a warning, after which they continued operating the account as before: https://news.ycombinator.com/item?id=46886719

They then made a top-level submission revealing the "experiment": https://news.ycombinator.com/item?id=46901199

These days every green username is a chatbot.

I've sent them an email and temporarily banned the account. We've been doing that lately in cases that are hard to classify and where the user might be a legit user who is misunderstanding the rules.

I guess we'll probably have to add this explicitly to the guidelines; le sigh.

Thank you for keeping this place sane.

This is new to me, so I did a quick search for a few examples of such documents.

The very first result was a 404

https://aws.amazon.com/compliance/reports/

The jokes write themselves.

But how is this related to the internet being archivable? This sort of proves the point that URLs were always a terrible idea to reference in your compliance docs, the answer was always to get the actual docs.

IME compliance tools will take a doc and or a link. What's acceptable is up to the auditor. IMO both a link and doc are best.

Links alone can be tempting as you've to reference the same docs or policies over and over for various controls.

Wayback machine URLs are much more likely to be stable.

Even if the content is taken down, changed or moved, a copy is likely to still be available in the Wayback Machine.

I would never rely on this vs just downloading the SOC2 reports, which almost always aren't public anyways and need to be requested explicitly. I suspect that that compliance page would have just linked to a bunch of PDF downloads or possibly even a "request a zip file from us after you sign an NDA" anyways.

I just want to clarify how extremely standard and often required it is to download and store your SOC2s and other such documents when going through compliance. You almost never can actually just link to a public pentest report or SOC2 etc, you almost always need to go through an NDA. It's just not really meaningful to say "but the web archive is reliable" when it's virtually never an actual option to begin with.

> Regulatory frameworks like SOC 2 and HIPAA require audit trails and evidence retention

Sidebar:

Having been part of multiple SOC audits at large financial firms, I can say that nothing brings adults closer to physical altercations in a corporate setting than trying to define which jobs are "critical".

- The job that calculates the profit and loss for the firm, definitely critical

- The job that cleans up the logs for the job above, is that critical?

- The job that monitors the cleaning up of the logs, is that critical too?

These are simple examples but it gets complex very quickly and engineering, compliance and legal don't always agree.

depends, if you don’t clean up the logs and monitor that cleanup will it eventually hit the p&l? eg if you fail compliance audits and lose customers over it? then yes. it still eventually comes back to the p&l.

And in the big scheme of things, none of those things are even important, your family, your health and your happiness are :-)

Thats when you reach out to your insurer and ask them their requirements as per the policy and/or if there are any contractual obligations associated with the requirements which might touch indemnity/SLAs. If it does, then it is critical, if not, then its the classic conversation of cost vs risk mitigate/tolerance.

At some point Insurance is going to require companies to obtain paper copies of any documentation/policies, precisely to avoid this kind of situation. It may take a while to get there though. It'll probably take a couple of big insurance losses before that happens.

Insurance is already moving that direction for cyber policies. Some underwriters now require screenshots or PDF exports of third-party vendor security attestations as part of the application process, not just URLs. The carriers learned the hard way that 'we linked to their SOC 2 landing page' doesn't hold up when that page disappears after an acquisition or rebrand.

> when that page disappears after an acquisition or rebrand.

Sadly, it does not even have to be an acquisition or rebrand. For most companies, a simple "website redo", even if the brand remains unchanged, will change up all the URL's such that any prior recorded ones return "not found". Granted, if the identical attestation is simply at a new url, someone could potentially find that new url and update the "policy" -- but that's also an extra effort that the insurance company can avoid by requiring screen shots or PDF exports.

It sounds like you work at Microsoft, they do that ALL the time.

Good lord no, I would never work for that massively evil corporation.

I do, however, work for one that is deathly allergic to HTTP redirects and that changes the user visible URL's each time they change/move/update servers (or for practically any other change). So there's a constant churn of "project X is deploying to Y on date Z, the new URL will be Q" announcements -- and meanwhile, you find that any deep links to URL Q[t-1] also got changed up when date Z arrives and the URL becomes Q[t]. And then in a few months, the same game, only with URL Q[t+1].

Digital copies will also work I don’t understand why they just don’t save both the URL and the content at the URL when last checked.

What if the TOS expressly prohibits archiving it, and it's also copyrighted?

Then said writers of TOS should be dragged in front of a judge to be berated, then tarred and feathered, and ran out of the courtroom on a rail.

Having your cake and eating it too should never be valid law.

Maybe we should start with those who made such copyright claims a possibility in the first place

They're long, long dead.

There are still people who help extending it

If copyright can be used to prevent the archiving of ToS documents, a copyright duration of 3 years would be sufficient. Not all objections to copyright boil down to "the Mickey Mouse Protection Act should never have passed!".

I don’t think contracts and agreements that both parties can’t keep copies of are valid in any US jurisdiction.

I think maybe because the contents of the URL archived locally aren't legally certifiable as genuine - the URL is the canonical source.

That's actually a potentially good business idea - a legally certifiable archiving software that captures the content at a URL and signs it digitally at the moment of capture. Such a service may become a business requirement as Internet archivability continues to decline.

Is it digitally certifiable if it's not accessible by everyone?

That is: if it's not accessible by a human who was blocked?

Or if it potentially gives different (but still positive) results to different parties?

Apparently perma.cc is officially used by some courts in the US. I did use it in addition to the wayback machine when I collected paper trail for a minor retail dispute, but I did not have to use it.

I don't know how exactly it achieves being "legally certifiable", at least to the point that courts are trusting it. Signing and timestamping with independent transparency logs would be reasonable.

https://perma.cc/sign-up/courts

This is an interesting service, but at $10 for 10 links per month, or $100 for 500 links per month, it might be a tad bit too expensive for individuals.

The first thing you do when you're getting this information is get PDFs from these vendors like their SOC2 attestation etc. You wouldn't just screenshot the page, that would be nuts.

Any vendor who you work with should make it trivial to access these docs, even little baby startups usually make it quite accessible - although often under NDA or contract, but once that's over with you just download a zip and everything is there.

> You wouldn't just screenshot the page, that would be nuts.

That's what I thought the first time I was involved in a SOC2 audit. But a lot of the "evidence" I sent was just screenshots. Granted, the stuff I did wasn't legal documents, it was things like the output of commands, pages from cloud consoles, etc.

To be clear, lots of evidence will be screenshots. I sent screenshots to auditors constantly. For example, "I ran this splunk search, here's a screenshot". No biggie.

What I would not do is take a screenshot of a vendor website and say "look, they have a SOC2". At every company, even tiny little startup land, vendors go through a vendor assessment that involves collecting the documents from them. Most vendors don't even publicly share docs like that on a site so there'd be nothing to screenshot / link to.

More likely, there will be trustee services taking care of document preservation, themselves insured in case of data loss.

Isn't the Internet Archive such a trustee service?

Or are you thinking of companies like Iron Mountain that provide such a service for paper? But even within corporations, not everything goes to a service like Iron Mountain, only paper that is legally required to be preserved.

A society that doesn't preserve its history is a society that loses its culture over time.

The context was regulatory requirements for companies. I mean that as a business you pay someone to take care of your legal document preservation duties, and in case data gets lost, they will be liable for the financial damage this incurs to you. Outsourcing of risk against money.

Whether or not the Internet Archive counts as a legally acceptable trustee service is being litigated in the court systems [1]. The link is a bit dated so unsure what the current situation is. There's also this discussion [2].

[1] https://www.mololamken.com/assets/htmldocuments/NLJ_5th%20Ci...

[2] https://www.nortonrosefulbright.com/en-au/knowledge/publicat...

Also, getting insurance to pay for cybercrimes is hard and sometimes doesn't justify their costs.

We already require all relevant and referenced documents to be uploaded in a contract lifecycle management system.

Yes we have hundreds of identical Microsoft and Aws policies, but it's the only way. Checksum the full zip and sign it as part of the contract, that's literally how we do it

https://www.page-vault.com/ These guys exist to solve that problem.

> I've seen companies fail compliance reviews because a third-party vendor's published security policy that they referenced in their own controls no longer exists at the URL they cited.

Seriously? What kind of auditor would "fail" you over this? That doesn't sound right. That would typically be a finding and you would scramble to go appease your auditor through one process or another, or reach out to the vendor, etc, but "fail"? Definitely doesn't sound like a SOC2 audit, at least.

Also, this has never particularly hard to solve for me (obviously biased experience, so I wonder if this is just a bubble thing). Just ask companies for actual docs, don't reference urls. That's what I've typically seen, you get a copy of their SOC2, pentest report, and controls, and you archive them yourself. Why would you point at a URL? I've actually never seen that tbh and if a company does that it's not surprising that they're "failing" their compliance reviews. I mean, even if the web were more archivable, how would reliance on a URL be valid? You'd obviously still need to archive that content anyway?

Maybe if you use a tool that you don't have a contract with or something? I feel like I'm missing something, or this is something that happens in fields like medical that I have no insight into.

This doesn't seem like it would impact compliance at all tbh. Or if it does, it's impacting people who could have easily been impacted by a million other issues.

Your comment matches my experience closer than the OP.

A link disappearing isn’t a major issue. Not something I’d worry about (but yea might show up as a finding on the SOC 2 report, although I wouldn’t be surprised if many auditors wouldn’t notice - it’s not like they’re checking every link)

I’m also confused why the OP is saying they’re linking to public documents on the public internet. Across the board, security orgs don’t like to randomly publish their internal docs publicly. Those typically stay in your intranet (or Google Drive, etc).

> although I wouldn’t be surprised if many auditors wouldn’t notice

lol seriously, this is like... at least 50% of the time how it would play out, and I think the other 49% it would be "ah sorry, I'll grab that and email it over" and maybe 1% of the time it's a finding.

It just doesn't match anything. And if it were FEDRAMP, well holy shit, a URL was never acceptable anyways.

> I feel like I'm missing something

You're missing the existence of technology that allows anyone to create superficially plausible but ultimately made-up anecdotes for posting to public forums, all just to create cover for a few posts here and there mixing in advertising for a vaguely-related product or service. (Or even just to build karma for a voting ring.)

Currently, you can still sometimes sniff out such content based on the writing style, but in the future you'd have to be an expert on the exact thing they claim expertise in, and even then you could be left wondering whether they're just an expert in a slightly different area instead of making it all up.

EDIT: Also on the front page currently: "You can't trust the internet anymore" https://news.ycombinator.com/item?id=47017727

I don't really see what you're getting at, it seems unrelated to the issue of referencing URLs in compliance documentation.

They're suggesting that the original comment is LLM generated, and after looking at the account's comment history I strongly suspect they're correct

Oh, I sort of wondered if that was the case but I was really unsure based on the wording. Yeah, I have no idea.

I think they meant that, now that LLMs are invented, people have suddenly started to lie on the Internet.

Every comment section here can be summed up as "LLM bad" these days.

No, now that LLMs are invented, a lot more people lying on the Internet have started to do so convincingly, so they also do it more often. Previously, when somebody was using all the right lingo to signal expert status, they might've been a lying expert or an honest expert, but they probably weren't some lying rando, because then they wouldn't even have thought of using those words in that context. But now LLMs can paper over that deficit, so all the lying randos who previously couldn't pretend to be an expert are now doing so somewhat successfully, and there are a lot of lying randos.

It's not "LLM bad" — it's "LLM good, some people bad, bad people use LLM to get better at bad things."

Perhaps those companies should have performed verified backups of third-party vendor's published security policies into a secure enclave with paired keys with the auditor, to keep a trail of custody.

Your experience isn't normal and I seriously question it unless there was some sort of criminal activity being investigated or there was known negligence. I worked for a decent sized MSP and have been through crytptolock scenarios.

Insurance pays as long as you aren't knowingly grossly negligent. You can even say "yes, these systems don't meet x standard and we are working on it" and be ok because you acknowledged that you were working on it.

Your boss and your bosses boss tell you "we have to do this so we don't get fucked by insurance if so and so happens" but they are either ignorant, lying, or just using that to get you to do something.

I've seen wildly out of date and unpatched systems get paid out because it was a "necessary tradeoff" between security and a hardship to the business to secure it.

I've actually never seen a claim denied and I've seen some pretty fuckin messy, outdated, unpatched legacy shit.

Bringing a system to compliance can reasonably take years. Insurance would be worthless without the "best effort" clause.

And for this we need cheapo and fast WORM, 100 TB/whatever archiving solutions.

It's interesting to think about this in terms of something like Ars Technica's recent publishing of an article with fake (presumably LLM slop) quotes that they then took down. The big news sites are increasingly so opaque, how would you even know if they were rewriting or taking articles down after the fact?

This is typically solved by publishing reactions/corrections or in the case of news programs starting the next one with a retraction/correction. This happens in some academic journals and some news outlets. I've seen the PBS Newshour and the New York Times do this. I've also seen Ars Technica do this with some science articles (Not sure what the difference in this case is or if it will take some more time)

On their forum, an Ars Technica staff member said[1] that they took the article down until they could investigate what happened, which probably wouldn't be until after the weekend.

[1]: https://arstechnica.com/civis/threads/journalistic-standards...

I'm not asking how you solve the problem of publications making mistakes, I'm asking how you know they're rewriting articles if there are no third-party records of article contents. You're talking about publications acting in good faith. I'm talking about publications using paywalls to make it easier to lie.

If your soc2 or hipaa references the internet archive, you probably deserve to fail.

Reply on news.ycombinator.com