Hacker News Reader

Comment by pan69 | Hacker News Reader

> all-powerful “tenant admin” accounts that were to be exempted from network logging activity

Is this normal to build this sort of functionality into a software system? Especially software systems that heavily rely on auditability?

View on HN

Sometimes, depending on the situation.

My company retains all e-mails for at least 5 years, for audit purposes. But if some troublemaker were to e-mail child porn to an employee, we'd need to remove that from the audit records, because the laws against possessing child porn don't have an exception for corporate audit records.

So there's essentially always some account with the power to erase things from the audit records.

Very true - this comes up constantly in blockchain questions - but in that case there’d at least be an audit log showing who deleted which records.

It sounds like you haven't actually had to face that situation, because it is more complicated than just having to delete an offending attachment. You would still have an audit log of the deletion of that email record by the superuser, even if the content is deleted. And there would be other records generated to document the deletion, like I'm sure a long email or slack thread from this getting discovered and sent up the chain, over to legal, then to the FBI, then back to coordinating the logistics of manually deleting something from the audit logs. So if for a completely unrelated case, a third party auditor stumbles upon that mess, they will be able to reconstruct why a single attachment cannot be found in the audit logs.

"No" is the answer to GP: there is no legitimate reason for a fully unlogged superuser account.

Yeah, superuser accounts? Of course you need them to exist. Superuser accounts that produce no logs? There is never a reason for that. Anyone who claims they should have a superuser with no logging is up to no good.

> You would still have an audit log of the deletion of that email record by the superuser, even if the content is deleted.

If needing things wiped from the audit logs happens often, you might indeed have an audited interface for wiping things from the audit logs.

But if it's very rare? Maybe I just request the production database password for "Incident #12345" and run some careful SQL.

> And there would be other records generated to document the deletion, like I'm sure a long email or slack thread

For sure - but the account capable of deleting entries from the audit logs exists

And if I am ordered to hand it over to someone who doesn't care to explain their actions on slack? Then there won't be any explanations in slack.

Ah man... back in the day I worked for a company that built out records management software. One of the big things on the side of the cereal box was that not even an admin could delete something flagged as a record within its retention plan. Fast forward to a company doing that for emails, messing up spam filters, and getting a blast of 'normal' porn that was all flagged as records. I believe they ended up creating security groups for those files that help keep those who were using it .. safe for work.

I don't follow this example. You could still have an account delete the email while generating a record that an email was deleted. Why would you need an account that doesn't generate deletion records?

No. Never. While it’s expected to have a “root” account exempting from logging serves no honest purpose.

Of course not. It's the exact opposite and every single person here knows this.

From a an old hackers perspective disabling shell history can have positive security implications. But in today's 'cattle not pets' systems mentality I'd expect all actions to have a log and not having that seems fishy to me. Keeping logging infra secure has a dubious, the log4j fiasco comes to mind. I'm not a fan of regulation for most things, but I think we need a higher cost for data leaking since security is an afterthought for many orgs. My personal leaning is to be very choosy about who I'll do business/share data with.

> “We have built in roles that auditors can use and have used extensively in the past but would not give the ability to make changes or access subsystems without approval,” he continued. “The suggestion that they use these accounts was not open to discussion.”

From the previous post, they had auditor roles built in that they purposely chose to go around

It's the same as domain admin in active directory.

You always need it to setup the system initially.

It's like root on Linux: it's an implementation detail that it must be possible.

typically the admin account can createthings like super users, and super users can do anything with the data, but not sure there's a use case where a single account can do both, and why can any of them avoid logging?

Root on Linux isn’t exempt from logging. I also don’t know any enterprise that allows admin accounts to bypass logging.

There is no legitimate justification for this request.

root on Linux can just kill the log forwarder and erase the relevant logs, or refill them with junk.

Yes. A more competent hack would have been to use their superuser permissions to do that kind of thing.

But instead they requested that logging be disabled, thus outing themselves as acting in bad faith.

That is a very serious design flaw, but I also believe it is a flaw that is addressed by SELinux. (Perhaps someone with a knowledge of SELinux can offer some input here.) That said, I'm not sure how widespread the use of SELinux is and doubt that it would help in this case since the people in question have or can gain physical access.

If your root, you can just turn off selinux

Not without a reboot though, and while I haven’t done that, it should be possible to protect selinux ‘s config itself with a policy, requiring boot loader access to bypass, at which point you’re dealing with a different risk level.

I’ll agree that Linux security is quite limited and primitive if compared with, say, a mainframe, but it can be made less bad with a reasonable amount of effort.

What would the mainframe be running that avoids this problem?

That’s a big rabbit hole, reading about RACF is a good place to start.

The short answer would be that mainframes come with RBAC from design, unlike Unix, which has a different security model from conception and then had rbac added on top of it in some cases (such as selinux).

At least at places I've worked, terminating the logger would cause a security incident, and the central logging service have some general heuristics that should trigger a review if a log is filled with junk. Of course with enough time and root, there's ways to avoid that. But that's also usually why those with root are limited to a small subset of users, and assuming root usually requires a reason and is time gated.

> But that's also usually why those with root are limited to a small subset of users, and assuming root usually requires a reason and is time gated.

I mean, if we were to apply the equivalent from the article, then no they would not have had a reason nor been time gated.

That still leaves highly visible log traces if you’re following most security standards (required in .gov) since you’d have the logs showing them disabling the forwarder. The difference here is that this was like an attacker but had backing from senior management to violate all of those rules which would normally get someone fired, if not criminally charged.

Assuming the Whistleblower is telling the truth, why would they make the request if they could cover their tracks themselves

There’s no possible need for an admin-level user that bypasses logging. If anything these users should have additional logging to external systems to make it harder to hide their use.

The question is whether it needs to be possible to turn off the audit logs for that role. And of course: No.

Reply on news.ycombinator.com