Hacker News Reader

Hacker News

Comment by tajen | Hacker News Reader

Parent Full thread

tajen

How does Google build such forsenics? Do they have spyware monitoring all their company laptops?

View on HN

Yes! It's open source: https://github.com/google/grr

Google probably has the largest team of internal forensics software developers.

Yeah, they definitely have some really specific accusations about what went on:

>>> To gain access to Waymo’s design server, Mr. Levandowski searched for and installed specialized software onto his company-issued laptop. Once inside, he downloaded 9.7 GB of Waymo’s highly confidential files and trade secrets, including blueprints, design files and testing documentation. Then he connected an external drive to the laptop. Mr. Levandowski then wiped and reformatted the laptop in an attempt to erase forensic fingerprints.

The bit about connecting the external drive is interesting but I guess there's probably a ghost of that action somewhere on a drive (assuming you could more or less restore the drive before being wiped).

My guess is he still had a network connection up when he did the copy. These kind of things like JAMF, osquery and so on upload what you do on your computer on a fairly frequent basis. After a certain company size all of them do something like this, and most of them do not spell out the amount of spying they do on their employee's work devices and office space.

Or he didn't do a secure wipe when he reformatted the drive and google inspected the computer when he returned it.

OR google modifies their laptops and has separate chips logging this stuff, which would be fairly impressive!

Or he didn't actually do any of this alledged stuff and it's all innocent.

We will find out in the court case either way!

I would imagine there are IDS systems running, given Google's state level adversaries.

Accessing large amounts of files you haven't previously accessed and shortly thereafter attaching an external drive should trip a competent IDS.

Unless it's changed since the last time I played with it osquery doesn't upload anything on its own–it's just a local tool/agent that you can use to gather data. Carbon black is a good example of something like that.

JAMF I believe gathers application usage data but nothing as in depth as what's being discussed. It's also comically handicapped. It somehow manages to do a poor job of everything it tries to do so "the world's largest online Mac administrator community" is forum post after forum post of half understood franken-scripts. I used to think it was milquetoast but after sitting through their sales reps crapping all over open source software (despite extensive use of OSS libs in their products) and seeing it fail to do the most basic stuff out of the box my opinion is that it's over priced crap.

Or, you know, syslog.

It could well be that they have comprehensive debugging enabled for all their employees on work supplied machines. I know they beta test through employees, so that would make sense, to some degree. If the debug log is remotely synced, as an added benefit they get to review actions after the fact, which might be what happened here.

Company-issued laptop probably tracks what software is installed anyway to mitigate malware. Services are probably designed not to accept connections from untrusted devices. Hardware 2FA makes it difficult to impersonate someone without their knowledge.

I can see the temptation—I've always missed having access to IP after leaving a company.

You cant restore SSDs. I would assume Googlers are rolling SATA SSDs, if not M.2 PCIE ones.

What Google could be doing is remote logging on the laptops - logs uploaded to ze cloud every time you connect to the mothership. Plugging in USB drive leaves trace with USB ID, volume information etc. Windows also logs this and more http://www.forensicswiki.org/wiki/USB_History_Viewing

Protip: to exfiltrate data with minimal trace your best bet is taking out the drive and reading it in another computer (using write blocker for best effect), this can still be traced if someone is logging SMART written/read data (I am, but Im paranoid), not all HDD/SSD vendors provide this info. Second best is booting from USB drive so the original OS never sees the plug/unplug event in the first place, I have no idea about current state of UEFI/AMT logging going on tho.

Disclaimer: I used to do forensics.

Really interesting! It's sobering to think about the ways that even in a system not set up for logging you can trace back through these actions.

I was asked to figure out what had happened on a system where some data had changed and 2 parties were blaming each other. After about a hour digging around I managed to piece together a picture of how Person X had got up on a Monday morning, discovered (on their mobile, home wifi) that they had made a mistake on Friday, then logged in on their desktop to fix it from home (first time they logged in at home), then went to work and blamed someone else.

What was remarkable was how many different sources there were to pick up bits and pieces from. In isolation there wasn't much to go on, but once you start the connecting the parts, it's really incriminating.

This was the noteworthy part for me as well.

They had to know that he:

1. modified the software on his laptop

2. logged into an area he should not have had access to (this is probably standard)

3. attached an external drive (possible, but standard?)

4. and they got all this info after he deleted the drive, which means they either went in and found remaining data on the drive or else they captured the info in real time.

I suppose if the drive is clean now, and they know he downloaded data, they can infer that he wiped it.

I suppose that if they know he accessed it, and there was software on his computer preventing him from doing so, they can infer that he downloaded something to overcome it.

But knowing that he connected to an external drive implies active monitoring. That's the part I am most curious about.

Just to be clear "his" laptop is "the laptop issued to him by Google".

I'd think that standard antivirus software detects and alerts external drives being attached.

1) makes sense if Levandowski did it over the company internet connection. There could be a record of his unusual requests (software download, software update).

> they can infer that he wiped it

For 4), Levandowski reformatted the hard drive before returning it, so there's no inference there.

It could be a lot simpler than that.

Perhaps the size of the downloaded repository is larger than the physical size of the drive on the laptop?

Since he was working on self-driving cars at Google, he probably was authorized access to the software and hardware involved.

Maybe video footage?

This is all information a rudimentary desktop auditing tool can gather and store on a server. Most collect both hardware (which would include connected devices) and software inventory. Anyone SHOULD be auditing company PCs on a relatively regular basis. It wouldn't surprise me if Google was auditing much more frequently than the average and could catch something like this in the act.

Yup, lots of firms using HIDS that gather all sorts of system data. OSSEC is what I've seen rather frequently for this.

What? Most version control systems have authentication systems, I'm pretty sure Google has good reason to keep audit logs of employee's access to schematics.

Even if they do, it is very unlikely they were built with provable nonrepudiation requirements. From personal experience designing the security of a PKI CA that passed gov security certifications, the audit subsystem is the most challenging part to do right. Could probably consult for the defense in tearing down the evidence :)

I'll accept that it's hard, but why do you think Google didn't do it right?

It would require a prohibitive amount of engineering resources to be done right, i.e. a chain of guarantees that from creation time to the moment they are inspected it can be proven that the logs cannot be tampered with by nonauthorized users. There are other requirements e.g. separation of roles that are expected on audit subsystems. I am positive it would not pass an adversary expert analysis.

Google's threat models include nation-state adversaries: I suspect the effort that seems "prohibitive" to you was seen as necessary after the infamous smiley on that PRISM slide. Security is an existential threat: if user's don't trust Google, they will fail.

Google also has an internal PKI CA - I think they meet and exceed that security baseline for rigor.

Yes, for purposes of issuing certificates I'm sure they are OK wrt auditing (I was just establishing my "credentials" with the CA comment).

The threat models targeting anti-Google malicious actions obviously worked since they have traces of the Otto guy's activities. What I am asserting is that these forensics logs they use as evidence can be attacked in court as not being sufficiently protected from tampering by an internal Google party interested in fabricating evidence.

I would assume that the servers that host big trade secrets would log everything happening. There wouldn't be a need to log the laptop itself.

Typical garden-variety endpoint security/IT management solutions such as Crowdstrike or Casper with a kernel module are capable of all of that.

Reply on news.ycombinator.com