Hacker News Reader

Comment by nl | Hacker News Reader

They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE)

The key question here is how they avoid the outside computer being able to view the memory of the internal process:

> An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communication channels that could be observed, with optional hypervisor mem- ory isolation that extends protection from software- enforced to hardware-enforced via ARM Stage 2 page tables at zero performance cost.[1]

I was under the impression this wasn't possible if you are using the GPU. I could be misled on this though.

[1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...

View on HN

This entire paper smells of LLM, I'm sure even the most distinguished academic would refrain from using notation to prove that the SIP status cannot change during operation.

While they do make this argument, realistically anyone sending their prompt/data to an external server should assume there will be some level of retention.

And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.

There's a difference between trusting Anthropic and trusting random mac owners.

I know where my answer lies in that; but i don't claim to be an objective truth.

For example OpenAI has been caught sharing data with the gov. agencies.

Apple Silicon systems have unified memory between CPU and GPU. The hypervisor page table trick is thus claimed to protect GPU memory from RDMA.

Macs do not have an accessible hardware TEE.

Macs have secure enclaves.

Good point!

But they argue that:

> PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments.

> Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-allow entitlement. The kernel denies task_for_pid() and mach_vm_read()from any external process.

> System Integrity Protection (SIP): Enforces both of the above at the kernel level. With SIP enabled, root cannot circumvent Hardened Runtime protections, load unsigned kernel extensions, or modify protected sys- tem binaries. Section 5.1 proves that SIP, once verified, is immutable for the process lifetime.

gives them memory protection.

To me that is surprising.

Looking at their paper at [1], there's a gaping hole: there's no actual way to verify the contents of the running binaries. The binary hash they include in their signatures is self-reported, and can be modified. That's simply game over.

[1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...

A note, as others have posted on this thread: I mention this as a concrete and trivial flaw in their whole strategy, but the issue is fundamental: there's no hardware enclave for third-party code available to do the type of attestation that would be necessary. Any software approach they develop will ultimately fall to that hole.

Couldn't someone just uhh... patch their macOS/kernel, mock these things out, then behold, you can now access all the data?

If it's not running fully end to end in some secure enclave, then it's always just a best effort thing. Good marketing though.

Right.

Apple is perfectly capable of doing remote attestation properly. iOS has DCAppAttest which does everything needed. Unfortunately, it's never been brought to macOS, as far as I know. Maybe this MDM hack is a back door to get RA capabilities, if so it'd certainly be intriguing, but if not as far as I know there's no way to get a Mac to cough up a cryptographic assertion that it's running a genuine macOS kernel/boot firmware/disk image/kernel args, etc.

It's a pity because there's a lot of unique and interesting apps that'd become possible if Apple did this. Darkbloom is just one example of what's possible. It'd be a huge boon to decentralization efforts if Apple activated this, and all the pipework is laid already so it's really a pity they don't go the extra mile here.

> iOS has DCAppAttest which does everything needed. Unfortunately, it's never been brought to macOS, as far as I know.

Apple's docs claim it's been available on macOS since macOS 11. Am I missing something here?

https://developer.apple.com/documentation/devicecheck/dcappa...

All lies. They mean the symbols exist and can be linked against, but

https://developer.apple.com/documentation/devicecheck/dcappa...

> If you read isSupported from an app running on a Mac device, the value is false. This includes Mac Catalyst apps, and iOS or iPadOS apps running on Apple silicon.

That really sucks! TIL. So app attestation is iOS 14.0+, iPadOS 14.0+, tvOS 15.0+ and watchOS 9 only.

Yes. Running attested workloads on macOS if you are not Apple is nontrivial.

You can probably just tap the HTTP(S) connection to spy on the data coming through. I think it's a mistake to assume any kind of privacy for this service.

The biggest argument for remote attestation I can think of is to make sure nobody is returning random bullshit and cashing in prompt money on a massive scale.

> PT_DENY_ATTACH

All you have to do is attach to the process before it does that, and then prevent this call from going through.

They quite frankly have no idea what they are talking about.

I'm not arguing anything. This is how it works. There is no but.

Protection here is conditional, best-effort. There are no true guarantees, nor actual verifiability.

Reply on news.ycombinator.com