If you look at [0] (the code they run in the CVM), there are a couple of things that worry me:
- They ship logs out of the CVM and worse, they send them to third parties (DataDog). Even if we could verify every bit of code running in the TEE, it's not enough to know the code doesn't maliciously ship prompts to a third party, we also need to audit what each binary logs.
- SGLang, the core inference engine, isn't reproducibly built. We have no way to verify that the thing they call "SGLang" is what they claim it is.
Really, it's the log shipping processes that worry me the most. Ideally, NEAR would minimise how much auditing needs to be done by having the minimal open-source proxy be the only thing with network access, making it much easier to audit potential exfiltration routes.
[0]: https://github.com/nearai/cvm-compose-files/blob/main/prod/G...