13 karma · joined April 12, 2024
As for intelligently - but probabilistically - determining confidentiality (if I read that correctly), that does sound pretty interesting in scenarios where metadata is just simply insufficient. Also tricky. Sounds like you thought about these problems pretty deeply.
We usually sync per user. That way we make sure that no information leaks to another interface.
We plan to implement unified ACL syncs to dedupe the data or even have 1 sync per org, but that’s mostly a cost optimization; Airweave will just scale horizontally until then.
1) the integrations are done traditionally so with REST/SQL. The MCP/REST search layer rests on the data that gets synced.
2) most providers are painless. Slack doesn’t want major exports in one go but most developers point at a single channel anyway so the rate limit errors don’t bite too much.
3) this is all orchestrated by the platform itself. Incremental syncs will receive the latest “watermark state” and sync from there. Hashes are used to compare data for persist actions (update/insert/keep)
You can compare it to how coding agents like Cursor work. This is the usual pattern you see: - The first step is reading your prompt - Then it goes through all the attached files and searches your codebase - The last step is to make code file edits.
Non-coding agents that use "regular" MCP servers completely miss the second part. It's very hard to go from natural language instruction, to a chain of API calls that actually work and don't end up in hallucination