alexblackwell_

Feel free to try it out and let us know if you have any issues. I was personally skeptical, but it now does about 70% of my "reverse engineering" work. Sometimes needs a bit of guidance on really complex fields, but I think the tool calls we have given it allow it to be pretty smart. Just my biased $0.02 though

Web apps like this are few and far between, and piggy-backing on the session/anti-bot tokens from your browser usually works. In really complex cases we do help companies on a white glove case-by-case basis.

Interesting. We essentially do the same thing, but with MITM. We have a chrome extension internally, but have found it's a bit of a clunky interface. Might be releasing one soon. The approach with executing script in webpage is interesting. Best of luck!

You can prompt the mcp to do this. Honestly considering adding this as a skill in the agent chat. Internally we do this all of the time for our white glove integrations.

The requests still route through your servers/the data still lives with you. Kampala is a powerful tool but I don't see people replacing the actual apps with it. Most of our customers use it for automating repetitive actions in legacy dashboards.

I wouldn't consider what we do evasion really. We are using real tokens that you have received from your browser as a result of browsing the web. Any good anti-bot will have enforcement for abuses of that token.

My broader point is that these ToS clauses are often so broad and vague that they're essentially unenforceable and not meaningful in practice. For example, "Do not use bots" covers a pretty substantial amount of ground, and intention isn't exactly something you can screen for. Is an autofill chrome extension a bot? If so what separates that autofill from accessibility extensions? Is someone using Whispr flow to fill forms considered a bot? AirBNB doesn't block Google's crawler. Why? A company can enforce its TOS as it wishes. My general point is that the waters are murky, and that automation is a sort of sliding scale.

sorry a bit confused on your question here. If you're asking about JSON RPC we handle this via parsing. The AI can then handle deducing structure most of the time given enough context

gRPC obscures the keys not the values. Enums and signed ints are sort of tricky, but the latter is just a mapping problem and the former can be figured out through some logical deduction. gRPC isn't designed to obscure request content, but for over the wire efficiency.

Noticed you have two comments here. I think my response to your other comment best answers this (https://news.ycombinator.com/item?id=47798259). Definitely open to discussing this more here. Not sure if I agree on the self-compromised CA bit. MITM proxies have been used for 20+ years for debugging. In fact, I use Kampala to debug our personal APIs/web app all of the time.

The goal is not to scrape sites en-masse, but to allow people to automate their existing workflows and actions that they perform already via a browser. I understand the concerns around this being unethical, and it's something I spent a lot of time thinking about when I worked on automations previously. I've written a decent amount about how I don't think that sneaker bots or ticket bots are ethical. I don't support mass scraping websites/making the web more inaccessible for others.

I do have to push back on the ToS comments though. Automation is used daily by nearly all companies. RPA is a billion dollar industry. Browserbase raised at 300M valuation. Is using puppeteer to automate a form submission a violation of ToS? If so then why is using a screen reader not? Is it the intention? Why is hitting network requests directly different? I personally don't think that automation is unethical (as long as it is not affecting server capacity). I don't think the answer to the ethical problems in scraping is just not to automate at all. Open to disagreement here though.

Yep essentially. I would argue that we're probably closer to a MITM proxy like Proxyman than Wireshark. We don't do general packet sniffing (yet), although internally we use our own packet sniffing tools for reverse engineering on-prem installations.

Definitely get that. Being hammered by scrapers is a massive PITA (especially with latest aggressive AI crawlers). We focus primarily on allowing people to automate their existing workflows. For all hosted workflows we have rate limits to prevent mass scraping/affecting server workload in any real capacity. In fact, because we don't load js/html and hit endpoints directly I would guess that we consume less server resources in the end.

Yep we handle gRPC and websocket. gRPC is a bit sketch/hard to do because of the way the protocol is designed. FWIW not many sites implement gRPC (some google sites and spotify being the only two I can think of), and if they do they usually have decent APIs. Feel free to try and lmk if you have any issues!

Yeah agreed this messaging is a bit confusing. Our focus is on helping people build automations, not do any mass-scale scraping.

Thanks Ben! For session re-auth we attempt to agentically find the session refresh/login endpoints and make those part of the flow as an auth provider. This can be a bit sketchy though and is the main bottleneck right now. Currently working on some cool workarounds for this that allow us to piggy back on browser that should land by next week :)

Unfortunately we can’t do much around SSL pinning yet. Not sure how deep you want to go, but there are several Frida scripts that patch common pinning implementations.

I also think mitmproxy (open source) has an option to spin up a virtual Android device that can bypass pinning via AVD. I have not tested how reliable it is though.

FWIW, it could also be a cert trust issue. I would try a quick Safari search to confirm the cert is fully trusted. ChatGPT is pinned, but the gym app makes me think it might be a trust or config issue on your device.

Happy to take a look as well. Email me at alex at zatanna dot ai.

I’ve probably spent on the order of months of my life in proxyman/charles/burp/powhttp. All are great, but I’ve never been completely satisfied with the UX/features for building automations. As far as differences; we don’t modify TLS/HTTP2 connections, have a fully featured MCP (each UI action is an api action by definition), and have built more robust automation tooling in the app itself. The goal is to be an AI-native burp suite/powhttp with Proxyman-like UI.

We’ve essentially been using that “recursion” to tune our agent. Having the agent build itself is not something I would have ever thought of though. Curious if you find it genuinely creates specific enough tools for it to be worth the setup time? I have a claude skill that takes in a chat and then offers tools/fixes to system prompt. Have found that + the anthropic harness engineering blogs to be super useful in making the agent actually do the work.

It was the (generated) name of the Conductor workspace when I started the project. We were going to rename it before launch but the name stuck lol :)

Super cool. I think this is where most automation is heading . Would be curious if you could one-shot the auth flow using Kampala and completely ditch the browser. Also FWIW you can import HAR into Kampala and we have a few nice tools (like being able to a/b test payloads/replay requests) that meaningfully reduce integration time.

Zatanna is a DC comic book character. I’m not sure if either of us have even read comics, so not sure where that came from. For Kampala, when I started this I was trying Conductor for the first time. The generated workspace name was Kampala (the capital of Uganda). We even have a 3rd name. We actually incorporated as NoPoll. That one’s a bit less inspiring though lol.

We’re currently running a variety of stuff for TLS/HTTP2. If you download you can see the full trace of the connection. We dump the TLS connection byte for byte with the different structured subsections. With tls.peet.ws and bogdann finn’s tls-client (which we use parts of with some modifications) I would say that http3/tcp fingerprinting is probably the remaining issue. We currently don’t support http3 connections (they’re niche + apple system proxy doesn’t support them well), and TCP fingerprinting is a bit too low level to build out tooling in GO currently. Possibly for a later release. Curious if you’ve tried bogdann finn/the existing tooling?

Oops now realizing that pattern where we send you to bottom latest download link is definitely confusing. Fixed so that the top button sends you straight to Download now.