jjjutla

Been looking at the agent skills security space lately. All the research so far focuses on what the agent does with SKILL.md at runtime, prompt injection, or malicious commands. But the installer copies the entire skill directory into your repo. That means a bundled *.test.ts executes on npm test with no agent involvement and none of the current scanners flag it. Wrote it up here, curious if anyone has seen this angle covered before.

AI coding tools figured out that AST-level understanding isn't enough. Copilot, Cursor, and others use semantic indexing through IDE integrations or GitHub's stack graphs because they precise accurate code navigation across files.

Most AI security tools haven't made the same shift. They feed LLMs ASTs or taint traces and expect them to find broken access control. But a missing authorization check doesn't show up in a taint trace because there's nothing to trace.

No we didn't build one, we use the main foundation models. We have evals for each part of the workflow and different models perform better on different tasks, overall the majority of it uses Sonnet 4.

Yes, that's exactly what we do. Some examples: https://github.com/eosphoros-ai/DB-GPT/pull/2650, https://github.com/dagster-io/dagster/pull/30002

We just need to follow responsible disclosure first by notifying the maintainers, working with them on a fix, and making it public once it is resolved.

Thanks, we use a similar approach to GitHub's stack graphs (https://github.blog/open-source/introducing-stack-graphs/) to build a graph structure with definition/reference nodes. For dynamic typing in protobuf, we use the language compiler as an intermediary to resolve dynamic types into static relationships, then encode the relationships into protobuf.

Yes, we don't feed entire codebases to the LLM. The LLM queries our indexer for symbols names and code sections (exposed functions, data flow boundaries, sanitization functions) to build up the call chain and reason about the vulnerability.

We’ve limited the free tier to one scan per user, so deleting a scan and starting a new one won’t work because of that restriction.

And yes, we don’t support C or C++ yet. Our focus is on detecting business logic vulnerabilities (auth bypasses, privilege escalations, IDORs) that traditional SAST tools often miss. The types of exploitable security issues typically found in C/C++ (mainly memory corruption type issues) are better found through fuzzing and dynamic testing rather than static analysis.

Thank you. SAST tools built on AST or call graph parsing will struggle to detect code logic vulnerabilities because their models are too simplistic. They lose the language-specific semantics in dynamically typed languages where objects change at runtime, or in microservices where calls span multiple services. So they are limited to simple pattern-based detections and miss vulnerabilities that depend on long cross-file call chains and reflected function calls. These are the types of paths that auth bypasses and privilege escalations occur in.

AI code review tools aren’t designed for security analysis at all. They work using vector search or RAG to find relevant files, which is imprecise for retrieving these code paths in high token density projects. So any reasoning the LLM does is built on incomplete or incorrect context.

Our indexer uses LSIF for compiler-accurate symbol resolution so we can reconstruct full call chains, spanning files, modules, and services, with the same accuracy as an IDE. This code reasoning, tied with the LLM's threat modelling and analysis, allows for higher fidelity outputs.

For all the vulns Gecko found they were manually validated by humans and have a CVE assigned by a CNA. The issue that curl had was because it was a paid bug bounty program they had an influx of AI slop reports that looked like real issues but weren't exploitable.

The confidence score is calculated by two factors: whether the function call chain represents a valid code path (programmatic correctness) and how well it aligns with the defined threat model for what it thinks is a security vulnerability. False positives usually occur from incorrect assumptions about context, for example, flagging endpoints as missing authentication when such behaviour is actually intended.

Was this an incorrect code path or an incorrect understanding of a security issue?

This is why we focus heavily on threat modelling and defining the security and business invariants that must hold. From a code level, the only context we can infer is through developer intent and data flow analysis.

Something we are working on is custom rules and allowing a user to add context when starting a scan to improve alignment and reduces false positives.

We've had a few request for Elixir and it's definitely something we will work on.

This is a bug, the email-address permissions have been descoped to read-only. Profile settings are either read/write or none, hence the former. If you're concerned about privacy, sign up using email/password.