Hacker News Reader

Comment by bob1029 | Hacker News Reader

If you are using LLMs to interact with sites like GitLab and GitHub, and you have the option to use a GraphQL API, you should jump on it immediately.

GraphQL is absolutely terrible for human developers to interact with, but it's like Facebook could see into the future back in 2012. I cannot imagine a more perfect API surface for agents. With the REST API on GitHub, you can consume maybe 10 issue JSON blobs before your context window is blown out. With GraphQL constraining the results you can easily read hundreds in the same token budget.

Additionally, the # of requests your agents need to make can be reduced in many cases since GraphQL can join across types whereas REST APIs cannot. You essentially get savings in two dimensions here. Quota and raw token volume per logical response.

View on HN

I work closely with the team responsible for a large, self-hosted GitHub Enterprise instance. This is good advice for clients/consumers of GH data, but it can very easily lead to a lot of strain on the server-side. It’s not obvious what fields that you request are simple reads from tables or actually end up invoking git under the hood.

You could argue the rate limit guards should better reflect that, but that’s just not the reality of the system. Likely speaks to a lot of stability issues GitHub has been facing lately.

Isn't it kind of part and parcel of any GraphQL deployment to update it efficiently?

That is true for almost any GraphQL backend not just GHE.

> It’s not obvious what fields that you request are simple reads from tables or actually end up invoking git under the hood.

I'm not familiar with graphql but what would make something "invoke git", is it a technical thing or hyperbole?

My over-simplified explanation: the graphql server will map a field on a request to a resolver. The resolver can execute whatever code it needs to return the value, up to and including calls into libgit.

So asking for the title of a PR might just be an extra column selected on a DB query. But calculating mergability status of that PR might be something else entirely.

All of the queries my agents use select fields like issue title, body, labels, createdAt, updatedAt, etc. That's about it. I would hope that stuff is cached and efficient to read. I do not think GraphQL is a good way to interact with git. Running git on the CLI is the best way to interact with git.

It's definitely one source of GitHub's issues. They never implemented limits properly and their codebase is likely decades of cruft and impossible to retrofit quickly (to say nothing of their horrific mysql clusters). If you want to scale and survive it, you have to build the thing right the first time.

You have to be so uptight about performance when you're small that people look at you funny, or your shit is going to fall right over under webscale (ai scale?) load.

Well 1. GitHub is the broken & bad one, and 2. isn't that kind of on them? If they're getting a ton of requests that read data awkwardly, the fix seems like the quintessential data engineering task.

You sound like someone who has not been informed about the problem GitHub is facing. No one else is facing anything close to it, so there is no meaning to "the bad one".

https://cursor.com/blog/git-at-any-scale

I put my ssh credential behind a tpm and now I notice that every time I switch a chat in ChatGPT it tries to see the current remote status of the branch.

That also doesn’t help with load. So besides what agents are doing themselves the AI companies also have a responsibility of being good citizens.

VS Code does this as well - or maybe the GitHub plugin does it. Either way, GitHub owns it.

> lot of strain on the server-side

That's why GitHub assigns "points" to each requests and deducts based on the data shape you request. For simple requests it's 1-to-1, but can quickly balloon

Although that may be true, the quota at least on GitHub depends on what you're quering. We worked directly with GitHub and a complex enough organisation hit with a GraphQL query can actually hit your hourly app limit before you even get a response.

As for GitLab, having hosted it for medium size organisations (~200 devs) and seeing how monorepo's work (they don't, we had GitLab's team show us that one page view made 50K db queries on our setup), please consult with your local admin team before firing GraphQL at it.

The quota consumption is based on the upfront possible number of connections given the query, not actual connections, so deeply nested queries can be very expensive if not aware and careful about it.

> With the REST API on GitHub, you can consume maybe 10 issue JSON blobs before your context window is blown out.

This is a solved problem. They just dump it into a file and `jq` or `rg` to find the stuff they need.

Agents are smarter than you think. They've been hill-climbing for generations in their RL environments.

The ones that get their context window blown out don't survive to launch

I digress, but GitHub's GraphQL API has been neglected for years. Or at the minimum, they have not maintained feature parity with their REST API, so there are a good number of cases where you simply cannot do what you need to do without using their REST API.

It depends. There are hidden limits in GitHub’s gql. Some will time out above certain quantities and it’s not documented, which probably means it’s a significant server strain to serve the successful responses. I find I have to maintain a test suite to probe those limits. All this makes REST continue to be appealing if testing the gql load for a service hints at any uncertain limits/instability.

GraphQL is not terribly bad to interact with. The schema system and field selection is really powerful that you can pick partial results, and that you can use an object oriented way to represent your domain models. It is pretty neat for cross-team collaboration.

Replacing GraphQL and REST + OpenAPI OTOH I think is much more terrible. You have two API description language (one on the URL path, maybe the Zod schema, another one on the OpenAPI schema). Things like tRPC or magic functions are just using Typescript type system and comment to replace the schema that GraphQL already has.

The only thing I would bitch about GraphQL is it is quite hard to build an ad-hoc GraphQL server from the first principle, while REST is really KISS till the end. And GraphQL typically needs a lot more attention to N+1 problem.

Now imagine an API that turns every prompt into a validated query within seconds, no prior schema knowledge required because we're building an index, so the model is not wasting tokens on that task even with multiple megabytes of SDL.

We're building such an API for some of the biggest enterprises in the world. Many of them have very large (federated) GraphQL APIs across tens and hundreds of teams. From an agent perspective it's a lot easier to consume a single unified graph where a single query can span 5 relationships vs making hundreds of N+1 rest API calls across many heterogenous APIs from different teams that all look slightly different.

mine just is the gh cli. is the advantage of graphql that they can compose a query that would take multiple cli invocations?

CLI still has the possibility of being a little more token efficient, at worst it may use graphql behind the scenes.

For our company, we advertise the graphql schema to bots and they can one-shot whatever task they're trying to do. I've found that it's so good that I cancelled building an an MCP server and any skill. Just a well documented GQL schema. It's pretty remarkable.

I think you might also have a GQL schema that's particularly well suited to what those bots need. Either that or your backend schema is relatively simple and the GQL schema covers all its possible data compositions.

> "I've found that it's so good that I cancelled building an an MCP server and any skill. Just a well documented GQL schema. It's pretty remarkable."

This gave me a chuckle, there's some subtle irony here- especially if the documentation of the GQL schema was work that needed to be done!

That's completely fair, I did spend a day or 2 having AI rewrite the field descriptions based on the code. for bots, by bots!

Can you provide an example of this schema?

https://action.parabol.co/graphql/schema.graphql

no the advantage of graphql is that the caller can limit the response to only the information that they need. in a REST API the caller has to filter out the extraneous information. using LLMs to do the filtering uses up context window

This is not fully true.

Most agents will use curl | jq to slice what they need (assuming a known API)

Yea- its just a matter of whether the work is delegated to the client or offered by the server. Making sure what is served is in a really quality schema is generally the most efficient path- ime

All practical GraphQL interfaces constrain you to precisely the query formats that are used by the front end, to avoid queries accidentally reading private data or blowing up computation time.

Having worked on the gh gql api, I wish you luck with timeouts haha

Reply on news.ycombinator.com