The new HTTP QUERY method explained
kreya.app
kreya.app
"Using GET with a Body works"
Seems like this is going everyone's head. You're not supposed to use GET with a Body, this is a hack, therefore having an explicit method makes sense.
Just because it works, doesn't mean its the right way
> Just because it works, doesn't mean its the right way
Hacky things not working is a feature, not a bug.
The use case of QUERY is because POST conveys non-safe, non-idempotent requests which can potentially modify stuff according to the REST spec. GET requests on the other hand convey retrieval of a resource. However, due to GET requests not having a body, there's a limit to the amount of data you can put in the URL and you also cannot put sensitive data in it.
Additionally, GET requests are meant to be highly cacheable by default while a lot of the QUERY type requests are usually meant more for one-shot access.
QUERY is meant to address these limitations.
I think the point made is that _all_ existing web servers have no idea what a "QUERY" is anyway, so changes need to be made anyhow.
Except it doesn't. Some API gateways outright strip request bodies from GET requests to prevent them from being forwarded.
It sounds like most people with the "just use GET" nonsense are far from having any experience in cloud computing.
So, change is required. Just change GET to allow for body and move on.
Most of the systems that are blocking GET/body could be easily tweaked to allow it. Today. As is.
QUERY will likely need firmware updates, core engine updates, etc.
Meanwhile, tweaking GET is a rule change.
"Using GET with a body isn't in the spec, WAFs and webservers that haven't been updated might reject it!"
Ok, QUERY wasn't in the spec when those were written either. What do you expect those appliances to do with a totally unknown verb?
It's a welcome addition but the new method is pure marketing. There's no reason the update couldn't have been to expand GET instead of add support for QUERY.
405 Method Not Allowed
We have existing standards for unsupported methods.
It's exactly the opposite. The HTTP spec does cover GET with bodies. However, what you fail to account is that the spec specifies they are invalid and a GET with body is meaningless, and represents a potential attack.
Actually you are quite wrong. HTTP already accommodaties nonstandard methods, so HTTP-compliant servers do support whatever string you put together as the method.
What you are failing to understand is that this proposal defines both a method and its semantics. This means the expected behavior regarding idempotency, safety, cacheabiliry. Nonstandard methods by design are interpreted as being unsafe.
What you are also failing to understand is that GET is explicitly designed to not have a request body. This means that a GET with a body is interpreted as something that violates specifications and is potentially an attack such as request smuggling. Again, some API gateways strip them as a security precaution.
> Most of the systems that are blocking GET/body could be easily tweaked to allow it. Today. As is.
Utter nonsense. You are talking about things like home routers and old phones.
> QUERY will likely need firmware updates, core engine updates, etc.
Not really, only if those devices do not comply with HTTP.
The real value proposition is the semantics of a QUERY operarion regarding safety and caching. It's not a coincidence that this proposal is backed by the likes of Cloudflare. All HTTP compliant requests are just forwarded through all internet boxes, and the likes of Cloudflare sits at the edge safely caching them.
Why not just standardize it? It seems to be a better way than adding a new method.
2) A plain separation between GET should only accept query string parameters and QUERY should only accept body parameters potentially reduces attack surface of attackers trying to mix and match the two to find potential order of precedence attacks.
But specifying this behavior would get you in the same situation as adding a new method: everything not up to date with the spec will keep behaving poorly but newer system would work.
The only benefit of adding a new method is for marketing/awareness: it may end up getting support faster than the alternative because it sounds as a sexy new thing to implement. This kind of benefit should not be overlooked, but we should also acknowledge its limits: most of enterprise stuff (WAF, frameworks, etc.) are not going to work overnight just because it's a new method instead of a spec change.
PUT is idempotent but not safe.
https://news.ycombinator.com/item?id=48568502 (4d ago, 173 comments)
https://news.ycombinator.com/item?id=29794838 (4y ago, 125 comments)
Will be fun when those see a whole new http verb, I bet that leads to at least DoS by the track record of that company.
It should adept reject what it does not expect.
Given that most websites only ever use a handful of methods (even once you account for REST APIs using PUT, PATCH and DELETE now), and that list very rarely changes, the WAF developers tend to look at this question from the opposite angle: when you know there are only half a dozen widely used methods, why would you allow anything else by default?
So is using QUERY requests for quite some time from now.
The biggest win is how intermediary boxes now have concrete guidance that a specific HTTP request is both safe, idempotent, and carries a request body. Up until now none of this existed, and at best developers could use unsafe methods to carry request bodies (see GraphQL and how it uses POST for queries)
One should adhere to Best Practices since one cannot control every device between the app and the user. Best Practice says “GET has no body. QUERY can have a body. If QUERY fails (405), use POST with the body.” And eventually, enough proxies will behave well enough that at least the HTTP bit of the app has a chance of working.
Do you test with a proxy? I know I don't, and if I would it'd be the latest version of squid.
Some of those non-conforming middle boxes could be unattended black boxes in the back of a closet somewhere on the 3rd floor. Who knows. Some customers would consider alternatives to your offering because it doesn't work, others would file a support ticket and there becomes an incentive not to use the new and shiny. The broken middle black box vendor can't be bothered and keeps selling broken boxes. Shit like this is why we can't have nice things.
All in all, I dislike the overall focus on the HTTP method when designing "RESTful" interfaces. If all we're building is, effectively, an RPC, why would the cacheability meta-information be the first thing we specify?
> I've had my fair share of situations where I wished for something like HTTP QUERY.
Using POST instead comes with no drawbacks
I'm not sure QUERY is a great solution, because in the context of a web application absolutely no one enjoys using a page that does not keep its state on refresh, so that really limits where QUERY makes sense, but if you have a case that is not driven by navigation, great.
Not so fast, champ. It matters nothing what your library let's you run away with. What matters is that every single box in the internet between you and the origin server will tolerate, and your pet library doesn't have a say in that.
> Using POST instead comes with no drawbacks
There's a hefty share of ignorance in your comment. Between POST being classified as an unsafe method and the absence of support for cashing, there are plenty of downsides of abusing POST for query requests.
As the RFC was initially proposed by someone from Cloudflare, were you aware that not even Cloudflare support caching POST requests? Their unofficial support for caching POST requests is to create a fake GET request to serve as cache key and use that to cache the response. This is the kind of hacks everyone is forced to go through instead of using something like QUERY
- request-with-a-body
- idempotent-request-with-a-body
- safe-request-with-a-body
because the additional constraints induce properties that are extremely useful to general purpose clients ("I didn't get a reply to my idempotent-request-with-a-body, can I resend it without risking loss of property?")
Would someone then come along an introduce safe-request-without-a-body method? After all, we can already meet that "need" with safe-request-with-a-body and content-length: 0.
Think rfc-5789::PATCH - mechanically, it's just another request-with-a-body, but with more tightly constrained semantics. But general purpose components can take advantage of the additional properties, and so we introduce a "niche" method with tighter constraints.
Document resource manipulation is a common case, so we probably end up with a family of specialized methods, in much the same way that we have a bunch of WebDAV methods.
Similarly there are good arguments that if you were designing this from scratch it would make sense to still separate QUERY and POST. To some extent they mean very different things: "search" versus "create"/"do". Some of that is a modern expectation from years of mapping the common "CRUD" concepts to "REST": POST ~= Create; GET ~= Read; PUT ~= Update; DELETE == Delete. But that's a lens that's still useful in designing the thing from scratch even if it wasn't necessarily in mind when HTTP was first designed (especially given the different verbs in HTTP terminology).
1. What the most popular HTTP servers actually accept
2. What the most popular HTTP clients actually send
3. What the RFC "standard" says
Personally I choose #1. I don't use the most popular HTTP clients
For example, under definition #1, I can do HTTP/1.1 pipelining with POST
The RFC "standard" often comes after the software implementation(s); most RFCs document what's already in use on the internet. As the selection of software expands, some competent software authors trying their best still often struggle to conform to RFCs. Go figure
To me, the source code of the most popular servers in use is the standard
Unless you're really going to improve things or the existing practices are really too painful, standards should follow convention. Even though GET+body is not handled the same everywhere, it's easier to make that the standard than it is to make a new syntax the standard.
Is there some research or study that you base this claim on? What is the reasoning? Can you elaborate?
Likewise creating a new syntax for something that already exists means you are just adding to the heap of stuff that needs support on mainstream servers, and as I already said it will probably create compatibility issues as the old deprecated/illegal syntax is removed. This is unnecessary friction.
And really what is the advantage of a new syntax? That needs explaining.
DELETE is intended to delete one specific object, pointed to by a unique URL, not to delete arbitrary objects matching some criteria.
Just as one example, among many, you could try WebSockets (or some other similar protocol) and then push anything over it. Your message could be plain text, JSON, binary, whatever. Web Sockets and protobufs are bidirectional (full duplex) too.
Just because it's old doesn't mean it is worse than alternatives, most likely it is quite the contrary.
HTTP accounts for roughly 70-80% of global web traffic, and if you narrow it to TCP-based application traffic, its dominance is closer to 95%. Comparing HTTP to COBOL is a bit like comparing trucks to bicycles...
yeah, I also don't get the hype, let's go back to everyone declaring their own communication protocol
So opensearch also allows you to POST search requests, but those are uncacheable
QUERY would fit here perfectly - it's probably trivial for opensearch to add but it will take some time for clients to catch up.
And yes,it would be fixing a flawed interpretation of what should be implemented.you are, by definition GETting something.
Tools dropping body from GET by default are violating the spec today.
Rules configured to drop it are just that, temporarily configured constraints readily modified.
Adding QUERY will make it unpredictable in effectively the same manner as GET/body. It'll take even longer to resolve it though.
I disagree. I think the adoption (or dismissal) of QUERY will show.
First thing that comes to mind is that the idempotency of GET resources are easy to handle. URL's have a fixed size, they can be efficiently hashed, cached and are unambiguous about how they serve this purpose.
It is unclear how the ecosystem will deal with the QUERY requirements. It's easy for apps, but browsers, http caches and servers will take some time to figure out solutions.
Fixing GET would have the same amount of uncertainty in addition to the need to keep current expectations valid. It's not easier, it's harder.
No, really, they simply won't implement it.
I understand the confusion around GraphQL's cached/persisted queries, but this is not the intention of HTTP.
A POST can return Content-Location: to the GET resource, but the browser can't assume that the same POST body contents create the same result Content-Location, whereas the QUERY Location represents the QUERY itself as a repeatable object. (Also, QUERY can return Content-Location instead of or in addition to Location for subtly different caching implications.)
So much simpler...
Exist stuff (caches, CDN, etc.) could serve private information because the HTTP GET is cached without checking the request contents. The new standard can avoid this because old stuff does not know about HTTP QUERY.
This is about HTTP.
And it does not break REST: None of the HTTP constructs that REST is built on change due to the introduction of QUERY.
Yes: If you're doing QUERY, you're (potentially) not doing CRUD.
But this enables a clean way to do CQRS over HTTP.
There's usually a reason why the simplest solution that pops into one's head is not "just" used by the people who put a lot more thought into it. Not always, but it can be useful to try to come up with it.
So, either way, I need to update all the tools.
Just fix GET.
If you make it a whole new request method, existing unpatched software should just respond with "Method not allowed".
A whole new method whose semantics don't really fit with the others is.. An odd way forward.
RFC 9110 states:
> [..] content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request [..]
> A client SHOULD NOT generate content in a GET request [..]
You left out the important part.
By my estimation, that's why they explicitly call out only the client and origin can know what a GET body means; proxies should forward as-is and ignore.
Those days of ISP forward http proxies are gone, but those semantics are still fine; the body means what the origin and client agree it means.
I have a weird feeling. Query body is encrypted by https. So CDN will not be able to cache results. In order to make it work right - whole topology of the internet should be redone. Caching on the backend server will not give any real gains for large scale apps.
CDNs already terminate TLS connections so they can cache GET requests.
* the body may be compressed.
These two payloads are actually different. You're talking about semantics, which is determined by the payload format; in the case of JSON, these two are semantically similar.
> GET gets this for free because the URL is already a normalized string
It's the same principle; the order of properties matters too.