People like me are why you shouldn't run a hosting company | Hacker News Reader

People like me are why you shouldn't run a hosting company

swizec.com

31 pointsingve33 comments

I think its weird that Vercel has this limit. There is no practical reason I can think of for having such a limit on URL characters that is so small. Chrome suggests a 2MB limit[0] for example. Firefox I believe if memory serves (I can't find the source for this claim atm) is 1 MB effectively, and I don't think Safari is any lower than that either (and may well be more inline with Chrome on this, at 2 MB).

The web as a platform has no defined limits at all.

Vercel is an annoyingly happy path platform and makes too many assumptions about how things should work or be built.

An explanation for such a limit would certainly be worth disclosing.

[0]: https://chromium.googlesource.com/chromium/src/+/master/docs...

The URL length limit seems pretty reasonable, especially since the URL is a highly user visible component, so overly long URLs annoy/harm users much more than web devs (just think of pasting this super-long url into chat or company wiki!)

In particular, I think the URL should be copy-pasteable at all times, and according to your google's link, that means 32KB limit. I'd lower it to 8 KB just to be safe in case there is URL encoding involved.

I don't find 14kb to be an unreasonable limit for URLs.

Agreed. Cloudflare limits URLs to 16KB, AWS ALBs are also 16KB, so 14KB doesn’t seem out of place.

While I have a sibling comment to yours saying I'm quite happy with the imposition of the limit, in my day job I frequently find code that pushes the boundaries of "reasonable" and I'm fairly sure we will exceed 14kB. But we don't use Vercel at work, so it's fine.

I suspect that while it's a reasonable limit for them, it probably won't work very well to impose a limit on an established code base that's not been built with a limit in mind.

I'm happy with Vercel's restrictions, especially considering I'm only using their very generous free tier.

The biggest practical reason I can think of is directly relevant to their free tier: they cache heavily, that's not quite the whole point of the platform but it's a large part of its appeal to me. They're obviously not going to be able to cache everything even with a limit as low as 14kB, but they're happy-pathing projects that are at least vaguely amenable to caching.

In the meantime, I get to throw code at them and not care about where or how they execute it, and I really like that.

There's no good reason for absurdly long URL's to exist in the first place.

URL's are meant for human consumption and use -- to be copied and pasted, to fit in a message, to be typed in. They're analogous to phone numbers and addresses.

They're meant to be a pointer, a location, an index. Not a data payload. That's what POST variables are for.

No, web browsers shouldn't introduce an arbitrary limit to URL lengths. But for web hosts, it's entirely reasonable.

I mean, what, do you think filenames or filepaths should be 2 MB as well? That's absurd. But URL's are basically just the filenames of the internet.

OTOH, they both are part of the message, the url having a more wasteful encoding. So it wouldn't matter that much, unless Vercel has a reason to store the URL in memory?

No, because the URL is not the message, it's the address. Per the acronym, it's the locator only. It's not for data.

URL's get stored in logs, analytics, and so forth, where they're expected to be short.

Payloads don't. So you put the payload part in the payload, not the address.

Maybe it is, maybe it isn't.

The nature of the web is changing, and has changed. Since GET requests can't have `body` data, if you have any hope of passing information along to a GET request, you must encode it in the URL in some way. This can be particularly important for things that contain unique but repeatable parameters that augment what you see on first load that would only be known by the consumer, and then can be shared by URL. For example, options set on a table you want to share with those options intact.

> Since GET requests aren't suppose to have `body` data, if you have any hope of passing information along to a GET request, you must encode it in the URL in some way.

Or in the request headers.

To be clear, I misspoke, I should have put effectively GET requests really can't have body data. Headers have the same problem. You can't possibly encode it without encoding it in the URL anyway

For which something like a 4 KB limit is more than enough, like options on a table.

The point is, once you switch from options to actual data, you switch from GET to POST. This is not a bug, this is a feature. POST is for posting data. GET is for retrieving it only, not for sending it.

More than enough in your opinion. It is perhaps your use case that 4KB works completely fine for, but this is not everyones.

We have a fairly complex set of options users can filter through (by design, and prior to its implementation, by high % of requests by users, FWIW) that need to be sharable on the GET request, they are definitely at least 12KB, likely more.

But that's far too large to easily share in things like plaintext emails or text messages.

If you're unable to compress it further, why don't you implement a TinyURL-type scheme? Save the options on your backend and give them a pointer to the settings. And it can expire after 30d or 1y or never.

Then it actually becomes the size of something shareable.

URLs are resource locators. You're not locating a resource, you're passing a gigantic configuration payload.

This is true, but it is the most efficient way that doesn’t induce complexity like having to serialize every option chosen to the backend immediately. Knowing we can re-create everything in first request is much more simple than other methods and doesn’t require the end user to get a special link generated for sharing they can just share straight for the URL bar.

Well there's certainly nothing that strictly prevents GETs from having a body other than no guarantee that all client libraries will handle them. I've seen a mix that allow it and disallow it. What is common is the libraries that don't support it don't seem to have any indication of this and from what I remember are likely to fail silently. Fairly annoying, so it makes sense why this is basically never used.

URL processing is the `/bin/sh` of browsers. You can stuff anything in there, even JavaScript, and it’ll mostly work across browsers long as you adhere to the syntax.

Should it be? Probably not, but it’s a nice-to-have if you need to do something quick and dirty.

Why have parameters at all if they're not meant to be used? I don't really buy the argument that you're allowed to use parameters for small things but not for large things. Personally, I prefer if sites give me a local-only option of storing my data and sharing it with others.

If you're implementing a web server you would want to allocate a fixed size buffer that can hold the request body or at least the request headers. Otherwise an attacker can cause excessive reallocations (memory churn) just by sending very long requests, and hold the memory until you time out their connection.

You can put much more stuff in the url fragment (location.hash). Like megabytes. No host can stop you, even vercel, as it stays on the client. As a bonus, you can keep all your logic in one place.

I'm not a fan of anything that has the word "rehydration" in it. Server is good at some stuff. Client is good at other stuff. Trying to blur the boundary seems like an engineering exercise in search of an application.

location hashes are an under-used feature of the web. I completely agree with this. It would seem however, that most applications that would benefit from this, the routing in use relies entirely on the web history API to determine what happens, but none simultaneously support both the web history API and hash fragments, which would really be the best of both worlds when used properly.

That said, there are many instances where you need server data and you don't want to get it post-render (thus avoiding the now dreaded spinners on first load), you don't have many options

I don't think there's anything about the standard history APIs that causes conflicts between history and url hash use. I don't really know how widely used client routers work, but this really sounds like a trivial fix. I'm probably missing something.

there isn't, as far I know, its simply that most client routers require you to make a choice: either use hash routing, or use history API. They don't have any built-in conveinences for using both together (namely, around things like getting URL parameters)

React Router, Vue Router etc. None of them have any consideration for using them together, they either don't support it at all or make you choose.

I don't follow, Why would hash based routing not use the history API?

I'm aware of the choice between path based routing, and hash based routing, which has some server side implications.

The History API manages entries in the client history, which I always thought a hash based router would still need to use. Maybe because a hash based router can work single page on the client without having to fake real resource paths?

edit: I'm only familiar with the vue side, but vue-router relies on the same code for both, I can only see minimal differences for the `#`, not much for the history management.

https://github.com/vuejs/router/blob/110c86929ac92e2428d8d49...

It’s more about the convenience features and melding of the two methods not so much that the underlying API prevents them from working together. Vue router doesn’t have a mixed mode for instance

I could be wrong but I thought there was a limit of like 4kb for a URL before you start running into issues. I don't remember if it applies to the hash but I remember one of my clients ran into that issue at one point trying to store their OIDC JWTs in the url fragment.

Experimentally, it seems to be at least 50kb, although I can't find a spec.

https://jsfiddle.net/Jz3ZA/

TIL: The limit of 255 characters in any distinct part of an URL seems to be purely fictional. I've always assumed it was a very clear cut standard, but now I can't find it neither in the RFC [0,1] nor the standard [2]

So, as we're down to opinion here, my personal pet peeve is: keep 'em short.

[0] https://www.ietf.org/rfc/rfc1738.txt

[1] https://datatracker.ietf.org/doc/html/rfc3986

[2] https://url.spec.whatwg.org/

The 255 limit would have come from distinct segments being limited by say the backing linux server rather than the web server itself

Or DNS for the hostname, which becomes 253 characters for us meatbags.

https://datatracker.ietf.org/doc/html/rfc1035#section-2.3.4

Compressing parameters isn't that overkill I think. In our Plinky synth web editor, we do some basic swizzling [0] to squeeze a 1152 byte array down to 213 url-safe characters so that the patches themselves are shareable with just a link, like so: https://plinkysynth.github.io/editor/?p=AR4CBNjaAFQCBUqATkGA...

[0] https://github.com/plinkysynth/editor/blob/main/src/lib/comp...

In case the author is here: In the first code block we have `<KeyInsight ... />`, should it be `<Improvements ... />`?

Reply on news.ycombinator.com