Browsers, JSON, and FormData | Hacker News Reader

Browsers, JSON, and FormData

blog.jim-nielsen.com

68 pointstannhaeuser52 comments

There's a lot of value in modern browsers asking "what are all these JS frameworks providing?" and providing primitives for the top 20% of features natively.

The existence of JS frameworks to build apps, in this sense, I think can be seen as a browser failure. The browser exists to provide a GUI for HTTP, and it's failing to provide "the common default" we now expect.

They tried, but made the mistake of stopping once the low-level stuff was done. Welcome to Webcomponents, it can do what you currently use Angular, Vue, or React for, but it only covers the plumbing, not the actual kitchen itself, so you either have to write your own kitchen, try to mimic the kitchens that vue/angular/react/etc already made, or use a kitchen made by someone else and hope that does everything you needed.

Which is an unfortunate side-effect of the whole "JS is about giving people APIs to make things with, not dictate what those things are" philosophy. In this specific case, that leads to APIs that technically cover the needs, while also offering exactly nothing that people can "just use" for writing UIs that are natively understood by the browser.

Meta: Plumbing and porcelain is the more popular pair.

https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Po...

What is the plumbing and kitchen in this context?

Plumbing = DOM updates and events, and in the case of Web Components, Shadow DOM for scope isolation. Kitchen = Declarative way to define DOM structures depending on application state. Plus optionally some conventions around updating application state. What it boils down to is "components".

Honestly sounds more like an argument for adding a <form enctype="application/json"> to the standard to me

Together with allowing nested forms (finally), I think this would be great.

Forms are for sending data to URLs, what would the use-case of a nested form even be?

Can't speak for the OP, but more than once I've run into a case where I need the form tags to be nested in the DOM, for reasons relating to the page structure but not the form contents. I would like nested form elements to just be allowed in the DOM, and if you submit the inner form then the outer should just be ignored, and if you submit the outer it should just submit all of the form values.

That sounds... confusing? What is the form supposed to do when you submit it if it has another form inside it but that form's values are... a separate form? Without a concrete use-case, I'm having a hard time understanding how this would make sense.

Today we have the form attribute [0] to alleviate some cases.

[0]: https://caniuse.com/form-attribute

Would this not be a(nother) good use case for htmx?

Nested forms would be great if you could prefix the name attributes easily. Like filling out a form where it asks for your name and your spouse's name. Now the first name and last name keys are in the form twice.

Okay but that's a terrible example: lots of people don't have the same last name despite being married. So you better use separate fields for that if you want "last names". But of course, programmers famously have no idea what names are, resulting in a never ending stream of "why the fuck won't you let me use the name that I have?" for anything from ecommerce to government websites, so you really shouldn't even have a separate first and last name field, you should just have a "name" field, and it bloody well better support all of unicode, because the only person who can tell you what their name is, is the person filling out the field, not you as form designer =)

[1] https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...

[2] https://shinesolutions.com/2018/01/08/falsehoods-programmers...

The quote in the beginning seems ridiculous on its face. I've heard a lot of reasons why people choose SPAs, and "I can make a transition animation between pages" is never it. In fact, almost no SPAs I've used have transition animations between pages.

I believe "swish transitions" refers to the lack of the page refresh flash, which really was one of the initial attractions to SPAs when they first came to prominence.

And now we have websites where clicking a link does nothing until a few seconds later the page content suddenly changes.

If you’re lucky, the page will scroll to the top as well!

> And now we have websites where clicking a link does nothing until a few seconds later the page content suddenly changes.

Worse yet, sometimes people do "page" transitions in JS event handlers, which means that all of the sudden you no longer can ctrl/shift + click these UI elements to open up the page in a new tab/window, or even copy the link.

Not that well made SPA solutions aren't pleasant to use, it's just that you can do all sorts of weird things that make using them miserable.

My health insurance provider has a website (seems mostly server rendered but with dynamic elements) that links documents that you can only access once (it seems). They're real links that will open a new tab when middle-clicked.

... but not of the document since the document is opened through some JS crap.

Thank you, Techniker-Krankenkasse.

I agree a lot of SPA implementations suck in the way you describe, just wanted to add that this was a issue before SPA navigation too.

Some websites used iframes to render the main content, and kept navigation controls outside the iframe. So if you open a link in a new tab (or window, before tabs existed), you ended up on a page with just the content without the navigation.

Some routers still use this way for page setup, for some reason.

Shitty implementations will be shitty, no matter how you do it.

I'd forgotten about those horrible iframes! It makes me wonder what other horrors I've left behind.

And then pjax/Turbolinks attempted to address that more surgically, without giving up on server rendering.

All modern browsers hold the old page for a bit so there is no flash.

Yet many web developers still manage to screw this up and cause flashes by doing weird things with fonts, dynamic loading and other such nonsense.

Well Cumulative Layout Shift (CLS) is bad on the first page load. It is not surprising it manages to be bad on every subsequent page load.

It is not that hard to fix either. There are several appropriate font swapping strategies built into CSS, and just reserve space by specifying the dimensions for images and dynamically loaded content.

Huh, I browse server rendered pages every day and rarely/never experience the page refresh flash.

The internet was a lot slower 15/20 years ago.

I'm not denying that it was accurate 15/20 years ago, I'm questioning whether it's accurate today.

Maybe there's an argument to be made for it in developing countries, but in my experience, SPAs are way, way worse to use than SSR pages on slow devices and unreliable connections. Buttons are unresponsive for a minute as JS is downloaded and parsed and executed, content can't render incrementally, and the UI softlocks waiting for a failed or timed out API request. People don't seem to be making SPAs to benefit slow and unreliable devices and connections.

I'm not arguing with you, I think most SPAs are unjustified, it's just that 15/20 years ago people really wanted to get rid of whole page refresh and all the problems that brings, which is a laudable aim, but has morphed into "why don't we make our own back button" and nonsense like that.

The justification nowadays seems to be (in truth) "if you take away my tools I don't know how to build things so I'm going to run npm init and go from there".

Looks like supporting an enctype of application/json was proposed, but never went anywhere.

Personally, I don't like the use of indexing to determine the JSON structure (like this `<input name="foo[0][1]">`). I think it's way more powerful to derive the JSON structure from the DOM tree. That's why I wrote some code to do exactly that: https://gist.github.com/mjepronk/5b33eaa90ecf11a5c2c47935a2c...

The use of `<fieldset>` and `<section>` may not be the best choice. You could use `data-` attributes on arbitrary tags for instance.

As someone who doesn't follow the standardization process, I would love an explanation how proposals like this fail even when they have widespread support and no downsides. Is there an easy answer to this or is it different from case to case?

From what I've seen, standards need to be picked up by browser vendors for them to become established.

Usually one vendor will implement it behind a flag, and if it gains traction then other vendors will follow suit before it becomes a generally available feature.

I wish front-end developers would use the right tool for the right situation. Not everything is a nail!

Forms are a great example. Browsers and back-end languages have supported those very well for a long time without the use of JavaScript.

Routing is another thing browsers and back-end languages do very well.

The SaaS we are building is in Laravel + Vue. Laravel has a powerful validation engine and top notch routing and session management. We leverage those 100%.

Vue shines for user interaction. Instead of a single SPA, we usually have one per rendered page (i.e. User Interface). Works very well.

I haven't found any downside to this approach vs a full-blown SPA. Only benefits.

Vue shines for user interaction. Instead of a single SPA, we usually have one per rendered page (i.e. User Interface). Works very well.

That means you have to parse all the JS on every rendered page.

Is this using Inertia.js? Did you consider Livewire?

> A big impediment to moving away from SPAs is that we’ve collectively spent years building up an entire infrastructure of backend services which power the internet by communicating with browsers in JSON

Something I like about Django/DRF (and likely many others I haven’t used) is that it makes the above an imaginary problem. Django trivializes accepting form or Json or xml or whatnot. It just wants a Python object after deserialization and usually you can get a few of those formats for free, or otherwise very little work.

This really strikes me as a shining example of "porque no los dos":

  "So many third-party services have APIs you can integrate with and guess what? They probably speak JSON exclusively. But browsers don’t send user-entered data as JSON by default, so what’s one to do? You can:
  
  "1. Create a URL that accepts the structure of a default browser POST request (<form method="post" action="/my-url">).
  
  "2. Throw progressive enhancement out the door, expect JavaScript to work for all your users, and drop a <script> tag on the page somewhere that handles the logic to submit a JSON payload to whatever service will store this information for you."

No, you do both: you create a server route that accepts that form's POST (or PUT or DELETE or whatever) and you write the code necessary to intercept the form's submit event and runs that same post operation backed by a Fetch request instead, then work the result into the page without needing the navigation action that a bare form uses. Congrats: your site now works both with limited-to-no JS, and as progressive web page when JS is fully available. All you had to do was stick with the basics: have a restful server.

Summary: article complains that JSON has a monopoly as the transport for web apps, they think the transport should be... HTML, you know the thing that doesn't know if it wants to be a programming language or a document mark-up, the thing with like a gazillion tags, none of which is about your domain. Hey what about GRPC, websocket... aka real transports?

Often, I'm going to have to be trucking an OAuth token around to make this API work and I'm going to want to respond to expired tokens and other exceptions in a rational way for the users benefit. Further, having History State and Custom Validation at my disposal are great tools to use when designing anything other than very simple forms.

I'm not sure javascript should be seen as a burden here.

People rely on browsers not being able to POST a JSON body to a (cross-origin) API to prevent cross-site request forgery.

Maybe this is obsolete in a SameSite=Lax by default world, but simply allowing browsers to support a JSON enctype has consequences that need to be carefully thought through.

If you wanted browsers to do this properly, you'd probably need to send a pre-flight?

this asymmetry between html form post to binary dump wire format bugged me a lot. So I made glue to convert formdata dumps back to html forms (which then can be processed with standard xml tools or viewed as is): https://code.mro.name/mro/form2xml

Works like a charm in e.g. https://mro.name/2021/ocaml-stickers/

this... this is actually what we did in 199x, before xmlhttprequest. there's a reason we switched away from that.

XMLHttpRequest is fine and groovy (if your user has Javascript enabled, of course). But why does everything have to be JSON?

Because it is a vast improvement over XML, the thing that XMLHttpRequest was designed to be used for. It was supposed to be an http request for a DTD'd XML asset, and thank god we stopped using it for that and went "no that's okay, it doesn't enforce XML compliance, we'll use it for any text content we want, thanks".

Which of course turned out to be incredibly dangerous when that text is (or is derived from) user-generated content, and that's where JSON comes in: JSON is a true data format, instantly turned into the native object that you can work with in the programming language that browser already has to speak in order to be considered a real browser. In stark contrast to XML, which ironically can't be turned into a data object, it can only be turned into a full fledged, separate document that you then need to use additional APIs for in order to extract data from.

XMLHttpRequest was a bad idea that worked because it was also the best option because it was the only option. Thank goodness we moved on.

I didn't mean to emphasize the 'XML' part of XMLHttpRequest; the payload can be anything. I think it always could be anything; I think it's always been possible to retrieve the unparsed raw data.

I know, hence the part where we decided to ignore XML and go for "arbitrary text", which is incredibly dangerous once we started working with user-generated content that we retrieved from a database and then injected into pages as innerHTML content, or loaded as off-page DOM fragment that we'd then copy nodes out of. In both cases, exploits were near-trivial.

Using JSON to communicate just the content that needs to be shown, and making the client-side JS load that in directly (no parser behind JSON.parse required) so we can either template that into the page, or create some DOM nodes that we know are safe because we're setting textContent, not innerHTML, is immensely better.

Could you do the same by sending yaml, or toml, or ini, or cfg or etc. etc. etc.? Probably, but browsers don't natively load any of those, whereas they do natively load JSON. Especially since the Fetch API landed way back in 2015. Native json loading has been a thing for 7+ years now.

JSON is close to using the most fundamental logical primitives needed to structure serialized data. It's a set of value types, an ordered collection and a key-value collection.

With a syntax that's ridiculously easy to write a parser and generator for (with the one exception of high unicode character escape sequences being encoded as two \uXXXX escapes after each other which form a surrogate pair rather than one \uXXXXXXXX escape, which is a bitch to deal with unless you use UTF-16 internally). It's certainly easier to write a mostly-correct JSON parser than an XML parser.

But why not just return hypermedia, assuming a web page?

What do you mean? JSON can be hypermedia, you can even click hyperlinks in browsers.

You can't return JSON to the browser. (I mean you can, but why lol?)

You have to interpret both up and down stream with a JSON parser (usually JS). In contrast, you can return hypertext/hypermedia directly to the browser and the user can interact with it immediately, without needing translation layers.

A primer: https://dev.to/rajasegar/html-over-the-wire-is-the-future-of...

Reply on news.ycombinator.com