Comment by Garlef | Hacker News Reader

Comment by Garlef | Hacker News Reader

Parent Full thread

I think the post is a bit unfortunate in its wording and this seems to have sent you off on a wrong track.

From how I read it, this the post is not specifically about JS and a discussion of the specific technologies mentioned but rather concerned with the following very general situation:

1. There's a user sitting in front of a browser.

2. There's a backend server providing data and points of interaction with that data.

I think the central point of the post now is that we don't have a satisfying technical solution for this situation.

Let's take a look at one of the points you mentioned. Maybe you might find there's actually some valid points in the post and give it a more favourable reread.

> UX when fetching data: What does that have to do with any of the above?

Here, the author of the post writes: "It’s a big burden for engineers to have to manually add loading spinners and error states, and engineers often forget."

I think it's more or less clear how you could implement this using only plain vanilla js. Cumbersome but doable: A very manual, imperative process.

Now let's envision a technology from a possible future:

    const twitterFeed = createMagicDataSource("https://twitter.com/...")

    const feedComponent = magicRendererComponent(twitterFeed, state => /* HTML like declarative description of the visuals */)

Imagine this was everything you had write in your code to get the following:

* state gets automatically loaded when the first instance of the component is created

* updates are automatically visualized in the client based on the internals of the declaration in the renderer

* you don't have to specify if the updates are done by polling, websockets or whatever: the two magic methods figure this out by themselves.

* you don't have to specify how the data is fetched in the first place. giving the a URI to the `createMagicDataSource` function is enough.

* additional instances of the component don't fetch the data again

* updates are efficient: the sync method only exchanges exactly the data required, only the minimal visual updates are performed

* marking feeds as "seen" by the user is also done by magic and syncs across devices (same for other non-ephemeral ui state).

Now: And I'm sure you'd agree that we are not there yet technologically. But I hope you agree that this would be really nice.

> But I hope you agree that this would be really nice.

Black boxes with tons of magic are great if you need to do exactly that one thing that creators had in mind. When you need it to work a little bit different you're usually either just out of luck, or the added flexibility makes whole api very complex (and usually buggy, as it goes hand in hand). It's just super hard to make things very high-level and simple, while still flexible enough.

Having a number of simpler functionalities that you're free to compose any way you like is for me preferred approach anytime. Most of the things that you mention can be solved with fairly simple wrapper libs doing just one thing (tracking state of connection, tracking errors, batching requests, etc.). Perhaps not in a perfectly reusable way, but good enough to allow easy recycling from project to project, and it's way easier to use and debug and adjust than if we had one huge built-in service object to handle it all.

> Black boxes

There are a few black boxes most people are very happy not to peek into; Having these black boxes is a huge productivity win:

* Compilers

* Garbage Collection

* OS Kernels

* File systems

* Docker

* App Stores / package managers

* Infrastructure as code frameworks (i.e. terraform)

* JS Frontend frameworks

I think there's three things at work here: 1. Familarity with treating item X as a black box (High for compilers, low for infrastructure as code). 2. Maturity of the interfaces (High for compilers, low for infrastructure as code). 3. Relevance of the details for your use case / business context.

> Having a number of simpler functionalities that you're free to compose any way you like is for me preferred approach anytime. Most of the things that you mention can be solved with fairly simple wrapper libs doing just one thing

I totally agree: That's how I imagine how the magic functions are defined.

> Black boxes with tons of magic are great if you need to do exactly that one thing that creators had in mind.

This totally depends what your business context is. For most companies, dealing with CORS in HTTP requests is very far from their business domain. Still, developers spend a lot of time configuring and building endpoints, API clients, etc. at a low level of abstraction.

I think parent is completely missing the point. In my opinion the way it should be is that in the backend you export a typed function like "getTasks(filter: Filter): Tasks

And in the frontend component you should be able to use it immediately and it would be correctly typed without having to duplicate types or anything. And if function is not complete yet it will not render the component before.

This should be instead of REST and Graphql, both of which have many issues imo.

So backend and frontend must have a wholly compatible type system? Even good old WSDL only got 90% of the way there. Keep going with fancier approaches and you end up looking at CORBA and its ilk.

Yeah they should be capable of having a compatible type system. So if you change a field name on backend, frontend won't compile unless you fix it there too.

> instead of REST

Why? REST doesn't have anything to say about this. You can design your application however you prefer, and publish input and output specs however you prefer, without tailoring your application to the HTTP layer.

Instead of having to implement REST apis is what I mean.

It could work with REST underneath or REST being auto generated based off your function names, but you shouldn't have to deal with that. And REST in general would probably not be optimal to transfer the data here.

I’m not sure what you think REST is suboptimal for here, the function you described maps trivially to it.

    GET /tasks?whatever=filters

I agree the part that wires up the HTTP behavior should be minimal and mostly uninvolved with your application logic. (I’m actually working on such a beast!).

Yeah, I'm saying that you shouldn't have to think of such things like URL mapping. What if now all of sudden you want to post json as a filter? You will have to start thinking of new arbitrary ways to handle this query etc. You have to both parse this new deep nested filter you thought of in backend and also you have to stringify it in the frontend. You add so much extra overhead to try to convert the query into some form of string URI that would adhere to URL rules.

Let's say you now want to add a filter for frontend where you can choose multiple dynamic fields and their conditions can be either greater than, equal, or IN some values, I think you get the gist.

You then will have to do it something like that tasks?status=in:todo,progress

Or status=todo&status=done But then you suddenly need greaterThan query. Are you going to do tasks?priority=gt:5 for example?

Or things like that. And you have to worry if this even adheres to URL standards. Things eventually get so messy. And you are very constrained in everything you do.

And how do you typecheck all of that? Also all of this is just error prune, you have to write documentation for it and every REST api is a little bit different, people often mess up when naming the urls etc.

> Yeah, I'm saying that you shouldn't have to think of such things like URL mapping. What if now all of sudden you want to post json as a filter? You will have to start thinking of new arbitrary ways to handle this query etc. You have to both parse this new deep nested filter you thought of in backend and also you have to stringify it in the frontend. You add so much extra overhead to try to convert the query into some form of string URI that would adhere to URL rules.

I’m not sure I even follow what you’re proposing. That URLs are insufficient for nested structural queries or that you want URL queries and POSTed JSON queries (which is not even REST) at the same time?

If it’s the former, this isn’t something a client or server should need to worry about. Server tools should make defining the API simple in the native language, and generate documentation which can provide client SDKs (again I’m building such a tool).

> Let's say you now want to add a filter for frontend where you can choose multiple dynamic fields and their conditions can be either greater than, equal, or IN some values, I think you get the gist. [...] status=in:todo,progress [...] status=todo&status=done

The most common way this is handled is either your second syntax or status[]=todo&status[]=done (or even with explicit indexes) to make clear it’s multiple values. AFAIK most major URL (de)serializers handle this automatically with no developer effort.

> priority=gt:5

Why not priority=>5? Again a library can trivially handle simple expressions like this, you don’t have to.

> And how do you typecheck all of that?

This is something the library should handle too. And it’s something I know about because I’ve built it (and again I’m working on one I can make open source). For a hint of how this might look and some prior art, check out io-ts.

> Also all of this is just error prune, you have to write documentation for it and every REST api is a little bit different, people often mess up when naming the urls etc

I bet you can already predict it, but the library should take care of this and I’m building it.

None of this should be so messy or so much work for people developing services. You got that right! But that doesn’t mean REST is bad, it means the tools for building and consuming REST services aren’t very mature. But they certainly can be.

Apollo client and Elixir LiveView are solving most of these.

Yes. But do they automagically create and maintain database indexes for the most common requests made by your webclients?

Should be doable. Applications only recently have a need for custom queries due to BI rising, so I guess DBs aren't yet there.

BTW IMO most of BI type stuff could be solved by just allowing direct SQL, but for this DBs would need to be written without buffer overflow bugs etc. I would love to have SQLite that has easy data-at-rest encryption amd that supports access control.

LiveView doesn’t allow arbitrary queries (or any queries really), so this is a non-issue.

Is PouchDB popular at all on the web these days? Things like Realm etc. on mobile solve this problem by just giving you a client side database to read and write from, transport to a backend is handled by a sync engine. I guess this is sort of what meteor was trying to do but it never really took off.

Reply on news.ycombinator.com