Do we need a web API? | Hacker News Reader

Do we need a web API?

liaison.dev

86 pointsfeross105 comments

RPC - this isn't anything new.

When I started doing web development in the late 90s for about ~5 years I was the designer, front-end, and backend developer all in one. Then over time things changed to allow for specializations and we now have many different functions, and that's a good thing. RPC has a place, and it's not for web apps, at least most of the time.

I've noticed over the years that JavaScript developers tend to try and put their JavaScript everywhere and I often wonder if it's because they don't want to learn other technologies. Don't get me wrong, I like a little bit of JS too - I even added JavaScript support to an old 2D Game engine using SpiderMonkey (before V8 was a thing) back in the day so that I could make desktop games with JavaScript. I just don't want to do _everything_ with JavaScript. Especially with so many interesting languages and technologies to play with these days.

Are JavaScript developers today the PHP guys of the past?

I'd imagine it's probably a business thing as well. The idea that they can easily transfer people from frontend to backend tasks or have full stack devs who don't need to know two languages.

Where I work, having one language has made things simpler when doing tasks. Don't need to look things up as much because you keep forgetting which one uses push vs append, etc.

> The idea that they can easily transfer people from frontend to backend tasks or have full stack devs who don't need to know two languages.

Oh, you're a poet, right? I need you to write some legal documents. You are already expert in English, so it should be easy.

Easier then writing it in another language. And you would get even better when you also know legal talk. That happened to me when I switched to full stack JS, I became so much better at JS, made it possible to write much simpler code. Doing very complex things with basic language constructs.

Wouldn't you prefer to talk in your mother language instead of using a new language for every country you visit?

At first, but not if it made it harder to communicate in each country I visited if I was visiting those countries regularly. I think the parent is likely onto something in that the spread of JS into backend services is not beneficial to those who spend most of their time on the backend, but it beneficial to those who don't. That's IMO creating an inherent tension.

Personally I have never understood backend JS as you get untyped, weird silent errors, under or completely un-utilized async features that end up causing more problems than they solve, and then all the quirks of JS to boot. I regularly hear the argument of the benefits of keeping the language the same on both ends, but besides slightly easier JSON between them (something handled by clean packages in almost every language these days with minimal effort/LOC/boilerplate) the languages don't end up actually sharing a ton given how you hook into a web framework like Express, database API's, etc. You don't get code reusability and I don't think the JS tradeoffs are worth the slight savings on context switching. If you have a serious project of any scale/long term use and have front end programmers that want to use JS on the backend, either bite the bullet and have them learn now or hire some backend devs in a language better suited to your project.

As someone coming from web dev to back-end JS I take with me the love for instant feedback, which allows for extensive manual testing during development. And I also use automatic regression tests before pushing to prod. Node.js is infamously known to use small modules, it however gives you battle tested code run by thousand in production.

I usually take a front-end first aproach, but there are also lots of people taking a backend first aproache where they rely more on static analysis, compilation, unit tests and packaging - rather then manually testing every LOC.

Instant feedback is pretty common on backend thought? I know that at least Flask and Django in Python have it.

And automated regression tests before pushing are in ever other language as well.

yeh, but why are there not tons of bugs in your code?

Is there anything you said that is specific to JS?

> the love for instant feedback

What languages can you not get this with?

> And I also use automatic regression tests before pushing to prod.

That's 100% language agnostic...

> Node.js is infamously known to use small modules, it however gives you battle tested code run by thousand in production

Many other languages have extensive packages pulled in, but that says nothing about how battle tested the code is unless all of the packages have high use. Python has plenty of packages with the same use level. Java has tons as well. The list goes on.

In the past when people have asked me questions like those they usually knew the answers. So may I ask, Why you are asking?

He's calling you out on the wild claims in your post that are not particularly nuanced or unbiased.

You shouldn't try to pull a fast one on HN readers, it won't work most of the time.

I would but it's impossible and/or impractical so I learned other languages like English and German.

Play with? Sure. But for actually doing "the real thing" reusing tech you already have expertise in seems more efficient than trying to gain an equal amount of expertise in another technology. Add to that, if you're in a hurry you don't even have the time to learn anything new.

For now, Liaison is JS-focused. So both the frontend and the backend have to be implemented in JS.

But the communication protocol (https://deepr.io) in between is language agnostic. So it is possible to imagine Liaison being ported to different languages in the future.

I've been thinking about language agnostic RPC as well. That would be so neat!

gRPC can already do that but gRPC's design is unnecessary bloated. The nice thing about RPC is its potential of zero-config setup. gRPC totally fails on that and it's painful to setup gRPC.

The RoR/python server could be used to automatically serve the RPC API. For a zero-config setup.

As other have noted, someone caught on to the simplicity of RPC.

It always baffled me how so-called RESTful APIs became popular, despite virtually all of them being RPC APIs with a veneer of rather pointless HTTP semantics.

Virtually none of those APIs do anything to serve the intentions and goals behind REST[1]. Furthermore, many RESTful APIs end up being wrapped into language-specific clients anyway, because they have poor ergonomics in their raw state.

I consider RESTful APIs the most significant anti-pattern not widely recognized as such.

[1] https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arc...

So true.

That said, for an API consumed by third-party developers, REST makes a lot of sense. Otherwise, which is like 98% of the APIs out there, RPC really is superior.

I've implemented a tool similar to what OP is building: Wildcard API [1]

> rather pointless HTTP semantics.

Agree. Actually, while implementing Wildcard API, I made sure to abstract away all HTTP semantics. With Wildcard, you don't think in terms of "HTTP verbs" but you think in terms of functions. Like you are naturally used to.

> It always baffled me how so-called RESTful APIs became popular

I believe REST became popular because it got lot's of exposure since all third-party APIs were using REST.

[1]: https://github.com/reframejs/wildcard-api

> the simplicity of RPC

Simplicity? Have you ever tried to integrate distributed applications that do not share the same codebase?

And calling REST anti-pattern tells me that you aren't that familiar with alternatives.

>despite virtually all of them being RPC APIs with a veneer of rather pointless HTTP semantics.

If you want to do RPC over HTTP and pretend that it is REST interface, it isn't really a problem of REST.

this is just JSON RPC? The "constantly serialize and deserialize it" statement is totally untrue too... The frontend makes 6 XHR calls just to render a single page and they're all just JSON. The idea of the frontend 'just calling' the backend with no API boundary is nothing short of atrocious for security.

The only feasible logical end to this is ASP, which was an interesting idea when it was launched 19 years ago

This has nothing to do with JSON RPC. Liaison uses Deepr (https://liaison.dev) as a communication protocol.

Regarding the number of XHR calls, to be accurate, it is not 6 but 4. Once Liaison is complete and optimized, the number of calls should drop to 2.

Also, please keep in mind that Liaison is made for building single-page applications. So, using it for a simple blog might not be the right choice.

And about your security concern, please head over here: https://news.ycombinator.com/item?id=21660785

In my view you just replaced an explicit API built on top of known protocols, by a semi-magical implicit API.

The problem of serialization, deserialization, versioning, endpoint design... They aren't an artifact, they're intrinsic to how Web apps are built.

But yeah, for small projects I think this could be really neat.

Having a properly defined API gives you a nice interface to test.

CORBA called, they want their ideas back

Yup. Looks exactly like it. SOAP is not that different either.

Add: my PTSD from a few months with DCOM is acting up :)

Even in JS, the Meteor framework was basically created to implement this.

Except they actually implement things like validation, because as much as one may want to unify both environments, the reality is that exposing all your methods to the outside world may have some pitfalls, to put it mildly.

What ever happened to Meteor? It was flavour of the month a few years ago, with many a frontend dev ranting about how great it was. Now seems to be forgotten about.

Hassles with installing NPM packages (they used their own package manager (!)), it was incredibly heavyweight, baked in reliance on MongoDB, and a weird templating language caused a kind of death-by-a-thousand-cuts situation.

Plus, React came out and blew Meteor out of the water.

One of the early Meteor developers, Evan You, went on to create Vue.js.

We tried to use it in real life for smilebooth. It was nice for little sample apps but as soon as you hit something it was not designed for, the workaround was more painful than if you had just gone with a plain rest api. Scalability was pretty limiting too.

Facebook came out with GraphQl and they went on to build Apollo for it and left Meteor behind.

Smart phones. Meteor came when people used browsers. Once the app model became prevalent, and users expected native integration, slow butt JavaScript in an HTML5 app on your 2012 phone CPU was a non starter.

I ran into a company recently that was still using Meteor (B2B with roughly 50 million USD of funding). It made me realize the consequences of choosing a technology for a major project before it's gotten widespread acceptance. Made me realize I need to be a bit more cautious, and that I've gotten very lucky.

I worked on an app that was Angular 1. That did get widespread acceptance at the time, but still got dropped like a hot potato when React came along. Probably the backwards incompatible changes with Angular 2 were a big factor as well.

And Opa language and framework. (To note, the language itself was awesome, too bad it was tightlyccoupled to the framework since that approach never seems to work)

By default, a Liaison backend doesn't expose anything to the frontend. For a method (or an attribute) to be exposed, it must be explicitly declared as such.

Yup.

Diagram from this project: https://github.com/liaisonjs/liaison/blob/master/assets/cros...

Diagram from CORBA: https://en.m.wikipedia.org/wiki/File:Orb.svg

Its good that expensive operations are explicit. Every now and then developers forget this and create garbage applications using frameworks that are set out in similar ways to this.

Don't hide the communication layer pls.

There are already frameworks doing this. For example Blazor (.Net core 3) and Phoenix Live View (Elixir).

They write the Javascript for you but you won't even notice it.

I don't have experience with Phoenix, but with Blazor I just write my complete SPA in C#. There is no API for communication between the server and client. Blazor creates all the JavaScript websocket code for me so I never need to write a single line of Javascript.

And when you separate your business layer from Blazor you can always add an API in the future when you need one. Just create some controllers for your API and use the same business code Blazor is using.

First Law of Distributed Object Design: "don't distribute your objects"

--Martin Fowler

A Liaison backend is stateless, so there is no shared state to worry about.

Pouchdb/firebase/backend-as-a-services were supposed to be the answer to getting rid of CRUD apis. The problem is authorization and validation becomes really difficult if you don't have a backend that you can trust.

10 years of writing web application backends has taught me that yes, authorization is indeed the reason we all still have jobs. And the occasional handling of a transaction failure.

This. If you can just trust the clients, lots of things get easier. Though even then distributed systems are not trivial -- just look at Lustre (where clients in a cluster are all trusted, and if you don't trust some client, then it must use a proxy such as an NFS server running on a cluster member client).

I think Firebase is a much nicer version of this in the sense that it essentially eliminates the backend altogether and allows the client to talk to the DB directly. It is definitely not for every project, but if your project pretty much only needs CRUD, it is amazing.

Liaison has nothing to do with a backend-as-a-service. You can host your backend anywhere you want.

About authorization, Liaison doesn't expose anything by default. Only the specified attributes and methods are exposed, and there is an authorization mechanism so you can customize what you want depending on the user.

As has been noted in other comments, there is a growing list of technologies that implement some variation of RPC such that events on the client trigger server events. Those events persist any data changed and then regenerate the current state of the UI and ship the diff back to the client, ideally via calls to the morphdom library.

What frustrates me is that it's not the API that is the source of complexity. ~80% of the pain of building most apps comes from managing the syncronization of state between the client and server. SPAs are the over-prescribed opiates here making the problem worse. Imagine what could be possible if you walked away from maintaining client state altogether.

We did, and it's wonderful. StimulusReflex let me build a reactive tabular UI with sorting, filtering and pagination with 115 LOC, none of which is custom JS: http://expo.stimulusreflex.com/demos/tabular

I honestly believe that the existence of things like StimulusReflex and LiveView demand we question the ongoing utility of libraries like React. Popularity is not a good reason to use something, especially when you realize you could be radically more productive if you don't use it.

Things like this have been tried many times before. (I made one myself back in the day). It always seems like a great idea up front, but never ends up quite working out.

One of the big things is that the benefit declines over time. The amount of code reuse you get is sub-linear with respect to the size of the project. As the project matures, GUI code and back end code diverges. That leaves you stuck in a framework that ends up not necessarily optimal for server side, and not necessarily optimal for client side, and the amount of code you end up writing to coordinate between them ends up outweighing the actual business logic.

The things that are successful tend to focus on problems that increase with greater than linear complexity as your app matures. Spend time focusing on these instead. Don't lock yourself into a framework that prohibits it.

> As the project matures, GUI code and back end code diverges.

This. This is why ASP.NET, ServerFaces, JSP+Taglibs, etc. became a nightmare.

Your point is valid, but Liaison allows you to build different layers if you wish.

If you worry about cluttering the business logic and want to clearly separate the exposed API, you can expose subclasses of your domain models instead.

I'd still say the established routine of publishing a public API and any shared code can go into NPM or whatever is the better approach.

Liaison and tools like it essentially create a monolith exactly where something is screaming to be two separate services. You want to be able to deploy versions of server side and client side code independently. You want to be able to ensure you don't break clients that have web pages open when you deploy the new server version. You want clients to be able to run where JS isn't available.

About API versioning, the problem is the same as any web API. It's possible to add backward-compatible changes, otherwise, you need to fork the backend into a new endpoint.

About interoperability with non-JS environments, I wrote an article about that: https://liaison.dev/blog/articles/How-about-interoperability...

I don't mean to knock it. It's a cool project and obvious you've done some good work on it. But, I wouldn't expect it to lead anywhere beyond a couple sample POC apps. It's been tried before and never quite seems to work out (I did something similar with VB6 forms way back when. It didn't work out. Meteor.js is one that probably came the closest to real-world success. It didn't work out.). There was a bit more interest in this approach back when SPAs were just coming onto the scene, but in the time since, the programming community seems to have converged around the idea of well-defined boundaries and interfaces, allowing use of the best tool for the job in each individual service, and abstracting out the distributed communication layer with a metaprogramming tunneling approach has somewhat fallen out of favor.

If your goal is just a brain workout then by all means keep going with it. But if you're looking to do something that gets some larger adoption, then I'd probably wrap this up and move on to the next thing.

(Not that you asked my advice, or that I'm in any way qualified to give it)

Having clear API allows to have a clear boundary between trusted and untrusted environment in backend. If you don't check your arguments in REST endpoint, it's obvious bug and easy to spot. If it's just another method, well, you're not treating input in every function as a hostile.

Please see my answer here: https://news.ycombinator.com/item?id=21660785

Many years ago I remember using a similar shim on ASP.NET where you'd simply decorate a server-side C# method and you could call it from JS like any other method. Not quite the way done here, but it had similar goals to blend server and client. So far, these things don't work in practice because of the so many things that differ between a browser and server environment.

https://docs.microsoft.com/en-us/dotnet/api/system.web.scrip...

I wish the "really" wasn't taken out of the HN title. It gives the title a somewhat different meaning.

I wish that too. :)

lol, coming from PHP, JSP and co, web APIs were the salvation.

Monoliths are a big problem and merging frontend and backend into each other only made it worse...

The readme says: "The current state of an instance (i.e., counter's attributes) is automatically transported back and forth."

How does this work? Do you need some cross-network locking mechanism when you need to make sure the shared object's state doesn't get out of sync between client and server?

What about passing object instances between client and server? Does it do what (I think) DCOM does to handle garbage-collection of cross-machine object references – keep-alives and timeouts?

A Liaison backend is stateless, so there is no shared state or garbage-collection to worry about.

I daresay that if you need to:

a) scale, and

b) adapt over time

you're going to find yourself decoupling and slapping APIs on both ends to adapt to the transport mechanism.

So for a boutique app, you might get away with a client-server model.

Liaison allows inheriting layers from each other. So, you can easily separate your API layer with your model layer, and you don't need to build a web API for that.

This is an interesting project, but there are very good reasons for having API's.

Its just about is it the right tool for the job.

I don't like their comment : 'They are just a necessary evil so the frontend can communicate with the backend'

I'm sympathetic to your position, but right now your comment doesn't carry much information. Why are they the right tool for the jobs? Which reasons are those?

Not to offer information, I'm just objecting to the stance they seem to make that : "Hey we're solving the 'API' requirement problem' when there really is no problem..

If your backend only communcates with an SPA and nothing else, then yup, this is a good tool for that job.

Actually this doesn't solve the core API challenges that SPAs have - versioning and compatibility. Clients will simply run outdated frontend code. Imho this is more trouble than worth as it will bring all the problems that we had with WSDL.

If i'd do a SPA that has 'node' backend, i'd just use typescript to model the API and you can have shared models and compile-time type safety.

https://news.ycombinator.com/item?id=21600859 seems related.

I've used a similar concept about 10 years ago if anyone remembers Wicket[0], and personally really liked it. The challenge was the UI and state were all server side and difficult to keep memory usage down. I'd be interested in how this model works now that client side UI and state are much more capable.

[0] https://wicket.apache.org/

I liked it as well. Web development had gotten way harder over the last decade.

Actually RPC is a good idea for front-end to talk to backend.

Because the separation is more or less obligated if people wanted to do interactive web pages, instead of people want to separate them as different services for different problem domains.

People who don't like write JavaScript can stay tuned, as WASM age comes, there would be a burst of things like this but not in JavaScript (eg: Blazor).

Dont trust the client! Serialization should be abstracted. Btw. how does this work without serialization? :P (I guess its abstracted). With an abstraction layer, sending a request can just be an azync function call. There is also the option to keep the front end stateless aka server rendering like in old school web apps.

Have a look at Wildcard API[1], it's similar to what you are building.

I'm the author. I'm glad to see that RPC is getting more and more traction.

[1] https://github.com/reframejs/wildcard-api

A project using Elixir + Phoenix would be way cleaner if one would like to leave out the API in web dev.

isn't this a weak ass version of RPC/gRPC?

https://grpc.io/

- and thank God gRPC never took off the way Google hoped, it was pure ego and politics like the majority of OSS projects these days (in the web world)

Can you elaborate on the "ego and politics" part?

I have a very personal history with gRPC

Is it bad? My operating systems class had a project based explicitly around it

As someone deploying frontend apps for a large site you should not underestimate how long people have their tabs open. I can get sentry error reports for versions that have not been deployed for months.

Does this handle "API" versioning in any real way?

For now, with Liaison, API versioning is no different than usual: preserved endpoint for backward-compatible changes and new endpoints for breaking changes.

Seems like a WSDL endpoint from which you can generate eg. a client for .NET.

Reminds me of https://github.com/pkamenarsky/replica for Haskell

It's always cute to see JavaScript developers reinventing things that have been in the industry for 30 years. Welcome back RPC.

Postgraphile is an example of reducing the number of layers: directly expose PostgreSQL schema (with row-level security) via GraphQL.

But now you added the GraphQL layer on top of HTTP.

one word: security

Liaison doesn't expose anything by default. Only the allowed attributes and methods are exposed.

Does this force you to use node for the back-end?

For now, yes. Liaison is implemented in JavaScript, but the communication protocol (https://deepr.io) is language agnostic. So it is possible to imagine Liaison being ported to any languages in the future.

>Sorry, this site requires JavaScript to be enabled.

For a flat blog post? Like, seriously?

welcome to the web 2.0

cooming soon: web assemply web 3.0 which (I fear) might feel a lot like flash web pages from the early 00s. I hope I'm wrong.

got to get the fuck out of the industry professionally at this point

Web 2.0 was like 15 years ago now. Web 2.0 was "supposed" to be about separation of style & content and gracefully degrading JavaScript.

I don't know want to call what we have now.

Probably the "Web App" or "Web SPA" era, since everything seems to need to be a single page app that exists mostly in the client.

There are some benefits, of course. Stable and standardized UI is finally available as a baseline for devs (much like early Windows/iOS software) and all roads are clearly leading toward a unified design model for phone apps/web/desktop. Bloat and needless complexity are the biggest issues at the moment, but all the frameworks seem to be (finally) focusing on becoming more snappy.

Webasm doesn't enable anything new that couldn't already be done right now.

but it will improve the ability to obfuscate webpages

I'd probably rather sift through a disassembler than obfuscated JS.

I was replying to someone saying that webasm will bring flash style webpages.

I had to disable AdGuard to display it.

I’m glad that the JS world is shifting from SPAs to SSR. Using a SPA for a blog is just bad on multiple levels.

IMO SPA's aren't that bad, but loading large assets from third parties is.

examples?

the OP message falls flat when it can't even be bothered to server render.

also, most offensive is the use of 3rd party js hosting. given the tiny size of the 3rd party libraries (relative to 1st party code), add them in to the code from your own site (which you're already bundling up. SMH)

[from jsdelivr.net]

polyfill.min.js (31.8kb/96.8kb) (wire/resource)

url-polyfill.min.js (1.9kb/6.1kb)

react.production.min.js (4.7kb/12.3kb)

react-dom.production.min.js (36kb/116kb)

[from liaison.dev]

bundle.a7c7d78d57.immutable.js (108kb/375kb)

actual website code: https://github.com/liaisonjs/liaison/tree/master/website

The actual text of the article (not including the table and image) is about 2700 characters long.

I don't understand why it is wrong to use CDNs to distribute common libraries.

About the fact that I have used Liaison to build its own website, I agree that it might not be the best choice. :)

For now, Liaison doesn't support server-side rendering, but it is something that could come eventually.

Reply on news.ycombinator.com