Ask HN: Where are the resources for complex architectures for Node.js?

> I'm having trouble finding conversations like that, regarding NodeJS. Has there been a debate about best practice? Where can I find it?

Here is the thing about high level modern interpreted languages — most of the time YOU DON'T NEED all those infinite pointless debates around what practice is the best or how we design another AbstractInstanceBuilderFactoryStrategy class etc.

Most of the time NodeJS just works. THere is really almost nothing to discuss, unless you face some specific unique challenge.

You just write clean single threaded code, and it just works as expected. And usually just scales horizontally to a very high volumes of traffic.

And the cost of change and dev experience is sometimes infinitely better than in Java/Scala stack.

https://www.zdnet.com/article/how-replacing-java-with-javasc...

Hence, you don't overthink about 120 amazing classes and interfaces in advance — you can grow your design together with the requirements.

Regarding the bottlenecks — the only nodejs bottleneck is pure sync calculations, so just never block the thread with heavy algorithms and you're good to go.

I'm speaking of personal experience working on loaded Java and Nodejs artifacts. It's just different, much less overheaded, way of thinking about the task and about the world.

Javascript is a simple language. The weird syntax edge cases can be avoided using a well configured eslint. In my experience any complexity in application architecture is a liability (Java programmers writing Javascript is a painful memory, don’t get me started…).

Before async / await, exceptions were not used much because with asynchronous callbacks you can’t really use them (there was the domains thing but that was discouraged). With await you handle any exceptions locally. The global exception handler is really only for last resort - I would only use it for logging and some sort of graceful shutdown.

NodeJS is designed to be a black box unless you are writing native modules which is not common. There are some useful knobs for things like the thread pool and heap size limit but that’s pretty much it. Low level performance tuning was officially discouraged because behaviour can (and has) changed dramatically if the compiler changes (e.g. Crankshaft to TurboFan change several years ago). It is also typically not necessary: nodejs is usually used for network services where network latency trumps all. Electron apps are an interesting new use case where performance tuning might make a lot of difference (e.g. vscode vs atom).

Usually software architecture is not language-dependant, so your knowledge from Java could be used here. For example, you can do hexagonal architecture in NodeJS. All of what you learned about exceptions can also be applied here. Probably most of what you learned about static typing too, if you work with TypeScript.

I think another part of learning JavaScript is forgetting what you learned in other languages and embracing JavaScript. At work most of the JS/TS code is namespaces and plain functions, with the occasional object when that makes sense. We don't use much the dynamic nature of the language in application code, but for testing we use it a lot.

For the performance part, I agree, it's hard to find resources. Part of it is because JS is "good enough" for most server-side software: the language is fast and asynchronous IO helps a lot. Most high-level advice is about being nice to the JIT: write code as if it was statically typed, let the JIT make easy assumptions and don't break them (objects that don't change, arrays used as arrays). Part of it may also be because the V8 team is constantly hard at work to improve performance. Their blog (https://v8.dev/) is a great read, I learned a lot here. There's another high-level advice that works everywhere: try to make a stateless app, so that scaling horizontally is easy.

I am in the same boat as you, Java longtimer-to-NodeJS/Typescript transplant.

My take after two years of production development with NodeJS is that the core libraries and runtime is fine, but NPM is a hot mess. The vanity projects and resume padding crowd out the library gems.

I do not like that compiled binary code can be included so casually into NodeJS apps and 3rd party libs, so I do not like to use it for deploying microservices to consumers. And for those use cases we stick with the JVM.

For developing internal apps and non interactive systems, NodeJS is very productive once all your mountain of tooling is in place. TDD is a must for large codebases, and we find Jest to be very capable and fast.

Exception handling is dependent on what middleware and libraries you choose impose on you. Generally speaking, you should have your code fail fast and let all thrown Error objects bubble up to a top-level error handler. Always throw objects that extend Error so that you get a stack trace that you can log out in the handler. Using async/await universally should allow your errors to bubble out through promises. I agree, exception handling is a tangle.

Performance is as simple as, use es6 classes where you can since Chromium can JIT those most easily. Set up your object properties in your constructor and do not dynamically add properties in any other methods nor change the properties' types. Assigning null or undefined to properties makes things a little slower, if you can find ways to avoid that, the JIT will favor you. Don't sweat it otherwise, Chromium is bloody fast. Get creative with WASM or workers if you absolutely must go faster, diminishing returns.

To your issue with lack of sage advice and books, just know that your having had that thought makes YOU the sage! You are the one you have been waiting for. Keep calm and own that role.

Be the tastemaker; the tools and libraries you picked are the best because you knew what worked best for your architecture and team. The low level system is not a mystery to you. You read the Promises spec and understand it the best of all your peers and find yourself explaining what's going on far too often to people who should know better. It's a toy runtime, or at least it started its life that way, and it needs your discipline.

Node.js doesn’t run on a full Chromium; just its JS engine, V8.

Ugh, I can't believe I made that mistake. Yes, just the V8 engine is included.

As others have pointed out, V8 is the place where low-level JVM-like things like hotspot compilation and garbage collection happen, and you’ll find most of those discussions under that label.

From a high-level perspective, the main simplification/complication with NodeJS and JavaScript is that it’s a single thread running in a loop, so significant amounts of work need to be moved off-thread to keep the main thread performant. This is where the “architecture” really comes in in my opinion (and is what makes NodeJS look “simple”: Everything complex is “outsourced”!) Deciding where that “outsourced” work will happen (workers, other node instances, database, etc.) and how will it be managed/monitored (queues and messages for example) is really the main architecture work of using NodeJS.

I think another part of what makes NodeJS look simple is that JS uses async a lot, especially in the browser, so when you do server-side JS and use async, it feels natural because that's how you do things in JS. In other languages, async doesn't feel as natural.

Over the years javascript has really evolved into a multi-paradigm kitchen-sink of a language, and because of its nature and evolution, there really isn’t such a thing as “idiomatic” JavaScript or widely agreed upon approaches to architecting a system.

From what I’ve seen, the architecture of a particular Node.js system is highly dependent on experiences of the team members that first built it. It could a hexagonal/DDD using a framework like Nest.js (the Spring Boot of JavaScript) or take a more FP approach using Express and looking more like Clojure/Ring. More often then not though, Node.js applications lack any sort of predictable structure because for a lot of teams, JavaScript is their first, and only, experience.

Long story short, I’ve been doing Node.js for 5-ish years now, and I have yet to come across what you are looking for. Every team/codebase is a unique snowflake in some way. Your best bet is to leverage the experience you have in Java/Clojure because JavaScript will be able to support 90% of whatever approach you decide to use.

you should be able to apply the same principals you picked up from developing in Java/Closure

node/js applications tend to be a bit more home grown than more traditional, convention driven ecosystems

for architecture: my "advice" would be to start with a single file, add things until you start to find friction (having multiple windows of the same file open is a good sign, constant searching too), break things into separate files based on your preference, folders and directories along the same lines, be consistent

your knowledge and opinions about errors/exceptions is valid here too

nodejs is fast but its not so much a performance focussed platform so ime, tuning options are lacking. there is an eco system of tools for managing stuff but it bleeds into the environment a lot more e.g. process management, ops and infra

Little side topic - care to say why are You switching to js? Is it Your decision or were You forced ?

I'm asking because at one point I was forced to work with react/node/graphql combo for over a year and wasn't very happy with it. After a year I could not be more happy to get back to something more mature and stablized. The whole js ecosystem seems really chaotic and feels bit like fashion industry but dictated by people that know nothing about fashion.

In the most recent case, I was actually hired as a non-coding high level tech consultant, to help hire and oversee a tech team, at a very chaotic but fast growing startup. For various reasons, we had to go most of 6 months with only 1 hire, when we had initially planned to hire 10 engineers. So I was forced to step in and start coding, because the team was desperately short of coding talent.

The startup is hopefully going to hire a real teach team early next year.

My typical consulting is something I talk about here:

http://www.smashcompany.com/business/the-ethics-of-being-a-h...

Coming from Lua, I kind of get your problem. Over there we often debate about whether or not localising module functions is worth the effort to save one table lookup per function call, but when you look for resources on optimising javascript, you often get much more high-level information that I had just taken completely for granted (and it's even worse in Ruby)

Would like to know about the resources for the complex architectures for Clojure as well.

Following this for more resources as I am also switching to node from jvm

My biggest pointer would be to remember that Java & JavaScript aren't named that way by coincidence. They're two different approaches to a similar problem. Java suffers from Enterprise Development (eg: Enterprise FizzBuzz[0]), JavaScript suffers from Ultimate Accessibility (eg: how many questions on Stack Overflow conflated jQuery and JS?).

> How should exceptions be managed? [...] Has there been a debate about best practice? Where can I find it?

I suggest you handle the errors you can and otherwise let it crash.[1][2] Debates in NodeJS-land have steered towards more monadic/Result-like structures and working synchronous-looking try/catch onto async/await. NodeJS and its various components are open source, you'll have a lot of luck looking around on GH for issues & PRs related to a feature -- same for the language, ECMAScript[3] officially.[4]

Since you mentioned Clojure, have you looked at ClojureScript?[5] That may be a good entry to JS authors & articles you'd enjoy.

> I have the impression that NodeJS is a bit more magical than the JVM [...] Is that correct? Where are good resources on this subject?

As other replies have mentioned, you're really talking about V8[6] for the "JSVM" executing that code. A thing I've seen throw some people for a loop is how minimalist the specification actually is.[7] The magic in NodeJS is certainly from V8 and the rate of optimizations there but also libuv,[8] what actually powers the infamous event loop.

Hope that helps!

[0]: https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...

[1]: Borrowing from Erlang, see Making reliable distributed systems in the presence of software errors, Joe Armstrong, page 104 "Error Handling Philosophy" https://erlang.org/download/armstrong_thesis_2003.pdf

[2]: _Most_ kinds of errors will cause the process to crash if you don't handle them, https://nodejs.org/dist/latest-v16.x/docs/api/errors.html . Promise rejections don't (yet) though it will log a warning, and callback-based APIs will always consist of an [error, data] tuple for the arguments

[3]: https://github.com/tc39/proposals

[4]: Because Oracle owns the trademark, of course: http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=75026...

[5]: https://clojurescript.org/

[6]: https://v8.dev/docs

[7]: "ECMAScript as defined here is not intended to be computationally self-sufficient; indeed, there are no provisions in this specification for input of external data or output of computed results. Instead, it is expected that the computational environment of an ECMAScript program will provide not only the objects and other facilities described in this specification but also certain environment-specific objects, whose description and behaviour are beyond the scope of this specification except to indicate that they may provide certain properties that can be accessed and certain functions that can be called from an ECMAScript program." https://tc39.es/ecma262/#sec-overview

[8]: https://github.com/libuv/libuv