Having the same language across our entire stack has huge, enormous, gargantuan benefits that shouldn't be underestimated, especially for a small team. Being able to easily move between backend and frontend code bases has had a gigantic positive impact on team productivity. Couple that with auto-generating client and server-side typescript files from our GraphQL API schema definition has made our dev process pretty awesome.
Our app doesn't have a huge need for caching, but we use a mix of in-server-memory caching ( https://github.com/isaacs/node-lru-cache ) and Redis when we need a global cache.
Writing a migration tool is pretty trivially easy - all knex does is let you write an up migration and a down migration, and it keeps track of which migrations have been applied in a DB table. Main thing is that all of our migrations are each in an individual versioned file in our source repo.
We use postgres COMMENT functionality to apply comments to all of our table columns.
We have services that don't ORMs and I have to wonder if it's worth the cost.
Basically, in my opinion ORMs just make the easy stuff slightly easier, but they make the hard stuff much harder, and when you're stressed out trying to fix some critical production DB query all they do is get in the way.
Joking aside, in a dynamic language like Javascript, especially in modern coding style which is not OO anyway, you don't need an ORM.
Joking aside, I do write SQL statements (or use a query builder, which is not the same as an ORM).
I don't "turn the response into classes by hand in every app I write" however, because the responses are perfectly usable as they are (in a more functional style), and OO is not the best way to model records anyway.
I know Ruby and Python have ORMs, ActiveRecord, SQLAlchemy and so on. They're not really needed. Heck, I've read authors of ORMs saying you don't really need one...
I think the answer depends on the type and load/volume of app you're working with combined with the dynamics, size, and skill level of your team(s). I'm extremely comfortable writing, profiling, query planning, and debugging SQL queries. Others aren't, and therefore having an ORM to query data in the DB with the syntax of the language you're using in your projects makes way more sense, if nothing other in order to speed your team up.
2. project search and replace on $COLUMN_NAME
If your column name is a common keyword, variable name, etc. in your code base and it's difficult to find using project search, that's unfortunate, but we organize our backend code and tests in a logical enough way that it's never taken longer than an hour to create a PR to create a migration to rename a column and update all places in code that reference it.
IMO an hour for a change you have so little confidence about is not acceptable when the alternative allows you to do it in a second with full confidence.
Keep in mind I've installed and used an ORM in projects where the ORM is used only for migrations, but not used in application code and this is absolutely a fine reason to use one imo. But adopting an ORM for migration purposes and forcing or using in application code simply because it's installed isn't necessarily a good approach.
ORMs are to SQL what static types are to programming languages. The conversation we just had was me giving you one example of the benefit of static types, which happens to showcase a huge weakness in dynamic typing:
- How painful is it to rename a class attribute?
=> It's a long search & replace exercise which results in a less-than-certain outcome.
The obvious take from this isn't that "renaming class attributes is rare".
ORMs have very basic support of current SQL standards and database specific features.
This means that using an ORM reduces the power of the database choice you made.
Also things like arbitrary SQL support imply you have to manually creat return value typings.
Having to leave the nice ORM wrapper functions for arbitrary SQL means you lose all the ORM niceties like soft deleted or updated_at fields
Overall I see very little use for ORMs.
We have a TypeScript Node.js API in production, and we wrote Zapatos to be that something: https://jawj.github.io/zapatos/
same here, except Apollo GraphQL, we still use REST. it is indeed nirvana. wish the community finally settle down on a stack at least for a decade.. shifting the backend every few years doesn't do good for dev productivity.. (had to recently use Go due to peer pressure. while performance wasn't a concern it was purely due to the "feel" that Node.js is not good enough for serious backend work - lack of multi threading , potential future performance and scalability etc.)
That aspect of Node.js can actually be a very good thing for serious backend work, as it encourages process decoupling using API's. Decoupling ends up scaling better later on: you build a distributed system that can be scaled out horizontally across a fleet, rather than running up against vertical scaling limits of how many threads you can get running on the same local hardware.
Whenever I've seen applications that use worker threads for backend systems they end up regretting it and wished they had decoupled into a separate process that could have been scaled independently onto other hardware over the network. Spinning up new threads on the same machine is a temporary crutch that bites you later on in your growth.
I don’t think this is a particularly compelling argument for the lack of multi-threading. The worker thread pattern is fairly useful and there is no reason a well architected application couldn’t use both horizontal and vertical scaling. Independent scaling is only really useful if there is a large variation in the amount of work done by the workers per query. If it’s well bounded, then it might be easier to just horizontally scale the entire backend.
There are also other use cases for threads that don’t fit into the worker pattern. For example, background tasks that happen outside the serving path.
Would you mind enumerating them? I have an idea of what they are but curious about other perspectives.
My sense is that code sharing is not that common between a Typescript frontend and backend. You mostly need generated request/response data types but I don't think there's that much shared behavior because you can't import any of your backend-y logic (database, auth, external APIs) transitively into your frontend.
I think primary gain is what you've hinted at: 1 ecosystem and it's easier to onboard fullstack devs.
// shared code - implement the algorithm
export async function loadPageChunk(
args: LoadPageChunkArgs,
loadRecordValue: loadRecordValueFn
) {
// ...
}
// client code - use the algorithm, provide client-specific IO
// eg on Android we'd use Sqlite.
const records = await loadPageChunk(cursor, SqliteService.loadRecordValue)
// Server code - same, but use the server's data stores.
// Behind the scenes, these loaders batch, etc
const records = await loadPageChunk(cursor, useCache ? CacheService.loadRecordValue : PostgresService.loadRecordValue)
Previously I've done this with 2 implementations (JS on frontend and Java on backend), but then keeping the logic in-sync is a nightmare. With a single language you can just share the library.
You are correct, though, the biggest benefit I see is not sharing code, but making it trivially easy for a front-end dev to add a small piece of backend code that they need without needing a back-end dev to do it, and vice versa. It just makes the overall team much more productive because there is very little "waiting on the back/front-end" to do it.
Oftentimes we'll have either team write up the schema for a new endpoint using the GraphQL schema definition language, then from that we autogenerate the TypeScript types, then usually the front-end team creates a simple mock in the backend so that they can fully implement the UI, meanwhile the backend team works concurrently on the real implementation. This process allows for much more parallel productivity than if, when something is broken or needed from the other team, you just have to submit a ticket and wait.
Problems I personally have with it:
1. no exact object types in ts as in flow - means they have to be emulated by destructing, sad, but you can live with it/you have to be careful
2. transpile times - but recent experiments with swc for tranpilation and deferring typecheck to run concurrently while tests are kicked off after swc finishes look promising
3. type system could be a bit smarter in few places, but no blockers so far
I'd recommend but with caution - spectrum of developer's competency is closer to php (almost anybody can do it) than the one of languages like ocaml/haskell/rust and others (where entry bar is higher). Vet your dependencies, hire competent developers and it can work very well.
Some of libraries we're using:
- https://github.com/appliedblockchain/assert-combinators - light, runtime type assertions ("parse, don't validate" style to avoid illusion of type safety at io boundaries)
- https://github.com/appliedblockchain/tsql - functional, tagged template based combinators for sql generation
I personally ran mission critical node services that interfaced with over 100,000 simultaneous compute nodes in AWS.
Node is not for UIs, so they're definitely not using Node for that. It was confirmed that the UIs in the Dragon capsule (which, sorry for being pedantic, is not really the "rocket") ran on top of Chromium. It's possible SpaceX uses Node under the hood somewhere, but I don't believe that has been confirmed anywhere.
Also kind of interesting to note that it's only the Dragon capsule with humans that has controls, the capsule (and rocket) are both autonomous. The controls on the capsule are only there "just in case".
So if they’re running their UI on Electron, it could be on Node.
Here was the meeting:
https://www.meetup.com/pdxnode/events/142646682/
* Ben Acker will share about some awesome drawings and tales of Nodejs within Walmart Labs.
I said Node is dominant in web/APIs, but Fintech I don't expect to go for Node, if we're talking trading, but also for CRUD work.
DDEX is the industry standard for metadata communication (artist, track, licensing, credits, etc.).
One 20-track Xmas classical music compilation album could easily have a single 150MB XML file!
Companies started 10 years ago tend to have Java / Ruby backends, but companies started in the past 2-3 years will often have NodeJS/Typescript backends.
But this is (way) less than 1% of code and typically performance is not the problem with Node, even for our use case. ~Everything we build feels fast the first time. If on rare occasion it does not, it’s a matter of rearranging the building blocks, not swapping them out for something else entirely.
That is, where "mission critical" means "critical to some particular business". Not guiding rockets or critical medical use, etc.
Also as a data service for connection pooling databases that don't support connection pooling with supplied drivers or server-side.
Live medical data processing, large ETL pipelines, and coordination systems that set 100% uptime as a goal and any incident would have _very_ thorough RCA, retrospectives, and accountability reports.
While traditionally these were built with things like Erlang, Java Ecosystem tools (Camel, etc), the only time that using node had serious downsides was when integrating to particular languages ecosystems as a second class citizen.
Examples:
• Kafka. Until recently the node-rdkafka wrapper had quite a few bugs compared to the java client. We ended up writing our own internal client in typescript. Performance was worse, but we had easy scalability of our producer services and we were able to track down and resolve any issues quickly.
• z3. There hasn't been an official build for node yet. We had built a wrapper to use a specific fork of z3 that we had, which allowed identification and distribution of shared subsets of problems. This worked with an etcd-like streaming consumer where you would get live-pushed keyed-problem subsets and utilize those in a local in-memory cache to speed up solvers that overlapped.
• Distributed Actor-Like framework. Obviously Erlang, Akka, Akka.NET, etc are the prior art here. It didn't take too long to have our team analyze these and build out mimics in typescript.
• Wrappers for specific C++ statistics libraries. Some of our ETL pipelines would enrich data with a pass on certain identifiers/classifiers/aggregators. For a few of these we created js/ts wrappers, but it would have been nice if they existed.
• Standard Library. We took a microsoft-like approach and just created a standard library for ourselves. Tested and with lots of features, it meant that we rarely had to reach outside of our ecosystem for Collections, Encoders, Crypto, etc, and that they were all documented to our internal standards. If there was an issue, you had someone you could ask and get an answer within the hour.
Unfortunately all of the above is proprietary and we weren't allowed to release any of it.
We did retrospectives on technology choices and limitations once a year, to learn from decisions made going into the future. Each time when Node came up, the general consensus was "we could have done it in Java I suppose, but the Typescript/Node combination was much quicker to iterate on and we felt more confident in the solution after the fact".
Were I to do it over again, I probably wouldn't choose node, but my problems with it haven't had to do with static typing or type errors--we don't use typescript. Where we've continually struggled is indeterminacy in process control and error handling and writing robust services in light of that.
* Async operations destroy stack information.
* It's very easy for someone to miss an error handler and end up with your process in an ugly state.
* Default error handling is "crash the process", no matter what else is going on.
* Lots of libraries that rely on buggy native code.
Typescript/Node.js/GraphQL back-end with React/Relay/Typescript on the front end.
https://github.com/coralproject/talk
It's pretty nice having the whole code base share types, syntax, structure, etc. It's served us well for many years!
Some of our clients include: The Washington Post, New York Times, Wired, USA Today, and Financial Times
IIRC, the performance of Node was ok but clearly worse than Go/Java/etc. Uber was using JS not TS back then, but the real issue (at least when I started) was lack of a defined interface for the API/mobile app communication. That was eventually addressed by adopting a forked version of Thrift.
It also used Node for a very core part of the app, and it was Node 0.10 to boot, but my understanding is that that's on the way to deprecation.
Microservices themselves are all in go or java these days.
Using Node.js for (large scale) mission critical backends, mostly in JS, but (on the topic of typed stacks) more and more of it is becoming Typescript.
> Wonder if Node.js will ever get wider adoption like Java got.
I'm not sure if it it will ever go as wide but it does seem to be going that way.
The dramatically larger ecosystem and community is great, but notably the static typing is far more powerful in practice. I really wouldn't pick Java to improve type safety nowadays.
Huh? Node is almost dominant for all kinds of API and web backends...
[0]: https://jobs.lever.co/substackinc/69f5ed72-9a51-404d-9db1-20...