Databases shouldn't be exciting. They should just work.
The main reason webcrap works is because the databases underneath work. If the Javascript crowd had to manage data storage, nothing persistent would work reliably.
The main reason webcrap works is because the databases underneath work. If the Javascript crowd had to manage data storage, nothing persistent would work reliably.
With enough experience with the alternative, things that just work are exciting.
With 30m minutes of time, you can turn it into a document db, log/time series db, columnar, graph, key value, whatever
It’s not a replacement for a real DB but it is damn close.
The documentation seems to suggest that it is in fact pronounced "SQL-lite": https://www.sqlite.org/fullsql.html
> Don't be misled by the "Lite" in the name.
Sqlite is still awesome but PG is more awesomer. And since the cost is the same I just stick with PG now. Unless I'm resource constrained (which for my work is rare)
> If the Javascript crowd had to manage data storage,
> nothing persistent would work reliably.
There are less JavaScript engineers writing backend code, and most that do this are generally making simplistic CRUD Node.js apps. However, there are companies and people that specialise in high-quality complex Node.js apps, and they try just as hard as backend engineers from other programming languages to do this well. (I'm thinking of projects made by developers on the core Node team. I've seen good backend code there, perhaps because they're more aware of lower-level concerns.)
I personally write a lot of JavaScript/TypeScript code and I try to do a good job, but a "good job" is defined differently depending on the kind of product you're creating.
Edit: I just realised you are _the John Nagle_, so you likely have a more refined taste in certain kinds of programming than I do, however I think my point about incentives probably still stands.
A complicated thing which just works is exciting.
The GP was referring to "Databases shouldn't be exciting" - not in the traditional sense of the word, but a metaphor that hinges on "excitement" being new features, frivolous decorations, unnecessary tangents, losing vision of what a database should do, frantic patching and heated bikeshedding discussions.
It wasn't meant as "Ooooo I am excited about a database".
All of these features are exciting to the people that can use them.
It never gets old to scold people that are not serious, disciplined and principled in their endeavors that affect the rest of the world. Javascript community absolutely deserves the avalanche of criticisms they face.
It is the reason why the modern web sucks, and frankly - I wish folks at Postgres would develop a new web standards including browsers that are not governed by corporations and can be implemented in a couple of weeks. Not trivializing what goes into building browsers, but making the whole standard small enough so any one can build a browser during a sabbatical.
It’s a big enough world where you don’t have to interact with every technology that exists, but this is one of the more prevalent ones - and you might find it useful to make peace with that community.
Anyway, good luck out there.
It just seems to be the nature of the web side of things, lots of sloppiness.
The devs are smart enough, they just don't think and it shows.
That or they use something but clearly don't understand the thing they are using.
I replaced nearly 500 lines of code with ~60 by taking a bunch of massive conditonals re-ordering them then extracting them into methods and returning on the simple case.
This isn't voodoo, you have to work harder and remember more to do it the wrong way.
Also comments, they don't seem to understand you document why you did the thing, not what you did, what you did is in the code, I don't trust your comments to tell me what you did, I want to know why you did it.
That and general documentation (cross link stuff, link at the top of the class to the documentation in the wiki, update the documentation when you get a ticket that changes the behaviour, write the high level flow in plain english a 10 year old could understand).
rant over.
I actually like my job, the people are lovely and the work is interesting its just frustrating when an ounce of thought saves a pound of cure.
That is quite possibly one of the best quotes I have heard on how to approach discipline in engineering.
> Inexperience grows from the bottom
In what way?
> I've gotten "better" by simply being too stubborn to get worse
How does one get better by wanting to get worse?
This makes no sense to me.
I'm not sure I agree, but I can understand the sentiment.
You know, I feel like this has bled into backend development too. And it's not entirely the fault of engineers... or so I like to think, to excuse myself :P
What worries me a lot is that I find myself being one of those people, not being able to fully help my situation.
It's just... too much. It seems like every few months a new thing pops up and everyone suddenly wants to adopt it. And even if you try and resist it because it's not necessary, someone from above (e.g. CTO) will decide that the whole org has to adopt that and now you're screwed.
In a typical backend application you have to deal with:
- Application framework
- Testing framework
- Logging framework
- Metrics framework
- Tracing
- Visualization tool(s) for the three things above
- CI
Which is already quite a bit, but obviously manageable and you could get pretty knowledgeable on all of that if they stay static.
But hey, now your org has to expose every service on gRPC. Ok, one more thing to tack onto your app...
Now everyone has to expose GraphQL endpoints too!
Oh and next month we're migrating to a new metrics solution. Because. Great, let's throw away all the knowledge built on the old one and scramble to migrate to the new things we don't have time to learn fully.
Alright, now let's start writing new services in this new language.
And now everyone has to throw events on this new message bus thing that some team over there built using Kafka!
Finally, let's move everything off of AWS managed services and onto Kubernetes! Yay!
All of that can easily happen within the span of 2-3 years. And in the meantime you have to deal with on-call rotations, customer issues, implementing new features... and if you're a senior dev you're also busy mentoring folks, going to meetings with other teams, etc etc.
You know what I'm talking about, right? Over time, I feel like it's becoming impossible to achieve mastery of anything. There are way too many things to learn to be able to make sense of the whole, and no time to do deep dives on all of them. And then there's all the new stuff that keeps being adopted.
And then things blow up and they're freaking mysterious and when you finally find out what it was, you realize you'd only have been able to catch that problem beforehand if you'd read the freaking source code for some library you adopted.
It's madness. It's pure hell.
Boy I miss working on low level code.
End rant.
I agree with all of it, particularly "Lets throw away X and switch to Y, we lose all the domain knowledge we have with X but Y others this absolutely tiny marginal improvement, so it's worth it!"
As a lead I spend a lot of time pushing back on adding stuff until someone has made the case why we should add it.
You want to use Y, great - document why you want to add Y, what benefit it brings, who runs Y, where is the code, how many maintainers does it have, is it well tested, if want to remove Y later is it a big job, what does it offer that we can't currently do etc.
Half my battles with my team are them wanting to pull in dependencies, sometimes it's crazy like a wrapper dependency to pull in a different dependency when it's simpler to just pull in the indirect dependency and use it directly.
It's very much a thing that comes from experience though, when you've been bitten by upstream enough to be a little circumspect about it.
It's not just "the web side of things". Which makes it even more scary.
As a developer who only develops in JavaScript, I feel personally attacked by your post, so I would appreciate if your attack could be more specific to the point of being constructive or at least informative.
Thank you, this has been entertaining.
That said, there is some truth in it. The thing about the back end is that there's no satisfaction in it other than the satisfaction of doing it well and earning the esteem of peers, so most back end developers are paying attention to the quality of the code they write. The front end has other attractions as well, and many front-end developers start out focused on those attractions, and learn by experience (if they learn at all) that writing high quality code is related to the cool stuff that got them into programming in the first place.
Yes, they are capable of making great software, but it’s exactly what gp said... they don’t think. I’ve seen so much code which looks nice from the outside, abd is all structured. But if you start reading, I’m riddled wtfs and questions about why things weren’t done in a more simple concise and stable way.
Recently had to deal with some Slack bot written in js at work, and my coworkers had to deal with me ranting about that crap for a whole afternoon.
The way I put it was, this code was written in the most complicated way possible, for no apparent reason.
I meant that the developer community, the folks that are at the helm of developing Javascript frameworks (gahhhh!) and other shenanigans, the entire browser based ecosystem just reeks of incompetence, lack of discipline and rigor.
Frontend and Backend engineers should get together and build a better system that serves both well. That's what I meant by asking Postgres core devs to build front-end tools, something like Gemini [1].
Wait until this guy learns what the JS in "json" column types means, and how the largest webcrap sites rely on them for persistence.
In 2017 Bryan Cantrill gave a wonderful talk titled "Platform as a Reflection of Values: Joyent, Node.js and Beyond" https://vimeo.com/230142234 In it, he talks about how even though they liked JS as a technology, they came to realize that the values of the JS/Node.js community differed from their values at Joyent.
In the talk (at around 19:30), he identifies that in 2014 Joyent's core values wrt node.js were "debuggability", "robustness", and "stability"; while the core values of the broader node.js community were "approachability", "expressiveness", and "velocity". And there's nothing wrong with either of those sets values, but they're not the same!
With that lens, I'd characterize the parent comment as contrasting the PostgreSQL team's core value of "robustness" with the Javascript crowd's core value of "velocity". That's a distinction in values, not in technology.
It's not entirely about persistence... it's about the ease and reliability of the persistence.
The people behind javascript provided the object notation technology that has been absorbed by Postgres. It is not only being used to exchange data (as a standard text field would be used), but can be used to QUERY data. The database is aware not only of the JSON text string representation, but of the object itself.