Hacker News Reader

> SQL

Generation after generation of programmers takes one look at SQL and decides it would be more convenient to believe that relational databases don't solve any of their problems.

If there's one thing I would recommend to engineers at any point in their career it would be to, if they haven't already, learn enough SQL and enough about relational databases to feel comfortable using them for what they're good for. You start to see a whole class of problems as solved. And there are still plenty of cool problems to work on that relational databases don't solve well.

And if you aren't convinced, SQLite (and postgres) implemented MongoDB as a feature ;-)

Which should be used very sparingly, unlike the actual MongoDB, which shouldn't be used at all.

haha.

I use it for application plugin configuration values which allows plugins to do whatever they want but also provides some structure just in case sweeping changes need to be made.

Basically where schemas aren't important or will be defined by someone else.

I’ve heard (way) more than one story about MongoDB mysteriously annihilating all the data. Launched into oblivion.

The worst imaginable outcome for a database.

Oh yeah, I have two of those myself.

jeebus

It's worth mentioning that MongoDB used to have a terrible storage engine--and it would do all sorts of stuff like this under stress.

But it was replaced with a much better engine (WiredTiger) with write-ahead journaling and checkpoints around the year 2016.

In 2020, MongoDB continues to fail consistency tests.

"Jepsen Disputes MongoDB’s Data Consistency Claims" https://www.infoq.com/news/2020/05/Jepsen-MongoDB-4-2-6/

"MongoDB 4.2.6" http://jepsen.io/analyses/mongodb-4.2.6

I've worked at MongoDB for seven years right next to our EMEA/APAC support teams. We've never come across and instance where MongoDB was "mysteriously annihilating all the data". We would take any incidence of that whether it was a paying customer or not very seriously. If you know someone this has happened to I would encourage you to get them contact us.

Joe Drumgoole Director of Developer Relations, MongoDB

> Generation after generation of programmers takes one look at SQL and decides it would be more convenient to believe that relational databases don't solve any of their problems.

I think this also goes toward the point of why they all do this... that SQL isn't what most developers like or want. SQL is an complex data manipulation DSL that most of the time is complete overkill for what you need and using something simpler and more specifically oriented toward your requirements makes more sense. Personally I was pretty good with SQL 15 years back but have moved on and don't regret not having needed it as the various non-RDBMS systems have more that met all my needs.

> that SQL isn't what most developers like or want

"Most" developers? That's debatable. But I can tell you a lot of people I talked to who chose MongoDB didn't have a single good reason for choosing it. For some, it's the only DB they are familiar with ("I used it before"). Others had vague, badly-argued reasons such as "the schema might change" or "it's faster to just use MongoDB". Seasoned DBAs almost always shook their heads in disapproval (and ended up being proven right).

> For some, it's the only DB they are familiar with ("I used it before")

I'm sure some of the reasons you use your RDBMS of choice over another implementation or a NoSQL system distill to, at some point, "I used it before". Familiarity is a feature. This is especially true if your org invests in tooling that improves the usability of a DB.

> badly-argued reasons such as "the schema might change" or "it's faster to just use MongoDB"

Why are these badly-argued reasons? I currently work on a codebase that is making heavy use of MongoDB. We are using it as a way to store protobufs which will change very frequently in structure (adding new fields, removing old fields, etc). Our software is built so that it supports older versions of protos which is already needed for backwards-compatibility in APIs. Because we're using Mongo I've written a simple connector that maps our protos into Mongo that allows documents in a collection to map well into proto access semantics (default to emptiness). If a frontend developer wants to save some new field into a database they:

    1. Add the field into the proto
    2. Validation field in service (optional)

We don't have any ops work needed. We don't have any roll forward or roll back planning needed. Since we're a really small team this flexibility afforded to us is a huge velocity boon. Since we use gRPC we push people towards not making backwards incomparable API which makes our mongo-mapper very basic.

We are also planning on making use of things like `watch()` which is possible to do in some hacky ways in postgres but it's very easy to do with this architecture.

I've never had as smooth an experience with PG/MySQL/SQLite, even with ORMs.

> Seasoned DBAs almost always shook their heads in disapproval

I'm assuming that seasoned typists shook their head when business people started to buy computers to typeset their own documents. I'm sure they were also proven right in the early days as documents were poorly formatted or took long to produce. But, it wasn't that way for long.

Any time you have a field of people that is hyper optimized on maintaining a single workflow/tool/thing it will take a lot of effort to replace. There being a large amount of effort to replace a tool does not mean it shouldn't be replaced.

I'm not 100% on "most"... but really most devs I've met prefer the NoSQL options and the devs who did really like RDBMSs seemed to eventually drift into DBA positions.

I'm not sure why you focused on Mongo... it is probably the poster child for a bad example of NoSQL. It focused on features and performance while leaving data integrity on the floor. Never understood it's success given its Jepsen reviews.

NoSQL excels when it is targeted. Like using Etcd for config management. Consul for discovery. Redis for shared, in-memory data structures. Cassandra for high throughput, write heavy systems. DynamoDB for no-ops, key-value storage. S3 for key-value storage with large storage reqs. Etc.... They don't do as well, like Mongo, when they try to be general purpose. RDBMSs rule the jack-of-all-trade-master-of-none zone.

You have a point. I focused on MongoDB because it's indeed used for general purpose, where it doesn't fare very well and often ends up requiring hand-coded solutions to problems RDBMS solve out of the box.

Agree 100%. I wrote a whole post about that. https://letterstoanewdeveloper.com/2019/11/11/learn-sql/

In a more general sense, you have to be able to peel back abstractions. Those can include data storage abstractions, network abstractions (no, a network call isn't free, even in the same AWS AZ), and deployment abstractions like Docker.

I totally agree that the most unpopular, foundational stuff is essential.

I do think there are some timeless bits of technical knowledge, that will serve you no matter what high-level tech (the ones you mention, and other) is fashionable.

In roughly descending order of importance, I'd put forward: the shell (bash, zsh, ...), gnu coreutils, modal text editors (ed, vi, vim, emacs), fundamentals of linux sysadmin, git, the networking stack up to HTTP (starting at least from IP).

these are all good examples of tools that have aged very well, provide a disproportionate amount of power relative to their learning curves, and have a very good shot at being timeless.

They all also have the dual-benefit of not traditionally being taught in schools. I guess it's assumed you can pick it up along the way, which is true, but these are all complicated enough by now that they merit detailed study from time to time. In any case, because most people in the industry come from universities, they don't natively have these skills, and they can be an edge for you.

When I interview people I focus on those things you mentioned and so don’t bother trying to quiz them on anything domain specific to their past experience or our future expectations for them.

Chorus: "You grill them on irrelevant details?!?"

I will upvote this, but it can unfortunately be seen as discriminatory towards the poor folk who have built their careers maintaining windows servers. :)

although, like oil and water, it's probably unlikely they'd apply for a position at a *nix-based shop.

A friend and I were joking today that we could probably create a career out of helping people/companies fix their git issues. I've been to several employers now where most of the developers have an anxiety attack when they get merge conflicts. Being good at this is really great, though I'm not sure how to capture that on a resume, because basically everyone lists git these days.

heh I've noticed this, too. There are a group of us at work that are known as "git surgeons", maybe that's the modifying noun you're looking for!

In general, claiming status as a <technology> <noun> will get you positive attention, as long as it doesn't come off too arrogant (e.g. expert, master, w/e)

e.g. git guru, nix nerd.

It reminds me of this excellent article [0] that I like to share with all the engineers I work with, every year or so. In that case, you'd be a "taco bell programmer".

But it's maybe too self-effacing depending on your opinion of taco bell :)

[0]: http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra...

> because basically everyone lists git these days

True! And apparently you cannot skip listing it. I recently changed jobs and was told one red flag in my CV was that I didn't list git as a tool I was familiar with. This wasn't a recruiter on a buzzword hunt, but the actual technical screener. I had explicitly removed all filler because I thought interviewers wanted to focus on the important bits... (got the job anyway, thankfully).

Yeah, I was going to say the same thing. They are all soft skills. I'll add a few hard skills for STEM jobs that are not necessarily specific to any field:

- Number sense. The ability to notice inconsistencies from hundreds of rows and columns of a spreadsheet, for instance, is critical to both projects and career. Being able to carry out back-of-envelop calculation is a crucial skill too.

- Statistical sense. I'd even venture to say it's a survival skill. Too many people make statements that this: I saw a pattern of cloud every time when an earthquake happens, therefore, if I see the pattern, I can predict that there is going to be an earthquake. Or how many people are fooled by Simpson's paradox in their daily jobs?

- Mathematical maturity. How many engineers missed the gravy train of the machine learning because they couldn't or were afraid to understand basic college maths, like linear algebra, vector calculus, or mathematical statistics?

- Mathematical modeling. We don't really need to find out the boundary condition of a system of PDEs, but the ability to model a system mathematically is of huge value.

- Intuition of your field. For backend engineers, for instance, the understanding of the basic theory of distributed systems, the understanding of performance modeling with queuing theory, the understanding of common data structures and algorithms in OS/databases, the understanding of concurrent and parallel programming, and the ability to code up anything reasonably prescribed, of course.

Came here to say this (mostly).

You mentioned a bunch of computer science stuff. There's a lot in pure software development/engineering that's rather permanent, as well. Was working on a project 7-8 months ago where we had a node codebase and a C++ one, and needed to share behavior. Rather than write it twice, I figured there had to be a way to share code directly, because I knew many node extensions have bits written in C. And of course there was -- but it required knowing about things like ABIs, reactors and event loops, linking and compiler internals, etc.

There's all kinds of timeless hard skills in software dev. Knowing how to modularize things properly, how to create the right abstractions, how to name things well, how threading really works.

Within the field of software development (as contrasted with computer science), there are also skills which persist beyond fads:

• unit testing

* source control

* documentation

* FOSS contribution mechanisms and licensing

* security concepts like defense in depth, threat modeling

Even if you learn such skills in conjunction with a specific technology, they won't expire. For instance, once you get accustomed to writing unit tests under one framework, you can easily switch to another — the hard parts are understanding how to get a good ROI on the time you spend writing tests, how to write robust tests, how to write testable code, and so on.

To add to your list:

* debugging

* profiling

* logging

Wait, MongoDB is still hot? I thought it was debunked years ago and was essentially a laughingstock.

MongoDB (NASDAQ: MDB) is >$10b market cap today.

Anecdotally, I've worked at multiple places that (For Reasons™) launched on Mongo, built a relational schema on top, and still exist today.

Your anecdote rings true according to my experience. Did people in many of them end up complaining a lot about Mongo too, but "hey, it is what it is, and it's what we use now"?

A consequence of this is later, junior devs have their experience shaped by this kind of places, and end up choosing Mongo for their next piece of software, perpetuating the cycle of... life?

That's the HackerNews echo chamber of received opinions in action.

It's still hot and used in successful businesses, regardless of its technical merits and pitfalls. It's pretty trendy, too.

HN is a strange echo chamber. The same place where more than one person told me "Java is not good for general purpose development". Oh, well.

That's why whiteboard interviews are sticking around; Someone who knows the fundamentals can constantly grok new frameworks or tech. Even if a new hire has never used Git, if he has ever studied graphs it will be obvious to him. The opposite isn't always true.

White boarding ability is really trending down lately.

Being able to perform a skill in an abstract matter verbally is the opposite of what the job entails usually so it makes a poor proxy.

Agree with you that fundamental skills are key. I don't think we have figured out how to measure them.

> Meanwhile, what I learned in university is all the unpopular stuff...

The stuff you listed is specifically what interest me. Most languages are quite comparable anyways, just different syntax but not fundamentally different.

I've thought about this too as a student and in relation to professional skills.

I call the concept "Engines vs Powerups", due to having played a number of racing games as a child where you can spend prize money from races on improvements to your car or other items between levels.

It's a spectrum, too! Computer languages fall in the middle. Knowing PHP is a lot more durable than knowing Drupal, but PHP probably won't stay relevant as long as Spanish will.

One thing I'd share, is that I've gotten a lot of mileage out of skills that were once hot and are now useless. In fact, part of how I got my first well-paid software job at Groupon was because I was comfortable with Backbone.js and CoffeeScript! Neither are used much at all anymore, but I learned a lot of other things at Groupon that have helped me since and they opened the door.

https://alchemist.camp/learning-machine/engines-powerups

It's always worth knowing and working on the basics!

Also, with the "what's hot" frameworks, just because something's no longer flavour of the month doesn't mean it stops working. It's perfectly OK to build something using whatever last-month's tech you're conversant in and if it does the job, what more do you need?

Aren't bootcamps aimed at people with little to no development experience? Your first dive into a full web stack - regardless of which stack that is - will give you all sorts of skills that never expire. I consider it like a CS101 but for the web - would you say that CS101 teaches you stuff that expires?

Yes and no. My CS101, CS304P, was taught in Pascal and just involved general, basic programming. It didn't so much expire as was largely irrelevant, except as a foundation for the next round of things: algorithms, data structures, formal math, assembly, machine architecture, and so on; these were then foundations for the rest of the education. Bootcamps are like taking CS101, then saying, "We're done. You can go get a job and pick up the rest as you go along." (Which is of course possible, but only for those who are very dedicated.)

Would you mind explaining how learning proofs has helped you? I'm currently studying computer science on my own, so I would appreciate any tips.

The ability to read and write proofs lets me confirm truth independently, without needing to blindly trust someone's claims. As a quick analogy, if you know how to do arithmetic by hand, you can check whether an electronic calculator is faulty.

In my opinion, here are the most important mathematical topics in computer science which are related to proofs: Basic algebra, inequalities, functions, set theory, Boolean algebra, predicates, quantification (for-all & there-exists), weak&strong&structural induction, formal definition of big-O notation, preconditions & postconditions, loop invariants, termination, recursion, regular languages, Turing machines, computability.

Example: If you don't know the true definition of big-O, you can still identify common patterns by counting the number of nested loops. But there are non-standard cases (e.g. Euclid's GCD algorithm) and contrived pathological cases where you must appeal to the formal definition in order to prove anything.

Example: Somebody hands you a red-black tree data structure plus algorithms, in either pseudocode or real code. How can you justify that their work is correct and won't corrupt data? You need to do proof by cases, structural induction, and big-O analysis.

Example: You design a new algorithm for analyzing graphs which doesn't resemble anything in published literature. You can't just reuse other people's analyses, so you have to write your own.

Without understanding how to do proofs, you're doomed to either cobbling together high-level libraries or creating things with subtle flaws. You'd be unable to synthesize new knowledge or fill in small gaps, due to not grasping the underlying theory.

Thank you for writing this. To be blunt I don't understand about half of the terms you used, but I eventually will. Have you ever read Euclid's Elements? I recently took one of the few classes left in the country that uses Euclid, and I've been thinking about what a computer science student could glean from it in 2020.