They don't even know the fundamentals
blog.royalsloth.eu
blog.royalsloth.eu
Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?
I myself had a similar fuckup with MongoDB a decade ago, because I used it "because it's fast", but failed to read the small letters, and wasn't aware that it achieves that speed by buffering the data for async save, and so has no guarantees it will actually be saved at all. So I lost a lot of data when traffic suddenly jumped, and since those were affiliate clicks lots of people got very pissed about not getting paid, and it almost ruined both my client's business as well as my own, as it was our core client.
Lack of understanding of (or even caring to know about) DB internals is also why like 90% of projects I've seen has wrong indices, or often no even a single index set at all. Because, hey, it's someone else's job to think about it, but in reality budgets are limited and team don't have a dedicated DBA or even a capable devops person to notice it, and it just ends up in the production like that.
So, in summary: no you don't need to know every in-and-out of every technology out there, no one can do that, but there's a valid requirement that one has to be familiar to those that they actually use - at least those that can seriously bite one's ass and cost a lot of money. And all DB related stuff is very much like that, that is probably the most critical part of the whole system, so no, it's definitely not a black box.
Same thing here, our legacy codebase avoided using transactions, and to this day, after migrating to a new codebase, we still get reports from clients here and there about errors which stem from inconsistent data
I suspect the author of this article would have no objections to probing if a candidate understands transactions and indices.
I mean, I learned ACID's definition, nodded because it made sense, realised that MySQL's default (at the time) MyISAM storage engine wasn't ACID, it flat out failed at C, so encouraged my LAMP stack using friends to switch to InnoDB, then switched to Postgres myself.
Likewise, I use a lot of distributed systems, but I don't think in terms of the CAP theorem on the regular.
Rather I just think about all the fun ways each distributed system can die in, and while ZK is CP (although I'm dubious about the P), and Kafka is CA, the failure mode is roughly the same - you lose enough cluster members and shit breaks.
I know that my k-safety (k being how many nodes I can lose and still have a system) for ZK is N - (N +1)/2 (assuming truncating int division), and as for Kafka, well... it really depends on the failure.
But the usual conservative design for a Kafka cluster is n * 3 replicas on a 3 AZ stretch cluster with min.insync.replicas set to num.replicas - 1,leading to a k safety of 1.
I'm a fan of the 2.5 stretch cluster, if I have to do a stretch cluster, I generally prefer separate clusters replicating.
So for a 2.5, you'd go 2 AZs with M brokers and N ZK nodes, where M % 2 == 0 && N % 2 == 0, with one ZK in the third AZ as the tiebreaker.
You can now lose an entire AZ, and still have a quorum, while paying 33% less inter-AZ traffic from your Kafka clients.
Basically, theory underpins all my work, but the theory and the reality are quite different things.
When I read that sentence, I gasped and thought the same thing.
I've spent a career writing thousands of lines of application code that would have never been needed if the database had been designed properly. But it wasn't because the deployer didn't think they needed "deep knowledge" that day.
We stand on the shoulders of giants who figured out the fundamentals just for us.
The worse the violation (different data types in the same column, ugh!), the more application code (10X) will be needed.
The more code written, the more more maintenance (another 10X) needed.
And none of it ever gets replaced, just migrated to something else years later (if we're still in business).
ASIDE: As for the interview question, I would never ask what "ACID" meant. I'm more interested in understanding than recall. I would just give the candidate a poorly designed database and ask what they would do to fix it.
If you are trying to hire, expecting every junior dev to know all this stuff means you'll skip a lot of candidates who can contribute a lot to your business.
They shouldn't be trusted with writing things from scratch. But if your codebase is mature enough, getting db connections, making endpoints atomic and stuff like that already should have a convention so your junior engineer wouldn't have to do it on their own.
And then there's the time that I rewrote a small database application, but couldn't really reduce the byzantine schema, as that's just how our business works. However, I did reduce the main query from 727 lines to 93, and made it run twice as fast. And then, of course, the project was canceled because of politics, and the group still struggles with their terrible application, 3 years later.
I should really write a book about terrible management in Fortune 250's.
I think there is a big difference between not knowing something off the top of your head and being completely unaware of it.
If you have awareness of something, you can make decisions based on that awareness without necessarily having a deep understanding. When you have no awareness is when the big issues arise.
For example, with your MongoDB fuckup if you had even a vague idea about the underlying behaviour you might have been able to consider the failure scenario ahead of time and prevent it from occurring. A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
Same for indexes. If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up". You don't necessarily have to understand how a B-tree works for an index to be effective.
The next level is being aware that the “I” doesn't quite apply in most databases with their default transaction settings, true isolation/serialisability being sacrificed for performance, though the distinction rarely matters is can be significant.
> You don't necessarily have to understand how a B-tree works for an index to be effective.
How a b-tree operates can be safely kept as a black-box thing, the important level of detail that many miss in my experience is how compound indexes can (and more importantly, can't) be used. Others include that in most databases defining a foreign key relationship does not create a supporting index by default, and that careless use of functions in joining & filtering clauses often blocks index use. If you have a DB specialist on the team there is probably no need for other devs to be intimately aware of even these details, but they should at least be aware enough of their existence than they know to ask the local expert (or look things up elsewhere) to check their working when the issues might affect what they are working on.
As a bit of a local DB specialist who has met some bad ones, I should point out that it is important that we, and other local “exports”, be approachable so the other devs feel they can run things by us for verification/improvement.
People only need to know about ACID if they are arguing about whether to use a database that doesn't implement it, or switching between databases some of which do and some of which don't provide strong ACID guarantees.
You may have been much better off, both performance and correctness, to relax MySQL's durability from flushing on every write, to flushing/committing every 1 second (using innodb_flush_log_at_trx_commit). Doing so improves performance by multiple orders of magnitude (it can be the order of 10-100x) to the point that your server is no longer sitting there queuing up queries for writes to complete. The database loses some guarantees about durability but the system as a whole gains better durability.
I could make similar arguments about the other properties as well, the point is that a competent person should actually know this stuff, understand it, not just memorize an acronym and boil the entire issue down to whether a database has ACID or not. It's not a binary matter, it's more of a continuum where each property can be tuned to various degrees. MySQL and Postgres both offer many options to tune the various ACID guarantees from having complete conformance to relaxing them with various upper bounds. Relaxing guarantees about a local property of the system can improve the guarantees of system as a whole.
On my current work database is riddled with indices over various status columns. Those indices are not selective and basically useless. Worse of all: it's not immediately obvious that those indices are bad, because inserts/updates are fast enough, but they slow system down as they accumulate and there's no easy way back.
In my experience the most reliable and long living projects treat the DB as its own separate product, whose interface to data mutation is via stored procs. The db can then be sanely designed with good data integrity rules (foreign keys, unique indexes and data column constraints).
Then external callers use the stored procs as their API, and then mostly don't have to worry about the internals. You can easily create a web based client, REST calls or a native GUI app that all work directly off the same db.
However they do have to worry about : -connection timeouts -query timeouts -deadlocks
But that's the same for all distributed systems.
I think in many cases, it's that even just having a vague awareness of something prompts you to double-check more often or to do something about it when you aren't really sure.
As an example, I find myself reading about the pros and cons of various PostgreSQL index types over and over again. It isn't really all that important for me to retain every detail about them for immediate recall but knowing that there's enough of a difference motivates me to read the documentation or run a quick web search before shooting myself in the foot.
I tend to think that testing people on this kind of deep knowledge isn't really helpful nor is it indicative of how developers really operate. What you need to test them on is what steps they would take when they aren't really sure and how they learn from it.
But… I’d like to say this is the kind of problem these quick-boot coding schools fall short on. Sure one can learn programming language X, but that doesn’t mean one has any sort of fundamentals in algorithms, data structures or general database technology.
This is going to require being conversant with the terms in that ACID acronym.
> A deep understanding of how MongoDB works probably wasn't required to prevent this incident.
That deep understanding is exactly what was required. If you don't understand how Mongo works, you don't understand the ways it can fail to do what you think it will do.
> If you're aware of them, you can trigger the thought process of "this query is pretty slow, maybe I should look into adding an index because I know that speeds things up".
So why not add an index for everything ahead of time, then every query will be fast, right?
Or maybe you need to have a sense of the cost of adding an index, then. Which may require knowing the different kinds of indexes and what the speed/cost tradeoffs are.
I think in all these cases, not every developer has to understand these things, but if you don't have anyone in your organization that understands them and is thinking about them, things can blow up for you at unexpected times that make your customers very unhappy. And then you might need to hire someone who understands them to fix the mess.
I don't remember what each letter of acronym means either. But if I was interviewee in this situation, I'd just start talking about transactions from user perspective -- how they either completeley pass or completely fail, and how even with power failure, you are not going to see partial data.
If I were interviewer, I'd accept this answer. No one cares about hearing exact words (0), "user will never see only a part of transaction" is good enough even if person never said the word "atomicity"
(0) According to some sources, Google runs scripted interviews where interviewer is non-technical, and grades answers against a checklist.
• the transaction is the minimal (atomic) unit, and must never be split up;
• all transactions (and everything else too) must map valid states to valid states;
• parallel database transactions should be viewed as a directed asynchronous graph over valid states (like Git)
• store stuff on the hard drive rather than RAM (i.e. the hard part of making databases; filesystems are horrible!)
I have fairly deep understanding of these things; I just didn't know the cool acronym mnemonic. (Database engine devs don't tend to use it in their blog posts about the internals of SQLite or Postgres.) “If they didn't learn how I learnt, they don't know anything” is one of my pet peeves.
Understanding the way your storage works is crucial, yes. But you ensure people understand it not by asking "Explain ACID to me" but by asking to design a certain system and observing them figuring out requirements and asking questions. You then ask follow up question about consistency and reliability, you probably even can ask about benefits of using transactions but I wouldn't because there's plenty of databases that can be used reliably that don't have transactions.
Life is more complex than ACID, that's it. Investigate if person understands life, not that they were able to memorise bunch of magic acronyms.
For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
Off the top of my head, "all child classes should be usable interchangeably when dealing with an interface/base class". I guess that's too imprecise to qualify as "written explanation"?
Yet the GP says exactly the same thing as you said. He just ignored that the property belongs to the base class.
Given how some people study for these interviews, you might be better off going deep either way, to avoid getting rehearsed answers on these trivia questions.
Who has time for that? Interviewers design interview for themselves, not for the interviewees. It's basically a trivia game at this point of time. A prime example of "Everything is obvious once you know the answer" syndrome.
Often you're time-limited and can't do actual code testing, and acronyms and buzzwords can be useful to do the initial filtering quickly. Of course, not expecting for candidates to reproduce what each letter means, but to show they've heard of it and have some idea what it's about.
> For the life of me I don't remember what SOLID is and I totally fail to parse any written explanation of liskov substitution principle but whenever I see code that violates those I immediately go "please don't do this, that'll cause problems in the future".
In my book this is a perfectly valid answer to that question, actually far better than just citing the book definition blindly. Interviews are all about trying to read between the lines, and often the whole impression of competence that candidate leaves to you is far more important than the actual answers.
There's a difference between knowing the internal implementation, and the pros/cons of different usage patterns. What you're talking about is having mastery over an interface. This is distinct from knowing the internal implementation.
As someone who used to work as a hardware engineer, I can assure you that there's a tremendous amount of hardware internal-implementation-details that you're ignorant of. But that doesn't get in the way of you being you a great developer, as long as you spend some time understanding the interface and the recommended ways to use the hardware black-box. The same applies to what the OP is describing.
My statement is that it was a terrible example, but the core principle stands. Yes, we should all definitely understand how the storage layer works (and anything else that could corrupt data) but no one needs to have the laundry list of acronyms memorized.
I think the article also touches on the fact that when you look up the definition of ACID you get a rough description of the acronym and then a description of how it is mostly a marketing term and all the ways real systems can deviate from it. So really the deep knowledge is not so much what ACID means as what it doesn’t.
The article is basically complaining that knowing what ACID stands for is not really a fundamental and that knowing how to use a database is. So I don’t think you really disagree with the OP much either.
If you’re implementing interactions with a fairly new database (let’s face it. 10 years ago MongoDB was basically experimental) then it is indeed incumbent on you to understand in greater details what you’re doing.
But if you’re building a CRUD web app, that’s maybe only used internally by your company and backed by SQL Server you probably have to never give it a thought.
Either way, lack of knowledge of a particular acronym, one which is described as basically meaningless in texts that describe it, is not a deal breaker of any sort. And it doesn’t apply just to ACID but most acronyms in tech today, such as REST/SOLID, etc like the author points out.
Until you get complaints that the hour-long reconciliation transaction you run in the morning is locking out all the users, or there's an API a customer hits that's erroring out 1% of the time because Sybcoughuh, Wonderful Microsoft SQL Server has detected a deadlock, and this is causing us to lose customer orders, or...
No, you really need to know what ACID means, and what tradeoffs are made to balance ACID with other considerations in your chosen database implementation. Lumping ACID in with vague heuristics like SOLID on the basis that they're both acronyms is just plain ignorance.
Sure, you're right, but I'd argue that transactions are is hardly "database internals". Transactions are day 2 of "using a database 101".
You don't have to know what ACID means or how transactions are implemented to use them correctly.
I'd rather hire a candidate that knows how to use transactions but has no idea what ACID is than one who knows what ACID means and how it's implemented but doesn't know when or why they should use transactions in real life. (Of course the second case is rare, but this is just an example).
> you don't know what you don't know
aka knowledge is what you have after you forgot everything you learned.
> But, that’s only because you are a noob and you don’t deal with the things that I have deal with.
> Someone on the internet
In conclusion: not all software problems require the same implementation.
I don’t run a DB for some deterministic applications I have that can regenerate the same outputs such I don’t need to store it all. I could, in theory, regenerate an entire data set with a seed. Maybe app developers are ignorant and lazy for not designing systems that do that!
What does your one story have to do with “software engineering” as a field?
Sounds like people are conflating problem solving and personal career demands.
Understanding why we should use database transactions is not the same, to my mind, as understanding database internals. When the author said they can treat the database as a "black box", my thinking was the they didn't necessarily need knowledge of the database's implementation. People can understand how to effectively work with a database in the general sense (i.e. use transactions) without a deep knowledge of every implementation.
When we start talking about MongoDB, I think that's really interesting because it is so different from our traditional, SQL-first databases like PostgreSQL, Oracle, etc. Our understanding of SQL-first databases isn't all that useful when we try to apply it to MongoDB.
At that point I think we are in agreement: it's unreasonable to think we can treat a MongoDB database as a black box if our base knowledge is only SQL and SQL-first databases. We probably do need someone who has already dug in and dealt with the implementation, how MongoDB really gets things done.
They invested incredible amount of time in learning technology and scaling their application but forgot about need to learn fundamentals -- data structures and efficiency.
Fast forward 1,5 years, the application as I left it ran on a single server using about 10% of its capacity. A second server is just a hot standby backup.
The service went literally from being able to process tens of transactions to a hundred thousand transactions per second on a single node.
More than that, we threw away most of the exotic technology that was used there -- greatly improving team productivity. The implementation is simpler than ever with layers upon layers of microservices replaced with regular method calls and a lot of infrastructure basically removed without a need to replace it with anything.
People who do not learn from history (fundamentals) are doomed to repeat the same mistakes.
I still remember some interviews within the last decade where single-core machines with a few GB of RAM were a default assumption for whiteboard designs, or that spinning disks were the default.
But in general I think that the problem isn't that people are not "updating" their understanding. Even 10 years ago it wasn't a huge problem getting hundreds of thousands of transactions per second on a single modest machine.
The problem rather is people relying on more and more layers of abstractions for vary small gains.
Example: I get that Python is a nice language (for somebody that does not know Lisp). But is it worth it to choose Python for a little bit improvement in productivity for a problem that requires a lot of throughput, to then suffer performance issues, to then spend many times more effort on trying to improve performance? I don't think so.
Or more in my space: is Spring Data (Java) worth the very incremental productivity improvements if it completely destroys your application performance? The application I described in my parent post used Spring Data MongoDB which kinda means it was fetching entities one by one which is extremely costly.
By replacing it with bare MongoDB reactive driver and ensuring data is being streamed in large batches (why read one user data if you can read 10k at a time) and getting rid of costly aggregations in favour of application side processing we have improved throughput by many orders of magnitude WHILE reducing load on MongoDB.
Granted, there is a little bit of additional complexity (on the order of 10% more of application code) but just the performance improvements mean that the team can breathe and focus on other problems like modelling the domain correctly.
I would consider transactions part of the api. I use them all the time in postgres and I'd hardly call myself an expert in the low level details
1. The interviewers aren't subject matter experts themselves but are hiring managers, so they've taken their questions from someone else.
2. The company gets far more good candidates apply than they can hire, so the process is less about removing bad candidates and more about having some arbitrary way of whittling down good candidates.
3. The interviewer is trying to prove how clever they are or the company is. Sometimes for their own ego but more often because they think it might make the company more attractive to work at.
If there is a theoretical question I'd normally Google / DDG instead of memorise then I'd usually be honest in the interview and say "it's something like xyz but in truth this is something I double check these days as, pragmatically speaking, it is not a problem that comes up frequently. However it is something I have done before [cite example]".
If the questions is asked because of reason 1 or 3 from above, then usually they're happy with that answer. If they ask it because of 2, well as much as I might disagree with their interview process, I'm still not the candidate they're looking for. So we still part of good terms.
There were no other questions - no white board stuff, no algorithm questions, no stupid questions like "where do you see yourself in 5 years?" etc. I still remember the interview nearly a decade later.
Obviously this won't work for all jobs, but this will work surprisingly well for a large number of jobs, because most jobs are ordinary. One doesn't need to know complex algorithms to do these
I might, too, but I frankly might do the same to an even more extreme degree for a particularly challenging real-time 1-on-1 interview question from someone staring at me.
All things considered, I think I'd prefer being left alone in a room where I don't feel them over my shoulder while constantly wondering what they might be thinking with each passing second. I can take a few minutes to relax a little, look over the code, and try to figure it out.
I have anxiety issues when an interviewer asks me some obscure syntax option for grep command or some b-tree stuff that I learned in college decade ago (and never used in real life).
What can you do to understand why this happens?
But I don't understand because I am not you. Different things give different people anxiety. Anxiety isn't something you reason your way into or out of.
They tested it after I left (a >100x improvement apparently) and called me the next day to ask when I could start.
4. Interviewer did not have time set aside to prepare for the interview, and sort of has to wing it. I was surprised to realize how often this happens.
I agree that you don't need to know anything about von Neumann Architecture and the likes in order to be a Web Dev.
But I wouldn't want to hire someone who doesn't have basic knowledge about data structures and runtime complexity. It's just too easy to have something work fine in development, but choke in production due to accidentally quadratic behaviour.
At some points either teams converged to people without basic knowledge of data structures or they rushed with something that works (when N in O(N) was small) then scaled up. They get used to lower performance slowly like boiled frog.
Then someone is tasked with optimization, but instead of attacking data structure at hand, usually those people tried to optimize pieces of the puzzle for smaller gain, then they ended up with small hacks to shave some execution time, with unmaintainable code which lead to serious bugs.
While academic algorithm astronauts write papers which include Greek math symbols and can be proven correct, most people do... something else.
Optimising the something else would be a huge benefit to the industry. But no one even looks at it. It's just a thing that happens, and there's no research into fixing it or making it better.
edit: the features I refer to are stuff like foreign keys, cascades, dynamic fields, complex joins, etc. Once your stuff is spread out you loose some of these by definition and if you're still sticking to a single DB with massive stores you're not getting much humpf out of it anymore.
I 100% agree there are way too many fools that think their DB is big and try non rdbms too early and waste too much time. But I still think their lack of recent popularity is due to their own limitations.
But 99% of projects do not come even close to the scale where you need that.
There are use cases that aren’t best suited to an RDBMS at any scale, like managing a collection of denormalized documents, or a graph. But in my experience, considering these factors isn’t what’s motivating adoption of Mongo, or Dynamo. Your key lookups will always be fast (until you get a hot shard), but when you cram a highly relational set into a document db, eventually you’re forced to realize that there’s a lot of mutations and access patterns that you simply can’t implement, because you chose the wrong tool for the job.
Either that or you’ll decide that it’s a good idea to implement your own concurrency control mechanism on top of your document database. Which I’ve personally seen at least 3 people try and do.
- interviewer asks candidate to determine the complexity of some algorithms
- candidate is nervous, has a minor brain-fart and flubs a couple of them
- interviewer evaluates candidate as lacking basic CS knowledge you learn in 1st year of any CS course
Ok in this case we're talking about big-O, but it could really be anything.
By my experience, some companies need a competent body in the seat ASAP, and some are looking for world-class engineers.
Think Sears vs Amazon-- which company would know what gold was if they fell into a pile of it?
Also I don't think a company deserves to fail if they've got a bit too much faith in a slightly flawed recruitment process. The entire point of us sharing posts like this on here and discussing them is to learn and even help each other, isn't it?
I think O notation represents the upper bound, so a complexity of N can also be said as O(N^2) or O(N^3), but that isn't particularly useful.
In other words, if f(x)/g(x) < inf when x->inf, then f(x) = O(g(x)).
Then there's omega-notation (lower bound, f(x)/g(x) > 0) and theta-notation (both lower and upper bound, 0 < f(x)/g(x) < inf).
But if I give a code sample that has three nested for loops over some array and doing string comparisons, the candidate should at least have a vague idea that this might be problematic and could better be solved using sets or maps.
The ACID example was a bad one, granted, but everyone seems to be giving up after identifying that. I also think that this post is not about interviewing. I am much more concerned with the line the author takes about knowing how a digital computer work is even necessary. How have we arrived at this point?
As an example, how seriously would you take an aircraft flight controls software engineer who willfully chooses to never learn how an airplane flies? What if they just claimed that their sole responsibility was to take flight requirements from “people who know how airplanes work” and implement code? Knowing how horrific the results of this attitude is in even low pressure business situations, would you get on the plane he designed the software for?
People who live by and spread the idea that one only need know the things he or she is ‘required to know’ are dangerous for engineering organizations. When something goes horribly wrong, your mission critical system crashes, when you get dragged before the higher ups to explain the situation and fix it, the excuse ‘I don’t have to know how this works to do my job’ simply won’t fly.
People have lost all respect for the pursuit of excellence. Ask yourself who you would like to have on your team: someone who stops at learning the bare minimum to get paid and inevitably ruins the integrity of a product or someone who continually pursues getting better at their job and accelerates the people around them? Would you rather work with someone who values knowledge or someone who doesn’t?
I think it is drastically foolish to defend the toleration of those who aren’t even willing to learn basics like how a computer works in the name of inclusivity, utilitarianism, or egalitarianism.
Furthermore, I know a slew of extremely bright, tenacious, and committed young people who are ruthlessly pursuing excellence in engineering. If given the chance and enough time they will easily outperform, outwork, and surpass anyone who adheres to these lazy practices. If you do think this way and don’t want these kids to take your job then I suggest you buckle down a bit and learn some basics.
At that point you just get another job and in the interview talk about how the company had bad practices that caused X bad thing.
You are so right about the pursuit of excellence being important but the industry is still way too fragmented. If a Doctor said they don't really need to know stuff, they would work it out as they went along, they wouldn't work for long. In Software Engineering, we are still treated (and act!) like some basic trade where the risks are low enough not to care.
This describes exactly the Boeing and MCAS fiasco. In fact, not only did Boeing hire do this, they outsourced the work overseas. People overseas are capable of doing great work, but they do the work exactly the the spec and nothing more. If you don't have an internal engineering process to account for situations unique to software, there's no way you can ensure that what they deliver is fit for purpose.
> Furthermore, I know a slew of extremely bright, tenacious, and committed young people who are ruthlessly pursuing excellence in engineering.
Those types of kids are going extinct. The kids coming up now are entirely smart-phone raised. An actual computer is an afterthought, they can't even explain what a file is. We're headed for the software engineering dark ages.
> As an example, how seriously would you take an aircraft flight controls software engineer who willfully chooses to never learn how an airplane flies? What if they just claimed that their sole responsibility was to take flight requirements from “people who know how airplanes work” and implement code?
is highly disingenuous given that learning to dig into what needs to be built is explicitly laid out in the article as the opposite of memorizing acronyms about how one might build it, and:
> I think it is drastically foolish to defend the toleration of those who aren’t even willing to learn basics like how a computer works in the name of inclusivity, utilitarianism, or egalitarianism.
seems to me to be far out of left field given that the article in question isn't speaking in the name of any of those things, but instead comparing to the value of other kinds of knowledge and expertise. If you feel the need to invoke "inclusivity" in this conversation, that only speaks to your own mindset.
I would much rather work with people who are able to analyze what kinds of knowledge actually drive their getting better results at their job than with people who assume going lower on the stack must equal "excellence in engineering".
I don't expect a web designer to know how NAND gates or how to program in C because it's irrelevant. If they spent time learning that on the job, that's likely a waste of time. Abstractions exist for a reason.
I agree completely though that lazy people are out there. Professionals should always be learning and trying to improve their craft. Learning the bare minimum shouldn't be the default.
Can we just collectively swallow our privilege and stop whining that an employer is asking for proof that you've actually put the effort to be an engineer before they dump a river of money onto you? Look at how lucky you're before whining that the world is unfair, and shut the hell up about fundamentals.
The interviews surely aren't perfect, but the constant whining is ridiculous. Why not just study it and be done with it already?
But that's the rub, knowing the acronym to ACID isn't proof of anything other than you read about it recently. Asking about the ACID test is like asking what Chipper Jones' batting average was in 1994 when interviewing to be a baseball player in 2021.
20+ years ago, I could recite all that shit from memory, but after building countless systems from the ground up which are still in use today, I can only remember reading about it but I can't recite the words of the acronym.
Hiring in IT is fundamentally broken and has been for decades and probably will be for decades. This is an example of it, that is the discussion.
Not to be disrespectful to anyone, but this "issue" is the personification of that gif with a man wiping his tears with 100 dollar bills. The tech community has become increasingly disconnected from the real world in the past 10 years.
It's not about fairness, it's about the ability for a company to hire what they need, and the person who best fits that need to be hired. That isn't happening now because the hiring process is broken. I think the main issue is people are afraid to fire employees who don't work out, although there are plenty of mechanisms (legal and policy) to make this frictionless. There is also a tsunami of unqualified people trying to get hired because it's one of the last few occupations that pays fairly well.
>As soon as you get hired you're set for life
That isn't anywhere close to true.
>The tech community has become increasingly disconnected from the real world in the past 10 years.
Sounds like you have an axe to grind. What are you basing this off of? I assume you don't work in tech.
It isn't about putting in the effort. It's that even after you've provably put in the effort, they still don't believe that you've put in the effort, and so they constantly test you to make sure that you've put in the effort.
Yes and no. For years I did little computer programming projects on my own. Built some neat things, for my purposes at least. But because I didn't have the words to describe things I wasn't able to build a concrete image in my head. When I finally took a CS course and had the words to describe concepts I'd used for so long it was like a switch had been flipped, allowing me to think about problems in a much better way. I knew what many things were but because I didn't have the concepts mapped to concrete terms they remained fuzzy.
> So I had learned already that names don't constitute knowledge... That's caused me a certain trouble since because I refuse to learn the name of anything... What [my father] forgot to tell me was that the knowing the names of things is useful if you wanna talk to somebody else.
I think an interview might count :)
Names also come handy when you need to search for something and search engines are not powerful at taking your description of what you're searching for.
This change in me was triggered when a candidate came in for a dev interview and we were doing the usual whiteboard problem thing back then. Someone on the panel asked "what would you do if you faced this problem on the job" and he said "I'd google for the solution". I recall not being able to make up my mind about whether to hire him right there or send him away :) .. but that was something I hadn't realized back then so the issue was not with him but with me. The lesson stayed.
So .. on the topic, if the candidate didn't know ACID .. an opportunity to google followed by a detailed discussion on it would work for me.
Also, things may be fundamental, but not everything is built up from fundamentals. Maybe math is the best place to look at this. When you go to school, you just do it. Number lines, equations, calculus, matrices. Even through the engineering course I did, which had all sorts of advanced topics, nobody mentioned ZFC or Peano or Cantor or anything like that. And while it's true that if you dig under the stuff that you get taught there's a whole interesting world, it doesn't seem to stop anyone from using math.
With coding, a lot of it is having the right attitude. Something's broken, I'm going to try to understand why and what other people have done to fix similar issues. There's a resourcefulness that beats classroom teaching in terms of usefulness. You want that person who is capable of understanding everything that might happen, what he needs to do to fix it, etc. That person is never going to know or have heard of everything to start with, but they can deal with things when they happen.
But of course it's easier to test whether someone has heard of this or that thing. Perhaps this is why senior devs find it easier to get a job than junior ones, employers are more likely to think that they can work through issues.
Nobody talked about implementing a specific variant of a red-black tree when the general topic of self-balancing binary search tree hasn't even be mentioned.
So depending on the actual exercise/interview question you might able to just show that, especially of you have to use pseudo code.
The self-balancing part is usually added after the search part as balance is trivial/unimportant when the order/position of the elements has no constraints.
I imagine there's quite a lot of them. Things like VP-trees and such get often built statically and could be implemented in a similar way. Not sure if the simple (2i, 2i+1) schema is optimal for this but perhaps it could be adapted.
That said, for most dev jobs implementing a binary tree isn't necessary knowledge. But if I need a dev who can get their hands dirty I would be weary of hiring someone who can't take a decent stab at it during an interview.
If you audition for the New York City ballet, they don’t expect you to ice skate.
Why? The only time I knowingly used them is with database indices. For everything else I do, they are an unnecessary implementation detail. It doesn't matter what Ruby's Set uses.
If you really can't, you should get familiar with how data can point to other data, it is fundamental and done all the time.
If the current node is a leaf, insert left if smaller and right if bigger.
If not a leaf nods, run the method recursively on the left it it's smaller, on the right if it's bigger.
If they asked you to write a self-balancing binary tree it's one thing, but there's nothing tricky about writing a normal ordered binary tree.
[data, *next] [data, *next] [data, *next] [data, *next]
It's easy to make an imbalanced binary tree.
I once decided that building a binary tree based only on a friend's explanation of how they worked; it seemed like it would be a good learning exercise for myself (I don't have a CS degree). It was like 10 minutes of coding to get it minimally working & tested, but then I had to work out all of the different rotations to make it a balanced binary tree (I don't know that an imbalanced binary tree is ever actually useful, as it could just work out to be a slower linked-list in the worst case scenario).
I did it once, took me a little while, but I don't know that I could do it again without having to reexamine all of the different cases that result in imbalance and the rotations that respond to each.
For example I think writing some logic to filter datapoints or timespans is much better and more practical. But it takes more effort than just opening and pocking binary tree.
I wouldn't want to have an interview with you, or you selecting people for a team I'm on.
People in general do not communicate on the cleanest ambiguosity-free way;
I’ve interviewed a _lot_ of people. The people who gripe the loudest about algorithms haven’t had a chance just from answers to situational questions.
Coding binary tree is not the way to discover that.
Presenting a task and observing people solving it and explaining computation complexity behind their solution and the thinking behind their solution is how you do that.
It's a simple data structures / algorithms question. Can you think in terms of how data is stored in memory? Do you understand the tradeoffs of storing data in one form or another? Can you work with recursive algorithms? Given a challenge of having to think about data in a form you haven't recently, can you wrap your mind around the characteristics of that data structure?
If you're getting asked all this for a ReactJS UI building job, then the interviewer is probably off-base. But otherwise, make sure you're applying for jobs you're suited for. If you don't want a job where coming up with efficient algorithms is important, that's fine - there's plenty of others out there.
If you're using a modern OS, you don't know this. The kernel remaps everything and pretends to give you a linear chunk of memory. You don't know what's in the various caches and what's been paged out of ram completely.
Everything is built on layers of abstraction. C is an abstraction layer that fits poorly into what modern CPUs and operating systems actually do, but everything is programmed to cater to C's assumptions so people don't need to rewrite their programs.
Higher languages don't necessarily use the data structure you think they do and they can switch the implementation details based on the size.
"Simple data structures" are only simple because you pick one specific layer of abstraction to work with.
If I ask if you can do something basic binary tree things, I will also not know if you are brilliant, but I can at least rule out if you are totally clueless or not.
Or you have a candidate who understands binary trees, can implement it but can't regurgitate binary tree operations from the top of their memory.
If they can't recall how balancing is done that would be fine with me, as long as they mention it.
I know dozens of good candidates who can't get there pretty quickly but are really good. I also know dozens of good candidates who can get there pretty quickly but won't be good enough on the job.
You not hiring for backend positions doesn't mean they are not good at the positions. It just means they aren't good at your signals.
Some of my coworkers are not nearly as strong as me when it comes to "CS details" like data structures or programming languages, but they have other attributes which complement me and which are good for the team.
Having a bunch of me's (or any of the others) would be a disaster.
find = lambda k, t: t if t is None or k == t.k else find(k, t.left) if k < t.k else find(k, t.right)
I implore you get to a state where this isn't hard for you, it will make you capable of more.
The author's casual arrogance about "pixel pushers" only caring about "hotdog stand colors" is just as damaging to the profession -- as it infects the hardest part of getting things right most of the time: teamwork.
I care far, far more about data-structures, eventual-consistency, data-design, and types than CSS and divs.
These days getting the design to look nice is my two-hour reward for nine days of data-stitching hell, writing backend-for-frontend lambda functions, and negotiating API contracts that actually support the product I've been asked to build.
We need to stop assuming that just because a programmer works on client code, that they are in any way less capable that a programmer that works on server code. My point stands: It's absolutely toxic to collaboration, which is the key to success.
My exact response after reading the first quote. It is indeed fundamental, and something I am mindful of with every database query I execute. But I haven’t thought about the exact definition and its mnemonic in over a decade.
Not being able to give any explanation or sign that you've at least heard of it is a completely different story from not being able to recall what the letter C stands for (consistency IIRC?).
As the saying goes: If you think you understand something, and you can't explain it to someone else, that you probably don't really understand it.
I sort of agree, but anyone typing out code at 120WPM isn't programming, they're writing. Programming is an intellectual activity, not a physical one so learning to touch type is worthwhile but might only gain you a few minutes a day of productivity over someone doing 40WPM in a home-grown hunt-and-peck style of typing?
Everything else being equal, typing speed will help you try out stuff faster.
Of course your senior engineer that wrote the thing will be more productive than your intern that writes 1000 WPM.
I am pretty sure that people who didn't learn to type properly have to switch their whole mind frame from the content into the typing, back and forth.
Some people even speak outloud, or at least move their lips according to, whatever they are typing.
Language might make a difference there. Writing C# with a QWERTY keyboard, you still have to interrupt a normal typing flow for your parentheses, braces, etc.
I suppose it's helpful if you're writing long comments, but for the most part, I rarely hit sustained bursts of my peak typing speed when writing code.
To be controversial, a language that rewards typing speed seems to me like a failure of tooling, why am I manipulating text like a caveman (ok.. ok.. Mesopotamian) rather than with the symbols that the language actually deals with.
It helps a lot with most programming languages. Snippets for if, while, and for can also help with the {}.
Imagine that you want to correct a block of code or a whole paragraph. If you don't touch type, maybe you will think twice before rewriting it.
Besides, only Java can make any programming task I/O bound ;p
I might think for a couple minutes and then come up with ten lines of code, and once they're in my head I want them in my editor now before I lose track of them.
i developed a pretty fast way of typing naturally, using 7 or 8 fingers.
unfortunately it caused wrist strain.
i could not use split ergonomic keyboards because some keys where "mapped" to the wrong hand for that.
i also could not simply learn "proper" touch typing on a qwert layout because my typing habit was too strong.
in the end i learned touch typing on dvorak to fix my wrist issue. maybe after 10 years i'll have unlearned qwert typing enough to go back? (not sure i would, i do prefer dvorak, it just feels silly to type at snail speed on others ppls keyboards).
Regarding speed - it didnt prevent to be TOP1 at university in typing contest :)
My naturally acquired typing style involves me holding my hands hands in just about the same angle as those "ergonomic" keyboards, but just using a regular keyboard. Fingers rest on LShift, A, W, D, and K, O, P, Ø with both thumbs on Space.
Should be noted that's on a "105-key" Nordic keyboard with US layout when not writing Norwegian, so the Ø becomes '.
For me this has worked quite fine for over 20 years. I haven't bothered measuring my typing speed because it's never the limiting factor.
So I simply asked her to teach me the proper technique and now, since decades already, I'm touch-typing the proper way.
Having said that: you can be very fast using improper touch-typing technique.
However, it might be a good shortcut for people learning. I was taught at a youngish age to attempt to touch type (box over the keyboard and a tutor program) I fucking hated it, and I was slow, it was dull and painful.
However, Then I had a need for typing, so I got faster and faster, and the initial structure that I got from the rigid teaching, allowed me to learn faster than my peers.
People can practice a skill for a long time, but if they don't focus on improving, they tend to just harden their existing habits.
Any activity on a computer is aided by reducing the friction involved in getting what is in your brain into the computer and from the computer back into your brain.
Sure, you can program with punch cards and no display. People did it for years. But having three monitors and a quality keyboard that you’ve mastered will be much more efficient.
Need to use the arrow keys? Outside the touch typing range. Mouse? Outside touch typing. Function keys? Numpad? Touchbar (blech)?
I have never felt limited by typing speed. Informal touch typing is good enough. Quite often it's the computer or network that can't keep up, anyway.
But I have encountered one IT professional (not a programmer) who despite being in the industry for 20+ years, could only manage about 15WPM, if that. His style seemed to include legitimate hunting for the next key, and only two fingers.
At that speed it was starting to become problematic: It was painfully slow to watch him do anything or help him with a problem.
Not an absolute deal breaker, as long as such a person is good at their job and is efficient with what they type (taking advantage of copy/pasting or scripting), it's workable.
If I can touch-type at 120WPM, then I can churn out a great deal more documentation per unit time than another engineer that can only type at 40WPM. It's less burdensome for me to document well, to write expositional documents and to communicate regularly than my colleague that needs to budget their typing more carefully.
While I might not be bottlenecked when working on a tricky piece of code at 40WPM, I'll likely be bottlenecked elsewhere often.
The keyboard is a tool.
To me, it feels disrespectful for a person to expect these high wages when they don't even make the moderate effort involved in learning to use this tool efficiently.
In my experience, this usually signals that a person isn't actually interested in continuously improving, feels overly entitled and they tend to be a nightmare to work with.
Most places I've worked require substantial pair programming or at least working through problems together on a regular basis.
If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
I'm not proposing we ridicule anyone, but being unwilling to type and claiming to be competent at a job that requires an enormous amount of typing does have a ridiculous look (to me).
Of course, I have worked with people working through a variety of disabilities, including dyslexia that make touch typing basically impossible. I'm not talking about them here.
I'm talking about people who just refuse to make the effort for whatever reason. It's a bad look.
PSA for newer players: Regardless of how you may feel about this subject, failing to demonstrate at least moderate typing skill in an interview is often a hard pass unless there's some explanation requiring an ADA accomodation.
>A carpenter who claims to be experienced, but can't work a hammer efficiently would be ridiculed on a job site.
It's just not remotely the same. How quickly and accurately a carpenter can hammer has a large impact on the quality and speed of the work, while typing should never be the bottleneck for a developer
Yes, but one of many.
> If you can't type, you're going to waste a lot of people's time and they may be too polite to say anything, but it's frustrating for everyone around you.
I don't think in my career I have ever been alerted to anyone's typing speed being slow.
Yes, I've heard endless noise about "the perfect keyboard" and the ins and outs of spring types and junk. But I've never noticed the output of someone because they type slowly.
And to be honest, I really couldn't care. Can they code? can they keep to a deadline? are they nice to work with?
if the answer to all three is yes, then I couldn't give a shit about how they type.
> but can't work a hammer efficiently would be ridiculed on a job site.
Only if they need to use a hammer effectively though. If they are laying veneer, carving, using a screwdriver like a sensible person, then a hammer is only rarely needed.
This all smacks of gatekeeping to me.
I do, when I'm pairing with someone, or in any synchronous meeting environment. It may not effect the 'output' of delivered code (for a developer) but it can hinder their ability to pair effectively.
I agree that some (maybe lots of?) people feel differently about this and of course, that's okay too.
To be clear, I'm not saying anyone needs to type 120 WPM, I'm talking about a single finger on each hand, eyes down, hunting for the next key to press.
Over the 20+ years of my career, I've been exposed to hundreds, maybe thousands of different folks doing different kinds of computer work.
And yes, I agree that there are likely people who can kind of get the job done without being able to operate a keyboard efficiently. I also agree, that it helps a lot if they're nice to work with.
But I have not yet met a single hacker who has demonstrable and deep knowledge of computers and the software that runs them who cannot type.
I'm sure they must exist, and it wouldn't surprise me if Donald Knuth himself dropped into this thread to let me know that he hunts and pecks!
I'm just saying, that I have not personally met or observed someone operating at the highest levels of this field who cannot type with a reasonable speed.
[update] Almost forgot about Gatekeeping, Yes!
Gatekeeping is an essential part of interviewing. In fact, that's the entire point. We interview people for roles with the specific intent of bringing on people who we believe (or at least hope) have higher competence and closing the gate to people who have lower competence.
It's incredibly difficult work and it's not perfect and we have a lot of work to do, to make it better and more inclusive, but expecting competence in basic, role-relevant skills is unambiguously reasonable IMO.
I most enjoy working with people who care deeply about their work, and in my experience, those people are often (not always), people who also care about being efficient. In fact, if you're writing code at 120 wpm, maybe you've found a technique worth sharing and teaching! :-)
People with mild to moderate dyslexia (in my experience) and people with injuries are generally able to demonstrate competence with the keyboard and computer operation.
I have dealt with moderate to severe RSI, and I have worked with a relatively large number of people dealing with RSI over the years and in my experience injuries don't mask operational competence. These behaviors look different from, "where was that key again?"
Dyslexia often surfaces in a variety of early communications and over the years, working with some especially brilliant people challenged with dyslexia, I've learned to assume (early on) that typos are not a signal for carelessness, but potential indicators of dyslexia.
I'm aware dyslexia is a spectrum, and at the extreme it can be fully debilitating for some activities. I haven't had much exposure to that cohort, but I would be happy to make reasonable accommodations and certainly would not consider it disqualifying.
Thanks for bringing this up, I think it's helpful to be able to clarify these details!
In the modern workplace there is often quite a bit of communication via a text chat system. A chat where everyone types at 120 WPM is about as fast as the low end of normal conversation speed (120-150 WPM). A chat with several people only doing 40 WPM is much less pleasant.
Email doesn't have the quick back and forth of chat, so slow typing usually won't be directly noticeable to your colleagues, but it can take up a lot of your time.
Someone else already mentioned documentation.
I think the best analogue is blindness: it dramatically reduces the bandwidth of the information channel from the computer to you, but there are highly productive blind programmers. Hunt-and-peck programmers are in a similar situation.
Dojo’s approach seem to be one step beyond, but it requires even longer commitment to the employee and an asymmetrical market opportunity and economic chance. Definitely will help people and become a win win.
Web work is exploitative because... the workflow is undocumented? If it pays well, who cares?
The issue is that training/skilling juniors or fresh university graduates costs you money and slows you down (as you need to assign senior engineers/mentors to assist them). It is not unlikely that a newly joined junior employees only costs you money and decreases throughput/value for your company in the first 6-12 months, but you still pay them a (decent) salary. So once you got them skilled, you as a company would probably need to raise the salary to market/senior level to keep them - given the mass amount of recruiter approaches we in IT get. In that case, you could just start off hiring seniors only in the first place.
This is an issue I see everywhere, nobody wants to hire freshly graduated and everybody is only shopping around for seniors.
Seems like paying at least market rates once they are trained up would solve that problem. Once someone develops a skill, a company can't expect to keep them at intern or junior salary levels.
> Let's turn to databases now. Can you explain ACID to me?
I'm not saying that the original article on RoyalSloth was wrong, exactly. It has to do with the way the interviewer approaches the question. For instance, for that full-stack developer I would be PERFECTLY satisfied with an answer that went like this:
> Hmm. Well, I don't remember exactly what the letters stand for. Atomic, something something. But look -- some databases, like MongoDB, can sometimes lose data after it has been "committed". Other databases, like DynamoDB, can read just fine from one table but they can't guarantee some basic invariants when reading from multiple tables at once. And some others, PostgreSQL and most "relational databases" fit in this category, jump through hoops (I don't know quite how it works) to make certain that what you see is always consistent.
If I were hiring someone JUST to do front-end web design then this wouldn't be relevant. If it's someone more junior then the above sentiment but without the ability to name specific examples would be fine. But I don't want to hire someone to work on interfacing with the database unless they understand it to at least this level.
If I were hiring for a database specialist I would expect a much more detailed explanation.
I've been railing about this for years. Please stop calling it a REST endpoint. In 95% of the cases it is a custom HTTP endpoint and that's it.
I try to just call these things an "http API" when talking about it myself, just to mean an endpoint that is intended for consumption by other applications, not by humans, but I'm not gonna fight other people using language the way it works in practice, where words just mean whatever enough people agree they mean, regardless of what the very first user of the word meant.
Even more so when those "fundamentals" are implementation details you don't think about much in your day to day job. Actively recalling knowledge that has become part of your intuition and understanding of the world is hard.
I know enough about many things to do basic implementations safely; everything else is paged out to google. The finer details of database transations? Usually not my job, but I used to know more about them and could relearn that information quickly if needed.
These days, that’s equivalent to saying “don’t ask questions about what the day to day job requires one to know”.
Sites like stackoverflow contain answers to problems that people encounter in their day to day jobs, and if you eliminate those as potential questions, then what should you ask? And what is the value in asking some obscure question, other than making the interviewer feel smart and special?
"I'm emitting 10TB of data a day with this schema {X} - it's produced in Avro. Currently we upload it into S3. But, instead, I want to stream into Apache Druid, but also want to make it available for ad-hoc analysis by BAs.
What approach would you take?"
Can't exactly google an answer to that.
The keywords there were "streaming" and "also ad-hoc analysis".
Answering those questions requires no knowledge of Avro or Druid, but rather, how to build streaming pipelines, and also make the data available for a query engine.
But that said, if you're in data engineering and are unaware of what Avro is, you're probably not at the level of knowledge I'd expect, and I'm happy to filter you out.
Likewise, if you're unfamiliar with S3, why are you even applying for a data engineering role?
I believe that "something else" usually indicates some previous data engineering work though, so it's an egg and chicken problem.
... you don't need to be in data engineering to use S3.
And can you be familiar with it? Read the docs, write some code.
BTW from my experience we are streaming a lot of data (similar to 10TB but definitely more) produced in JSON then converted to Avro using Spark and finally loaded into Vertica and the analysis pipeline. I have never used Druid but from Googling it is a column based data store which I guess is not too different from Vertica. Is there any specific difficulty you think would be encountered with Druid? Because the wording seems to indicate that Druid is not particularly good with ad-hoc analysis.
Bingo ;)
I was trying to phrase it like I've heard people do so in interviews.
And tbh, a candidate saying "I'm not familiar with Druid, but it sounds like you can't use it do ad-hoc queries with it? Can you tell me more?" would be a very good sign.
This is kind of thinking is why people hire all these leetcode jockeys who can't write working software, and won't stay beyond a few months anyway.
But for God's sake please Google before asking any general question. Some questions are literally within the first 5 results of the first page.
Maybe I shouldn't blame them though, sometimes I did that too :/
If you don’t know that, for example, debiting one account and crediting another needs to be done inside a transaction - or the converse, what you need to do if you don’t have access to transactions - then you shouldn’t be anywhere near a database.
Fundamentals are actually important. You don’t have to know everything, but you should know the fundamentals of the field you work in.
Such as "I" - what is it again? "Isolation"? "Transactions are independent". Well--yes.
...I think that was the entire point of their post.
> I guess we would be much happier if we dared to admit that one man’s fundamentals are another man’s trivia
And I would guess that many developers working with relational databases don't actually understand the more subtle differences between transaction isolation levels. I think asking backend developers to know the basics of transactions is reasonable. But e.g. knowing the exact difference between READ COMMITTED, REPEATABLE READ and SERIALIZABLE (and what your specific database actually implements there) might not be necessary in many cases. Though it certainly is good to be able to spot when you're about to enter territory where these differences might matter.
I agree that a backend developer working with databases should know what transaction are. But I don't really need the definition of ACID, and you can really push this question into details that are not relevant for all developers.
The point of jargon is to allow concise communication between experts. Knowing the jargon is part of the job.
On the other hand, if you are actually able to design for all those complex points you may still not have one short definition to quote available, which would not speak against you at all. Not having a short definition for ACID to quote may happen to both real experts as well as people who really don't know anything. Because for real experts that definition is too simplified to be of any use.
I saw similar issues a lot in school and in academia, when having to answer some test questions: When you actually know a lot more you may actually get many answers wrong, because the one asking used a much simplified model (and therefore more inaccurate than yours) as basis for creating their question. For example in biology, there are some key tenets, but when you go deeper you learn about all the additional quirks and exceptions. Using the simplified tenet an answer to a question "can this or that happen" would be never, but for someone who knows a lot more that answer would be wrong. As an expert, you no longer use that simplified model you learned early in school.
Example blog post showing the world is more complex than the simple definition: http://www.bailis.org/blog/when-is-acid-acid-rarely/
That was the point of the article, as shown by this example quote:
> How many similarly vague “fundamentals” we have floating around us? Plenty: RESTful, MVC, SOLID are the first things that come to my mind. If you start a discussion about the meaning behind those terms, you will find yourself in the middle of a food fight where everyone will have a slightly different opinion and no consensus will ever be reached.
All I have top do is read HN whenever such a topic comes up. It usually ends up with a lot of back and forth of people who are all experts but have a slightly different angle, and the initially clear definition falls apart when you read the discussions.
So asking about ACID is a fine thing, but expecting a text book answer is silly.
Edit: ivanhoe said it better in their post and I changed my phrasing to be less inflammatory. (My point of different hiring mind sets on different continents is not the point.)
If you think you have a better way, tell us about it without passing judgement on others. Tell us why it works, how great it is, give us some qualitative and quantitative data. Most of all, enjoy your competitive advantage. But for god's sake, quit telling other people they are doing it wrong.
School exams are supposed to be fair, in the sense that they only cover the topics covered in class, and only to the level of detail and difficulty covered in the class. They test for weaknesses relative to that standard. When a school exam asks a question, it's because you're supposed to know the answer.
If someone takes the school exam definition of fairness into job interviews, they are going to feel terrified and aggrieved. Interviewers talk to candidates from vastly diverse backgrounds. Other candidates didn't go to your school, maybe didn't go to school in the same country as you, maybe didn't go to college at all. There is no fair, agreed-upon set of standard knowledge. If interviewers restrict themselves to "standard" topics, they will undervalue a lot of people who have unique backgrounds and experience. Not only that, but job interviews can't just measure a candidate's weaknesses against a set standard. It isn't even really a matter of "strengths" and "weaknesses" but wanting to discover a candidate's full capabilities. You don't want to miss a candidate's depth of knowledge in an area simply because you failed to ask any harder questions that would have revealed it. When an interviewer asks a surprising question, it's often not because you're supposed to know but because you might know and they don't want to undervalue you because they didn't ask.
Drop P and you'll end up with both consistency and availability, right? Right?
("This is a trade-off we are ready to accept")
This "almost always" works.
Put quite many people say "you cannot have all three of Conistency, Available, Partition Tolerance", which sounds like you can keep any two / drop one at will.
But the right starting point is: "Partitions will happen. When they do, do you drop A or C?"
You could probably talk about the important properties of interactions with a datastore well enough to please a reasonable interviewer. Why assume they aren't reasonable? I really hate the trope of, "XYZ came up in an interview, and therefore I assume the interviewer is making hiring decisions according to a such-and-such rigid criteria I imagined when I went home afterwards."
(The conversation with "Joe" of course is imaginary, because you wouldn't already be started with interviews if you hadn't talked about how you were going to evaluate candidates. If this conversation happened in real life, the blog post would be about planning for interviews and not going rogue with your interview agenda when you're part of a team process.)
Interviews are designed to figure out where a candidate is with their knowledge, all the way from "they don't know anything about databases" to "they can write a little bit of SQL with help" to "they have 10 years of experience with the database we use for our OLTP, and our expert had a great conversation with them about the technical details of a quirk that bit us in production last week." Depending on the position, these might all be acceptable levels of knowledge for a position, not to mention that companies are often hiring for multiple positions, not all of which have been publicized yet, and are often open to redefining a role for the right candidate.
Every candidate has their own idiosyncratic mixture of strengths and weaknesses. Candidates have a hard time believing it, but a hard or unexpected question is usually probing for strength, not weakness. If interviewers only asked questions that a minimally qualified candidate can answer, they would miss your strengths and undervalue you as a potential hire.
I was in awe... and for a while after that, I thought I understood the basics of electronics.
Last year I learned how tubes work, especially when slightly gassy, and thus ignitrons, etc... turns out I didn't know how all electronics worked.
There is always another layer to learn about. The ARP address layer of the network stack, for instance.
The Unicode layer they added between my days as a Turbo Pascal programmer in MS-DOS, and the present. (Which apparently can be used to hide trojan horses)
Always, always, always another layer.
The best we can do is to try to understand all the layers that definitely affect us, and a layer or two more as insurance (or just because it's cool to know)
Asking about fundamentals in an interview is, surely, a given? I don't necessarily need everyone to know everything about everything but if I met a dev who didn't know some software principles or a DBA who couldn't explain ACID to at least a basic level, why would I want to employ them? Because they tell me that they just work it out when necessary?
I've cleaned up too much code from people who think "just use an array" or searching through a massive list because they don't know how dictionaries work.
The interview questions sound more than reasonable and then the OP tries to destroy this by saying that not everyone knows all the details of everything which is not what most people would call questions on fundamentals.
It doesn't matter if you're a frontender or writing a shell script to automate something. If you don't understand atomicity, race condition, locking, etc., not only are you going to shoot yourself in the foot and tear your hair out when shit breaks randomly, but you're going to subject the poor souls who come after you, to unnecessary suffering.
One doesn't get good at these fundamentals by getting a Comp. Sci. degree. All of it can be learnt in a few days. What's important is to keep evaluating your code and design all the time against it to ensure you're not building on crooked foundation.
The I (Isolation) part is the one you'll need to understand whenever you scale up to a point where concurrency starts to become non trivial in your product, mainly because there are multiple levels of isolation, which each provides different guaranties, and you'll need to understand what level your database uses and what it means for your application.
As for the overall point of the article, it's just a question of scale. The more you scale the more you need to cut through the abstraction to understand what the hell is going on. Hire accordingly.
For example, to me a fundamental for C++ development is knowing the difference between an std::vector and an std::list. Can you really be an effective C++ programmer without knowing that?
Yes the field is vast and lists and vectors can mean different things in different contexts, but you're applying to a C++ development job so I expect you to know some basics about C++ specifically...it doesn't seem that unreasonable to me.
Yeah, maybe this is all fields, which is a shame.
But interviewing in general is tough (I'd go so far as to say it's a shitshow) and there's lies on both ends -- companies and managers and folks on the inside just want the best of the best -- and candidates just want a chance to get in the door -- the incentive to embellish is too high.
Some "fundamentals" sit at the base of a knowledge pyramid. If an OS developer doesn't know pointers, there's no easy fix for that. You want someone with years of experience building things on top of pointers.
But other "fundamentals" are more like things that are used very commonly and thus one expects that experts would be familiar with them, but they sit on top of knowledge pyramids. It's not hard for experts to pick them up.
I think this lens can help clarify a lot of these discussions!
We have two caches and we really just needed to use the dB in a not criminally bad way.
But the market didn't care and the people who did the abuse are now director or architects or manager level engineers at other (good) companies.
There's some reality that knowing this stuff matters but it honestly doesn't for a huge number of tech startups, at least in this market.
Furthermore, many real-world databases do not fully provide ACID guarantees, and yet you can reason about the database anyway.
Well, in the absence of evidence in the contrary, they probably did.
This is a really important point that we don’t talk about enough, because everyone gets uncomfortable and feels judged. But I can’t help myself, and I think there’s a lot more to dissect here.
I think programming is slowly bifurcating. We’re growing into two disciplines.
One discipline requires deep systems knowledge. A CS degree is recommended and if you can’t reverse a binary tree on a white board then you won’t be able to read our code base. Developers in this camp work on rendering engines, Vulcan drivers and schedulers. They spend lunch telling stories about that one time we wrote a game engine in J. Or arguing about type systems.
The second group makes most of the software (measured by lines of code). And almost all the software humans actually interact with. They spend their time working around weird safari quirks, arguing about electron, and enjoying the sense of ease and mastery that comes from staying in a single ecosystem for a decade. Getting better at their job looks like getting better at talking to users and customers, making on point estimates and making sure the team releases on time.
The part that gets all of us in trouble is that the first skill set is (arguably) harder - and thus much more prestigious. Everyone wants to be seen as an electrical engineer, not an electrician. So people write “software engineer” on their resumes. And then cry foul when they get assessed as a member of that first group. “I’ve never needed to know about red black trees in my career. It’s stupid for anyone to ask about it in an interview. Everyone just crams leetcode. JQuery isn’t flashy but it works and I can solve real problems with it. Hire me you fools, and I’ll show you!”. They’re sort of right. We’re just collectively confused about what sort of job they’re being hired for.
I think this confusion hurts everyone. I once wore my fancy engineering hat at a normal consulting gig. I coded up a complex solution to a tricky problem they had and called it a day. After I left, nobody on the team could understand or maintain some of the code I wrote. That caused huge problems for my client after I left. I think they might have regretted bringing me on board at all.
But we can’t even talk about this because we don’t have the language. We treat application developers like they’re just junior, in training versions of deep CS systems engineers. But practicing building kick-arse iOS apps doesn’t make you magically good at data structures. And vice versa - doing performance work doesn’t make you good at working with clients. (Ask me how I know). The majority of professional programmers do plumbing / user facing software. We need to destigmatise this stuff so people can be hired explicitly for these roles, based on the right set of skills.
Uh, what? Most humans find social skills much easier to learn than crunchy engineering skills. Good social skills are way more common in society than, say, deep knowledge of algorithms and data structures.
But it’s not a competition. As you say, programming is rarely a solo activity. Our projects can fail due to a lack of teamwork. Or a lack of technical skill. Or a lack of empathy with our users. Or a lack of leadership. Or a dozen other things.
We need (and need to value, depending on the project) all of this stuff.
These are disparate skillsets and both of them are difficult to master. Claiming one is harder than the other is really quite shallow. Some people are going to naturally have an easier time picking up one skillset while other will have a naturally easier time picking up the other skillset. I think rather than trying to pretend like people should stop focusing on hard skills and start focusing on soft skills, we could realise for a moment that it does nobody any good to take people who are good at one task and insist that they must retrain for another when there are already plenty of people who are good at the other task.
It would make more sense to get these groups of people to learn just enough of the opposite person's skillset so that they can effectively communicate and as a result form a well functioning two-sided system where one group of people talks to users and analyses their requirements and the other group of people actually works off of these more concrete better defined requirements to actually implement the software.
If you think that "soft skills" are harder than "hard skills" it's probably because they're harder for you, that doesn't mean they're harder for everyone, and instead of pretending like this is some kind of major flaw in your own professional development, you could treat it as a gap which could be filled by people who have the opposite view of difficulty.
It has always been like this. The distinction used to be: base software developer vs applications developer.
Base software developers went to work on operating systems, databases, web servers, etc.
Application developers worked on business software.
And most decent business software development shops kept a couple of devs with decent knowledge of base development just to go for when the needs got technical.
Hell yeah, I have hardly ever seen a correct REST API in the wild. People mostly consider any non-SOAP HTTP API RESTful.
I still think any programmer using any database should know what “data consistency” you sacrificed for using “Read commited” instead of “serialisable” and how hacker can use it to steal million of dollar of your e-commerce company
You can build bridges that don't fall down without knowing the difference between stress and strain; you can build an audio amplifier without knowing the difference between voltage and current; and you can build a database-backed application without knowing what ACID means. But you shouldn't, except maybe as a learning exercise, because it's an enormous amount of effort to produce mediocre results. Understanding ACID allows you to treat a database as "a black box with an API" with enormously better results.
I have no patience for the kind of glorification of ignorance in this post. Perhaps it's true that "the majority of programmers" think "something being RESTful means that you can feed it chunks of JSON over HTTP." The majority of audiophiles might think that oxygen-free copper will improve their sound quality, the majority of computer users might think that Google built the Internet, and the majority of people don't speak English. That has no bearing on the knowledge needed to actually build a working hi-fi system, debug an internet connection problem, or successfully converse in English, and neither does popular ignorance about REST (or ACID) have any bearing on the knowledge needed to successfully design a distributed system that works reliably in the face of constant partial failures.
(Ivanhoe's post at https://news.ycombinator.com/item?id=29065438 is an excellent explanation of what can happen when you lack the understanding of these fundamentals but mistakenly think you can use the database as a black box anyway: "I lost a lot of data when traffic suddenly jumped, and since those were affiliate clicks lots of people got very pissed about not getting paid, and it almost ruined both my client's business as well as my own, as it was our core client.")
If over 40 years you spend 83000 hours working, and you spend 10% of them on programming, that's 8300 hours. (If your job title is "software engineer" you probably spend more like 50% of them on programming, but maybe you're a web designer, a database administrator, a sysadmin, a mechanical engineer, or a biologist, so programming is a relatively minor part of your work.) The optimal fraction of that time spent on getting to know the fundamentals --- real fundamentals like ACID, not fake fundamentals like SOLID, though this presumes that you have someone to guide you who knows the difference --- is surely not 100%, because then you'll never get any work done. But it's not 0% either; you won't get much done that way either. It's probably in the range of 20% to 60%, which would be 1700-5000 hours. Learning in your bones what ACID is might take you 5 hours spread over a month or two, so it only takes up 0.3%-0.1% of that time.