Hacker News Reader

You shouldn’t need a DBA for a small or medium project. Any developer who hasn’t taken the time to learn basic RDMS theory like normalization, indexing, foreign keys, etc. doesn’t take thier craft seriously. Knowing the basics of how databases work is a required skill in almost any implementation.

The issue is “framework developers”. I am not opposed to learning and using frameworks but you should always know at least one level below the framework you are using.

If you are a web developer who uses Angular/React and Bootstrap everyday you should know JavaScript, CSS, and HTML well.

If you use an ORM, you should know enough about how databases work to have a mental model of what the database is doing.

> you should always know at least one level below the framework you are using

I've never heard that good advice put so succinctly before. Has this advice been around for ages and I just missed it, or is it a newly relevant concern?

BTW, what would two levels be to, say, a React developer, out of curiosity? Would it be the browser engines? Knowing how JIT compilers work, etc.? Or would it instead be a deeper understanding of the language spec, so that you know more internal functioning? Not that there has to be a clear-cut answer.

I heard it on a podcast and I am going to butcher the explanation but they explained it as knowing the “abstract machine”.

When I started learning AppleSoft Basic back in the 80s, I knew how to optimize my BASIC programs because I knew how the interpreter worked and so knew assembly.

When I started learning C#, I knew both C, Win APIs and how the CLR worked.

But for a web developer who communicates with backend servers, I think it’s important to understand HTTP.

I've heard a similar idea on Hanselminutes. The way he states it is that everything is layers of abstractions, and in order to be great at whatever your chosen layer is, you should be pretty good at the layer below it and of passable quality (think junior dev level) at the layer below that.

Actually it might have been Hanselminutes. Do you know which episode?

I've heard it a handful of times on various episodes but I know he's brought it up when speaking to folks who work primarily one layer below him.

What podcast was that? Sounds legit! I'm always on the hunt for new dev focused podcasts.

I don’t remember the episode unfortunately. It was an episode of Software Engineering Radio (http://www.se-radio.net/)

I would think one level down from React is the DOM API, and one level down from that is the rendering engine (i.e. knowing about how DOM operations affect repaints, whether repaints block the main thread, etc.

Language-wise, I'd say babel "javascript" is the highest level of abstraction. One level below is understanding what things like JSX and class properties transpile to. One level below is understanding JIT strategies (e.g. when hidden classes deopt), memory usage of various patterns, etc. One level below is understanding specific semantics of high level APIs (e.g. exact accuracy of a high res timer implementation), or regexp memory footprint by looking at browser source code.

For a react developer I'd say one level is understanding reactive design and maybe how you would implement it in pure JavaScript without react. One level below that would be how you would implement JavaScript and maybe a bit of knowledge how the engine works and shadow Dom.

If you're running into performance problems, knowing those things isn't enough. You need to know about query plans, different complexities of each kind of join algo, etc as well as how to get the query plan, how to profile your database query, etc. Basically databases are hard and you probably will need more than basic theory for anything that is more than a low-traffic CRUD app with latency SLAs on the order of hundreds of milliseconds or more.

This isn’t rocket science either. The theory behind database optimization is basically the same across every RDMS I’ve ever used (SQL server, Postgres, MySQL, and to a lesser extent Redshift). The second thing I learned about databases is how to look at a query plan and how to optimize it.

Why wouldn’t a developer’s job be to learn how to optimize a database query but would be to learn how to optimize a program?

I didn't claim it was rocket science nor that developers shouldn't learn these skills; only that optimizing a database is out of bounds of what most would consider "basic relational database theory". But beyond that, lots of developers aren't interfacing with a relational database. In a past life, I worked on embedded electronics. Other people work on operating systems. Others specialize in compilers or graphics or etc. Databases are a good skill to have, but it doesn't mean everyone should train like a web app dev either.

Currently on a project where it is taking 48 hours to run an upgrade.

No one has thought to run a query plan

(N + 1)*10000000000000000

>Any developer who hasn’t taken the time to learn basic RDMS theory like normalization, indexing, foreign keys, etc. doesn’t take thier craft seriously.

Yet I've never once been asked about this in an interview despite using it maybe once a week for years.

O(n) on the other hand...

I had to do test on these. Some companies do it.

>you should always know at least one level below the framework you are using

I run into this problem whenever I'm interviewing for devs on my team. As a java shop, we usually get people who come in and claim to be proficient in Spring, but when I start asking them questions about the underlying technologies they blank.

My favorite weed-out question when I'm worried someone is too tied to a framework (especially when its one that isn't actively in use on a project) is to ask them to give me some examples of the limitations of it. To me, being able to recognize the things your favorite framework is not able to solve effectively is the best indicator that you aren't using it as a crutch.

I can’t think of any limitations of ASP.Net MVC with all of its extensibility and I have a fairly good understanding of both the frameworks and the underlying technology (HTTP). I’m not sure I could answer that question.

Now on the other end AWS CloudFormation for infrastructure as code, I could go on for days about its limitations.

> I can’t think of any limitations of ASP.Net MVC

1) It's a web framework. So, not suitable for command line tools, desktop GUIs or persistent message queue workers. But you can do all of that in other kinds of C# .NET app.

2) However, .NET is a Garbage Collected language and should not your first choice for e.g. a real-time device driver or OS kernel.

With that line of thinking the limitations are endless: I wouldnt use asp.net MVC to build a house.

Mathematically, you're not wrong. You can draw a line around the set of things that any particular software platform is good for, and the area outside of it will be infinite.

But people do try to do some of those things that I listed, using ASP.NET.

I was thinking you meant in the context of creating a web application but yes. You should never use ASP.Net to run any long running process even if it is kicked off via a REST call. You should use out of process web workers or some other type of queuing mechanism.

Good for you for taking the lead.

As someone who has had DBA in my job title before, it saddens me to think DBA was needed to explain the importance of foreign keys or indexes (this should be general developer knowledge at this point). As a general rule, any field used in a join or where clause is an index candidate. And foreign keys (along with other constraints) are like type checking in code. They help you not do anything stupid later.

lately I discovered that many developers don't know SQL and didn't attend to any DB course at university

My DB course taught us relational algebra and calculus and you were expected to learn SQL independently.

I am constantly bemused tho’ at the extraordinary lengths some devs will go to to avoid SQL. I’ve known people who will grudgingly do SELECT * FROM table; then extract the one row/column they want on the client. Then they complain that the DB is slow when it’s loadavg is 0.01...

To be fair, SQL isn't exactly a friendly language. It's basically relational algebra and a grab bag of utilities for dealing with whatever technology was in-vogue at each iteration of the standard. Relational algebra + xml + json + etc. And even in spite of all of the extra tacked-on features, you can't push standard SQL very far before you need to depend on implementation-specific features. So yeah, everyone should learn SQL, but I can appreciate the reluctance to dive in.

I find SQL queries (select, insert, delete...) to be fairly friendly. It's kind of like semi-broken English where you fill in the blanks. My first programming language was Python, and SQL queries weren't a big deal, once I knew how to order the clauses.

Though, once you get into more advanced things like stored procedures and triggers, SQL becomes a little hostile.

It is like the developer's mind is somehow incompatible with SQL. SQL = relational algebra + xml + json? That is most certainly not going to be friendly. Xml and json are for NoSQL. Keep it that way and you will eventually see the light.

I'm not sure what your point is, but you may have missed mine. My point is that SQL has had a lot of things bolted onto its relational algebra roots. XML and JSON were two examples, but stored procedures, triggers, window functions, are others and the list is quite long (and growing!). The problem isn't so much that SQL can do a lot, but that there's no wholistic theory belying it all; it's all just bolted on. And you can't just stick to the relational algebra subset of SQL because that won't get you very far; the features that were bolted on solve real problems, but ideally we'll find a more powerful theory than relational algebra that will do a better job at wholistically solving the set of problems we're trying to make SQL solve.

There is a theory underlying it all: relational algebra, as you mentioned. The bolted on parts are just that. My point is discard those (XML and JSON in particular) and SQL is not at all unfriendly. It is in fact the friendliest way yet discovered to deal with data in large shared data banks. XML and JSON (as a model for how to store large amounts of data) are an absolute disaster by comparison. XML and JSON features exist in databases primarily to check a box on sales presentations. They aren't needed. If you are using them heavily, you are making your life more difficult than it needs to be. XML and JSON database features aren't completely useless. They are one of many tools you can use to get data out of XML and JSON and put it into normalized tables where it can be efficiently and elegantly stored, accessed, and otherwise dealt with in the future. The relational algebra subset of SQL (tables, rows, columns, primary keys, referential integrity constraints, check constraints, indexes, joins, where clauses, etc.) will get you very, very far.

My second point is that something about relational databases and SQL just doesn't sit well with (many) software developers. As a result, we have a lot of XML and JSON blobs stored in key/value databases, a lot of software that doesn't work very well, and we are still dealing with a lot of problems that Ted Codd solved 48 years ago.

I followed your point, but my point is that the relational algebra part of SQL isn’t sufficient to solve all or most of our info search/retrieval problems; you end up needing other things—analytics, window functions, polymorphism, etc. You can’t push vanilla relational algebra SQL very far before you need some extension.

Funnily, and contrary to the article, in my previous job QAs did SQL to validate the operation of the app, and usually better than the devs themselves.

Here's a nice counterpoint.. I was on a project where the chosen database was Vertica. I think everyone assumed the DBA had used columnar style databases in the past. So later we found out the DBA didn't know what a projection was. Projections are the REASON you choose a database like Vertica in an analytics project.

In the end many of the various contracting companies started to work together to work around the DBAs ineptitude and work on ways to speed up performance, including projections. We kept him in the loop, but it just felt like the only thing the DBA cared about was permissions and user accounts. In fact that's the only thing we kept calling him for was one of our accounts would suddenly not work. He was also extremely insistent against tools like Liquibase to keep all environments in sync and standardized. Instead our Dev and QA through prod would all have different tables and columns all over the place because the DBA WAS IN CHARGE OF THAT.

This was a large organization, and 99% inefficient. There IS a reason large companies are trying to run projects like startups. DBAs are often like pizza flippers, whereas developers don't like doing things multiple times.

Maybe your company hired a DBA with the wrong skillset for your environment. I don't expect every DBA to know every DB technology, for exactly the same reason I'm good with some sw development technologies and bad with others. Still, that DBA could have picked a company with a better match to his CVs.

You're absolutely spot on.

But it's not DBAs as in database admins that are needed. Every project should start with the correct data architecture and for that you need an architect. There are endless number of projects that eventually failed because the data architecture did not scale. That's due to a programmer who designs the database according to the way that their preferred tool works best, which is usually bad for data.

For any project that requires a database, always start with the method that data will be reported. If a report takes a long time to be generated because of your design, then you need to fix that first.

My experience has been very different. We scaled a project with a naive developer approach on Postgresql and we brought in a DBA when we started getting scaling issues. The few optimizations took the DBA 3 days to setup and we were on our way.

I think the reason DBAs are not hired is because we don't need them as often and for as long as we do devs.

The problem with coming in later is that data models are hard to change once a system is built around them. Sure, a DBA might spend a few days optimizing what's there, but the structure of the model itself is something you'll have to live with for quite a long time. Getting it wrong at the start means pain every time you interact with the database.

This is a real problem, in my book. DBAs also have very specialized knowledge, and tend to be expensive, but they're at their most productive when they're doing schema cleanup and index optimization work that can probably only keep a good person busy for a person-month or two per year, tops. And they're also at their most productive when they are preventing problems rather than trying to clean up stains that have already been set by a mess of runtime dependencies.

And all the rest of the time, you've got a very smart person whose job is basically just to make sure the backups are running smoothly.

I'm not sure what the solution there is. Only the largest enterprises can really keep a good DBA busy with work that isn't just skull-splittingly boring. Maybe merge the DBA role with BI or data engineering?

warehousing, projections, dev ops, cluster management, tools for disaster recovery, chaos monkey. There's _never_ enough time to do everything and I feel that claiming "there isn't enough work to do" is just not taking the subject seriously enough. For fucks sake, most of our applications are effectively just SqlServer value-adds.

I think the reason DBAs are not hired is because we don't need them as often and for as long as we do devs.

Good DBA's are also really hard to find. We have been looking for one for the past 6 months with no luck

> discovered that many developers don't know SQL and didn't attend to any DB course at university

My school's DB course taught you to build databases. You're expected to learn SQL on your own. And we all did. Not knowing SQL is an ongoing lifestyle choice.

> I remember when foreign keys were taboo among Rails developers because Rails didn't have a way to express them in its DSL.

What? Relationships (and thus foreign keys) have been around since before 1.2.

Relationships are not foreign keys. Relationships (has_many, belongs_to) are a Rails construct. Foreign keys are a database thing.

I started using Rails almost at the very beginning. See https://stackoverflow.com/questions/2373262/are-foreign-keys... from 2010 for a bunch of links to DHH opinion on the matter (from 2006.)

Foreign keys made it into the DSL with Rails 4.2

http://guides.rubyonrails.org/4_2_release_notes.html#foreign...

https://apidock.com/rails/ActiveRecord/ConnectionAdapters/Sc...

Before that we were using gems or writing SQL into the up and down migrations.

https://robots.thoughtbot.com/referential-integrity-with-for... (from December 2014)

Just to be clear, you keep saying "foreign key", but the phrase most people know is "foreign key constraint" (because it's just a constraint).