Hacker News Reader

Comment by docker_up | Hacker News Reader

I think "ease of use" over SQL is not the hill I would die over if I were trying to displace SQL.

It's far too embedded throughout the entire industry and as a data analyst, learning EdgeQL vs SQL and then being locked into a new startup database that could disappear in a year doesn't seem like a high probability strategy.

I wish the people all the luck but unfortunately SQL is "good enough", pretty standardized (I can use just about any relational database and get useful data by knowing the basics). The inconsistencies may be mathematically "ugly" but it's not hard to wrap your head around and overcome.

View on HN

A good replacement tech usually gets piggy-backed on the tech it's going to replace.

That is, a reliable tool that translates 99% of normal SQL into readable Edge SQL, and vice versa, would help adoption a lot.

Remember how new JavaScript features became mainstream though transpiling, long before native implementations.

Agreed. I think this is one reason that Looker's LookML has been successful. Not that it's entirely what I'd want out of a "SQL replacement", but it's an enhancement that "compiles" down to SQL rather than looking to replace it. Plus, you can always go direct to SQL, in case you need to take advantage of some specific feature or complexity that their language doesn't address itself.

No it won't, because you add immense friction for no added features.

Composability is quite a feature for me.

I accidentally built an in-memory database that now lives prominently in our production stack. It works great, its incredibly performant, the codebase is relatively simple (it makes heavy use of code-generation), it will scale very well - but not a day goes by I don't think what if I had just taken the time to adapt an existing solution to the problem set.

There are just so many free things you get with SQL and established RDBMS that deeply impact application features, quality, stability, operations, and much, much, more. I've had to write a custom mongo-db like interface for querying, as well as a fair number of hacky bits to effectively cover the surface area of SQL in an inferior way.

I've learned tremendously, but I just wish people don't follow in my exact footsteps because that's probably wasted time.

As someone who landed up doing what you did, I also had days where I felt what you have described here. On the other hand by going custom you can usually exploit some understanding you have of your problem to great effect.

SQLite is a wonderful tool for such a scenario.

The middlebrow dismissal strikes again!

Why is it so hard to imagine something displacing SQL? A simpler, more predictable syntax seems perfectly plausible--it could ship alongside SQL. Do we have StockHolm Syndrome?

The negativity is surprising and at the same time predictable.

Because of the decades of things not displacing it - when someone suggests "oh we'll just do it simpler" they often are not seeing the forest for the trees.

Simpler languages have been shipped dozens if not hundreds of times, and they generally tend towards expressing the things they missed or not giving enough functionality for the things they missed.

I am not saying its impossible, but you're going to have to do a lot more than hand waving to justify the reverse position.

It's sort of like the mouse trap problem. Mouse traps and SQL are already incredibly simple and incredible effective, that's why you don't see a reinvented mouse trap at home depot and why SQL remains unseated despite many efforts to replace it.

You really think SQL is incredibly simple? I think TFA makes a pretty convincing argument that it is anything but.

Certainly there are complex aspects of it but you could teach someone how to do most of what you need to do in SQL in under an hour.

Most common SQL statements read like somewhat stilted english. Many non-programmers find this particularly accessible.

Yes you can make some lovecraftian horrors if you really want to, but SQL is one of those things where just a little bit of knowledge goes a long way. If you can understand the basics you can get a lot of work done.

It's a lot like Excel. You can do some really complex confusing stuff in Excel. But you can also teach the basics to non-programmers quite easily, and command of the basic skills will be very empowering. Basic knowledge of Excel, like SQL, gives the user new ways to leverage computers when creating their own solutions to their own problems.

Most common SQL statements read like somewhat stilted english. Many non-programmers find this particularly accessible.

The problem is not reading SQL, but writing it.

It's a lot like Excel. You can do some really complex confusing stuff in Excel […] Basic knowledge of Excel, like SQL, gives the user new ways to leverage computers when creating their own solutions to their own problems.

I can’t speak to your experiences, but I’ve never in my life encountered someone who was not a professional programmer ever even contemplating using SQL for anything let alone creating their own solutions to their own problems. I think it’s safe to assume that the overwhelming majority of people who use SQL, are programmers who most certainly cannot get by with just basic knowledge.

"I’ve never in my life encountered someone who was not a professional programmer ever even contemplating using SQL for anything"

I have. That's only an anecdotal observation, but it seems to me that the existence of Visual Basic and the popularity over the years, combined with the utter disdain for it by "real programmers" is evidence that, more generally, there are a huge number of kinda, sorta, programmers who are outside the IT culture.

Librarians and secretaries are two examples of "non-programmer" careers where functional knowledge of SQL is pretty common. Less-so these days with secretaries, but moreso for librarians. Not to mention tons of researchers across countless disciplines have SQL in their toolboxes. I've even met government bureaucrats with professional backgrounds in regional banking that know SQL. Previous programming experience? Using HP-12c calculators...

>I think it’s safe to assume that the overwhelming majority of people who use SQL, are programmers who most certainly cannot get by with just basic knowledge.

Most demonstrably do though, so there's that. If you cast a wide net when polling programmers, I think you'd find that mode level of knowledge was relatively low. You don't need to be a SQL rockstar ninja dude to do what most professional programmers are doing with SQL most of the time. Obviously advanced knowledge is good for any professional programmer to have, but the fact is there are a TON of people out there who only know the basics, and that works for them.

In the business world (e-commerce in my case) I know plenty of business analysts who write SQL as their only programming experience. That’s part of what I like about SQL. It bridges the gap.

Our support guys are mostly hired from our customer base. They have great domain knowledge, but do not have any formal tech training.

After some time, most can handle enough SQL to help customers with basic issues that cannot be handled in the application. Some have become quite good at it, and can do quite non-trivial stuff. None of these folks write any code beyond SQL.

Yes you can make some lovecraftian horrors if you really want to, but SQL is one of those things where just a little bit of knowledge goes a long way.

I agree. At the same time a little bit of knowledge is incredibly dangerous.

That query, which worked so brilliantly on the test system during devlopment, suddenly grinds all of production to a halt.

The basic problem is that indexing and other physical performance boosters were not really rerquired when those queries were tested with 3'000 customers.

Being set-based that's quite different when you suddenly deal with 30'000'000 customers and a number of joins, which may not be supported by indexes, since that was never obvious in development.

That said: I'm not arguing against SQL. It's a great language for its purpose. What I do argue for is to have an SQL domain expert and an expert on how it phyisically maps to the underlying database engine for more complex projects.

Such a resource can be immensly valuable in assisting application developers to avoid major mistakes when they deal with the underlying database.

Edit: A couple of issues, which actually negated my argument upon reread

"that's why you don't see a reinvented mouse trap at home depot"

Interesting for you to say that. The last time I went looking for mouse traps, I found that every place I went had a reinvented type, and traditional style were extremely hard to find. Yet, it is true that the traditional style is simple and effective. I've learned from experience that anything else is likely to be useless.

It could happen. However, the probability of success for any given attempt is low. Consider all the attempts at improving on JavaScript before Typescript.

> The middlebrow dismissal strikes again!

Yes, it was that, but, still, there's a big and growing hurdle here that many similar efforts, with similar objective merits compared to contemporary SQL implementations, have failed to overcome, and not a lot of reason provided to think EdgeDB is better positioned.

> Why is it so hard to imagine something displacing SQL?

Because systems providing just as good solutions to largely the same set of SQL deficiencies have been produced and failed to displace SQL for a couple of decades.

The problem isn't doing better than SQL. It's doing enough better than SQL to overcome the depth of knowledge, experience, support, tooling maturity, and comfort people have with SQL. And that most gets deeper over time, on top of SQL getting internal mitigations, if not actual solutions, to some of the problems over time.

That said—as I’ve done with several before—I’ll probably download EdgeDB and try to do some stuff with it.

That's like saying nothing will replace COBOL. Or nothing will replace Fortran. Or C++. Or Java.

There's no reason the industry can't move to new technology for new projects. No one is going to rewrite legacy applications in the new language, at least not until the transition is so far along that SQL specialists are aged out and costly. But for new things? It's totally acceptable to pick new languages.

And btw, this has already happened in some areas. Most developers I know don't code in SQL: they use an ORM provided by their language runtime or a support library. That's essentially the same thing.

> And btw, this has already happened in some areas. Most developers I know don't code in SQL: they use an ORM provided by their language runtime or a support library. That's essentially the same thing.

This works well enough until your ORM shits the bed and you're forced to figure out why your SQL database is "slow".

SQL isn't slow, ORMs are just terrible when you hit an edge-case or when pretend there isn't a relational model behind your opaque materialised objects.

I used to be one of those developers you mentioned, but I'm not anymore after years of debugging "SQL performance issues" (hint: it was the ORM), and actually taking the time to learn the language and take advantage of specific RDBMS features. My preference has shifted to just use a lightweight library to materialise objects (like Dapper), and write the queries myself.

Oh I couldn’t agree more. I didn’t mean it as an endorsement of ORMs, but it does demonstrate that SQL isn’t entrenched as the sole user interface to RDBMS.

Would you adopt an extra abstraction layer (bugs, maintenance, incompatibilities and surprises included) over your database to get a coherent handling of nulls and a few other optimizations of this level?

I know I'm staying with the nulls. I wish the best luck for them, and if it survives to maturity, I'll haply go get the ~1% (probably less) more productivity they offer. But right now I'm not moving. It's sad, really, but things are stacked against them. Change is costly, so we lose all the small changes that could compose into something huge.

You even don't have to "imagine" it anymore, it already exists. See the projects list at http://www.thethirdmanifesto.com .

The crux is : it takes much much more to achieve that than what the average C# or java coder yup has to offer. So the average C# or java coder yup will dismiss those things too.

Totally agree. It’s a bedrock technology that’s near-universal as far as company needs, at least in startup and web development environments.

The JS framework churn makes me super appreciative of the stability that SQL brings.

JS framework churn isn't what it was five years ago

You're right, it's far far worse now.

Citation needed? I've been using React for 5 years and don't foresee not using it any time soon.

Even if it's still true, "churn" translates to progress. It would be hard to argue complex web application UI development isn't better off now than it was 10 years ago.

> Citation needed?

I don't have a citation, but a colleague of mine who's working on front-end projects repeatedly complains that today's JS stacks require half a dozen base JS packages, which in turn download dozens if not hundreds of ancillary packages, not to mention requiring a couple of transpilers and a bunch of tooling.

And for what? Well, just to be able to render some text and a couple of buttons.

Nowadays we have whole server projects that take less than 50MB of source code and dependencies to build, while a miserable SPA with a login screen and a couple of menus and buttons requires nearly 400MB of JS.

That's pretty bleak.

Serious question: why do you care about 400MB vs 50MB of dev dependencies? Is your internet connection slow? Are you running out of hard drive space?

There are projects like create-react-app and Parcel which offer very reasonable zero-configuration toolchains.

If you care a lot about runtime dependency weight, there are lightweight libraries like Preact (3kB).

I know. Things were so much nicer 5 years ago just before React, and the other myriad frameworks we have now, came out.

Out of curiosity, which frameworks/libraries are you referring to, and why do you think they are nicer than the "UI as a function of state" style libraries like React?

SQL is good enough for the existing set of applications, but that's not really saying much. There are lots of other applications for which SQL is not good enough, and those applications either don't exist because an affordable alternative doesn't exist or they implement their own proprietary database (e.g., many popular BI tools). It's safe to say that your use case--using SQL to perform one-off queries--is fine; writing a program that can dynamically build (performant) SQL to access data of arbitrary schema is quite a lot harder even if you can assume a single implementation. And much of this difficulty comes down to lack of composability.

Perhaps SQL is fine if it's your interface for accessing data on a one-off basis, but if you're trying to build a complex tool on top of it (say, an analysis tool for arbitrary data), the inconsistencies and performance concerns mount. People often end up inventing their own proprietary databases to do these analyses (e.g., virtually any business intelligence tool) assuming they can afford to do so. Perhaps EdgeQL isn't the ideal alternative, but as it is SQL is not good enough for many use cases.

SQL is extremely expressive, it's almost impossible to build something that cannot be expressed in an SQL query. In most cases when people feel like SQL cannot do something it is either because the Database does not implement a part of the standard or because they are not familiar with some of the more advanced usage of SQL. Simple SELECT FROM WHERE clauses, even including JOIN, are still fairly simple compared with what you CAN do if you want.

I'd recommend reading the PGSQL manual, they go very in depth about many of the supported features and how they are implement and can be used.

"SQL is extremely expressive, it's almost impossible to build something that cannot be expressed in an SQL query."

That's kind of orthogonal to what I think is the issue being expressed here.

SQL can do many things; the problems tend to be when a query doesn't perform consistently and predictably. There's always a balance to be struck between communicating what is to be done, and how it is to be done, and SQL leaves so much of the "how" out that the query interpreter/optimizer is incredibly sophisticated and does a fantastic amount of work and yet frequently gets things spectacularly wrong, maybe due to misconfiguration and maybe due to fundamental limitations.

Obviously more information on how to do something is not always better; otherwise we'd be using assembler. But there is a balance.

Experts tend to say "write everything in one query, and if it doesn't work, fix the configuration of your database" which is not helpful given the division of responsibilities in any company. But they will say that because they are devoted to the idea that all that expressiveness is good for something.

> SQL is extremely expressive, it's almost impossible to build something that cannot be expressed in an SQL query.

If that were true, people would build RDBMS’s in SQL.

Most of a RDBMS is actually build using SQL. (For example, in PG constraints and foreign keys are done using triggers and SQL Functions.)

SQL isn't a general purpose language, it's a data query language.

It fulfils its purpose and does it well.

That's only true tautologically--i.e., if you decide to constrain "its purpose" to the set of things SQL does well. If you want to do something perfectly reasonable--like programmatically building queries to access data of arbitrary (read "unknown at compile time") schema, you'll find it's quite hard to do this, at least if you care about performance at all. Largely because SQL doesn't compose well.

Optimization requires knowing the schema, and query usage patterns, and data stats - which competent RDBMS engines use to great effect already.

Sounds like you're looking for a magic silver bullet - there's no free lunch in our field though.

Lastly, the comment I was replying to can be paraphrased as "well is SQL is so great why aren't RDBMS' built using SQL, huh?". Which is a ridiculous question since SQL isn't the right tool for that job - its very name tells you that.

If you want to continue arguing against strawmen do it with someone else.

Not true.

Since the addition of SQL/PSM (1996 IIRC) it has become a computationally complete language (procedural like many of the others) with variables and loops and what have you.

Same goes for JavaScript. It's so ubiquitous that it's worth putting up with the downsides.

It'll take several unicorns and Fortune 100s hiring thousands of engineers to code in an SQL-alternative to create an ecosystem large enough to eventually overtake SQL.

This situation seems comparable to Typescript in this analogy. It has taken several years, but I feel like it is beginning to take hold.

Agreed on all of that. Further, I wouldn't embrace something intending to displace SQL without it being authored by someone like Anders Hejlsberg (Turbo Pascal, Delphi, C#, TypeScript). That sort of involvement grants confidence that it's as "correct" as it can be for most users. That matters for buy-in. It can happen, Kotlin is a decent example, but is not embraced as widely as TypeScript has been. I'm sure that confidence plays a big role there. The stakes are much higher here with SQL, than in Java or ECMAScript.

There's plenty of brilliant people out there, but when you're talking about replacing the most successful data storage language in the history of mankind, you need everything. SQL has been "killed" many times, everyone wants to sell something. It would probably take involvement from a FAAMG entity. I think the first clue that there's no real room for technical innovation and we're staring at only the opportunity for technical churn, is that no FAAMG players, who definitely operate at-scale, have tried to displace SQL outright already.

TypeScript or Kotlin are really the closest, best and most recent examples of what would need to happen. For me as an end-user, ubiquity and skill-reusability matters. If you don't like SQL, there are ORMs.

In the financial industry, I have a seen a couple places where SQL was the interface for their payment systems. Mind you that these payment systems were not written using any kind of relational paradigm.

Reply on news.ycombinator.com