Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.
Developer here. I can't speak to what you see as the weird trend, but I can speak about sq's history:
- I created the first version of sq circa 2013 as my own personal dev tool to address some pain points (amongst other things: getting JSON out of various DBs from the command line in a consistent manner, and wanting an easy way to inspect DB schemas, i.e. "sq inspect")
- It was starting to be a minor PITA dealing with colleagues asking for the latest revision of the tool, so I open-sourced it. In 2016 or so I think?
- sq is FL/OSS and will remain so, no funding is sought, not even one of those "buy me a coffee" thingies
- I didn't create this HN post about sq, nor do I know the person who did. But thanks for sq's 15 mins of fame, kind stranger
Personally, I can see a spot for sq in my developer toolbox. Instead of reaching for DBeaver’s bloated interface, or psql’s perplexing TUI, I like the idea of picking up an sq multitool to answer some simple questions quickly about my databases.
Thanks again!
That’s really harsh and misguided. If people didn’t do “paper thin abstractions” projects on their own time for the simple pleasure of doing it we wouldn’t have 90% of the successful projects we have today.
Let people have fun and don’t judge their motives when they’re making something Open Source. I can guarantee the person just thought “this would be cool to have” and implemented it.
> the person just thought “this would be cool to have” and implemented it.
Correct. This is true of much OSS, or at least I've always suspected so.
This! Is! HN!
They’re not moving anyone’s goalposts all usage is voluntary.
The fact that HN is mentioned is also confusing. If something had no value beyond paper thin abstraction I doubt we’d be seeing it on the front page with 500 votes.
Even if this project turned out to be less than hoped it seems counterproductive to complain about people creating and exploring. It’s a good thing to see and natural selection will do any sorting needed.
Why does pgcli exist. What about dbeaver, datagrip, pgadmin, postico, LINQ, ORMs? It's almost as if people value different interfaces to databases.
Maybe it's not for you, but it's not hard to imagine that someone, beyond the author, might find it useful. Maybe it's just me but to dismiss those people as individuals that should just 'learn SQL' is a pretty rude thing to say.
Those who don't understand SQL are doomed to reinvent it, poorly
Any language that features the ability to embed/interpolate unescaped quotes is not secure enough to be used directly by professional developers. Just not worth it to track down all the potential injection attacks.
And if an abstraction lets you understand the structure of queries (eg for sharding them) and even writes the code for you and balances parentheses, flags other syntax errors at compile time etc. then it’s a net benefit.
And of course there is the Adapter Pattern to abstract away connection logic and differences between mysql, pg, etc.
I wrote the “Db” library 12 years ago in PHP and still use it.
I never released it on HN for reasons like this. But I might. Here it is, embedded in my larger library, but I might break it out:
https://github.com/Qbix/Platform/tree/main/platform/classes/...
Documentation:
https://qbix.com/platform/guide/database
https://qbix.com/platform/guide/models
One of the coolest features:
https://qbix.com/platform/guide/models#relations
If you read the above, please let me know your feedback!
Question: Should I release it as a Show HN?
And the irony is of course, he never wants to use a relational DB to avoid SQL, so No-SQL DB it is.
One recently after
Here's my theory: some developers see simple languages that are easy to learn and want to build something more complex to output that language. Maybe its a sub-conscious thing.
HTML is another one, if explained simply, HTML and CSS basic use cases should be easy-ish to pick up by nearly anyone. The fact that we have so many over-engineered frameworks and libraries out there that generate these is evidence of over complicating simple things. Maybe its called resume driven development? Maybe people see genuine useful frameworks that get wide adoption and are wannabes?
The generated SQL is output in sq's logs, when logging is enabled.
Personally I really like working with SQL and find it quite elegant, I always encourage people to use it as it really is a job-superpower if you can just dig up issues directly in the DB quickly. It has a long history so for every question you have there's many answers or avenues to ask them.
https://www.scattered-thoughts.net/writing/against-sql (by another former Materialize employee) is a good takedown.
It’s not a very composable or consistent language, so I think it makes total sense that we see so many abstractions over it.
FWIW, nothing forces an extension to do so. I'm pretty sure there are several that do DML using lower level primitives.
If you have cases where you might need to conditionally join, or have any reasonably complex filtering based on input, building up a plain SQL statement using string interpolation at the call site gets very messy very quickly.
Verbosity is in the eye of the beholder; To me, it's the verbosity of error handling that makes the algorithm clear. Of course I recognize with others that opinion probably changes depending on whether the person reading a given bit of code views error handling as part of the algorithm.
I think SQL is often quite awkward, at least when you look at people exploring alternatives, like https://prql-lang.org/
Not just “the standard” variety, but also all of the vendor specific varieties that you’ll use in practice (MySQL/MariaDB, PostgreSQL, SQLite, Oracle, SQL Server etc.) and how the features offered by each differ, how the support for custom types differs, how the procedural languages differ, the actual experience of using them, seemingly half baked error messages when compared to most imperative languages, varying support for getting the query optimiser to do what you want (e.g. hints), query plans that aren’t pleasant to stare at, often no support for automated analysis of how things are running and suggestions for indices (e.g. Oracle had that sort of feature, it is sometimes helpful, but the automatically generated indices are or at least last I checked were treated as their own special thing and you couldn’t easily delete them). Even things like varying support for working with geospatial data, JSON or time series data.
Not just that, but also the tooling (or the lack of it) - good luck debugging the execution of stored procedures in your DB or placing breakpoints in there, good luck hooking up observability/tracing solutions as easily as you would for your back end, good luck debugging why your database link calls take 100x longer when executed through JDBC in your app but not in the developer tooling.
Not that ORMs or NoSQL make everything much better, you’d just trade one set of annoyances for another, especially when you end up trying to generate dynamic SQL with the likes of myBatis XML mappers. Don’t get me started on people over fetching or ending up with N+1 problems with most ORMs, or executing bunches of queries against the DB just to retrieve some data that you might as well get in one go with a well crafted DB view. Not that you can’t make your DB schema a mess as well with a liberal application of EAV or OTLT (or just attempting to make polymorphic links).
I think SQL as a language makes sense, but PRQL often feels more sensible. I feel like the current RDBMSes out there are very powerful, but that there are also aspects about them (and the differences between the ones you’ll use) that absolutely suck. It feels like where back end languages are getting progressively better DX, databases instead gave us… CTEs? Some syntactic sugar for JSON in PostgreSQL? Feels like they move painfully slow.
For what it's worth, that's why I welcome every new tool or attempt at improving things, even if they won't replace the usual enterprise stacks of having some RDBMS, a JDBC/ODB/whatever driver and probably an ORM on the back end.
I've done data processing in Perl and Python before. THAT is painful. What I think people don't realize is that doing procedural "querying" in a typical PL is, like, 10x the amount of code as an equivalent SQL query. I don't see how the alternative is much better.
SQL just doesn't work like a typical PL, but that's a good thing.
Things that seem potentially worth it to me:
* seL4
* Google's SQL syntax tweak
* Rust
* GraalVM
* systemd
* Tree sitter
* LSP
* CMake
* Bazel
These all get you a step change improvement in comprehensibility, safety, or something else important.
Things that seem like more churn than they're worth:
* Noise protocol (relative to TLS)
* JMAP (compared to good old IMAP)
* Nim/Zig/etc.
* Wayland (fait accompli now, but still)
* Varlink
* Fish shell
* YAML/TOML
* Sq?
* Meson
I wish we, as an industry, invested more in improving existing technologies instead of continually abandoning and replacing working solutions.
I would move Zig (innovates a lot in language and compiler design) and JMAP (IMAP is horrible and needs to be replaced) up and CMake and Bazel down (I count them the same as Meson). I would say the jury is still out on Google's tweak and on seL4.
Additionally, if people like jq, they don't need a special tool like this. They can just get the database to output json and use json/jq - another tool isn't needed. In postgres you can do something like this
select array_to_json(array_agg(row_to_json (r))) from (
... put your sql query here...
) r;
I'm sure other databases have similar functionality.
- no defined encoding, so it may be anything from US-ASCII over ISO-8859-x to UTF-8 and it's always a guesswork what it actually is
- no definition of the separator sign, usually it can be a literal comma, a semicolon, or tabs
- escaping of the separator sign in column values is always fun
- escaping of newlines is even MORE fun
- line endings are not specified
Every piece of software there is has their own expectations on what specific kind of format it expects, and to make it worse, using "Microsoft Excel" as a target/source isn't foolproof either because it behaves differently between the Mac and the Windows version!
JSON in contrast is clearly defined in all of these issues and has battle-tested implementations in every programming language there is.
For example, here's your list fully handled in postgres: 1. SET CLIENT_ENCODING TO 'value'; (eg 'UTF8') 2. COPY ... with FORMAT CSV DELIMITER 'delimiter_character' QUOTE 'quote_character'
Now the output format is fully specified and everything just works fine (including for input into excel)
The values in each record are separated by the DELIMITER character. If the value contains the delimiter character, the QUOTE character, the NULL string, a carriage return, or line feed character, then the whole value is prefixed and suffixed by the QUOTE character, and any occurrence within the value of a QUOTE character or the ESCAPE character is preceded by the escape character.