Hacker News Reader

I think you are missing the big twist. It's not just tables as 1st class citizens, but allowing logic to be driven by the tables.

Instead of config files, you update the table. Changes to the processing flow? Update the table, including dates for when the new rules apply. The tables held code which drove the processing, along with tables holding data.

It's not just orm or persistence, and not just programming in the database as stored procedures. It was an odd melange of all of this.

I ran into this in the 90s, and it was great for RAD. But it felt odd to have to code into tables, and each tool was proprietary such that moving off of the table system was a full rebuild. They usually allowed migration to new databases systems to scale, but that was all they had.

I don't expect to see a language like this come around again anytime soon, but the ideas were really interesting in a world before git-ops and yaml configs.

>, but allowing logic to be driven by the tables. Instead of config files, you update the table. Changes to the processing flow? Update the table, [...]

I didn't miss that angle and I think it's actually a minor part of his thesis. If you look at the entire essay, the vast majority of his bullet points and supporting examples are mostly about ergonomics of builtin syntax to manipulate tables. If his ideal _language_ (aka the syntax) did that, it would naturally support table-oriented-programming (aka the philosophy). He starts the essay with critique of OOP-the-syntax.

But to your point about config and code itself being persisted in the database, the SAP ABAP environment already works like that. SAP has over 10,000 db tables for configuration -- instead of YAML or JSON files. Change the values in the config tables to alter behavior instead of modifying IF/THEN/ENDIF statements in code. And when ABAP programmers hit "save", the code gets saved to a database table instead of a text file. So if one squints a certain way, the SAP system is a giant million-line stored procedure in the database.

Well put. SAP is the big survivor from that mindset. And perhaps the best example of why we won't see another. :-)

> In any case, I don't see any trend where a general-purpose programming language will include DB SQL as 1st-class. Even the recent languages like Rust and Zig don't have basic SQLite3 db persistence as convenient built-in syntax.

There is a trend, but you’ll have to look further off the beaten path than even Zig to find it. Languages like Eve [0] tried to do this circa 2015 in the tradition of Datalog. Code was written in “blocks” that resembled Prolog horn clauses, but which featured set semantics on selected records. Natural joins happened automatically on records using identifiers. The whole language was actually a database!

Eve died [1], but you’ll see many such projects that have the same ethos in communities in the web, such as this one [2].

There aren’t a lot of users of these languages, but this is where a lot of big ideas are percolating right now.

And we can verify it’s a trend because the hallmark of all CS trends, the formation of a conference, has made itself known in this area [3].

0: http://witheve.com/

1: https://groups.google.com/g/eve-talk/c/YFguOGkNrBo?pli=1

2: https://futureofcoding.org/catalog/

3: https://www.hytradboi.com/

> I remember reading this essay when it first came out. To try and reword it using modern terms: The author wishes that programming languages had database persistence capabilities as 1st-class built-in syntax instead of cumbersome bolted-on API functions.

This reminds me of M/MUMPS, used by Epic to power the biggest EHR system by market share in the US.

Perhaps the big difference is that the M "database" is key-value structured. True tables are flat and do not distinguish part of the tuple as the "key" and part as the "value".

I wonder if this is the source of the oft-discussed "mismatch" between the programmer's model of data and the relational model of data. Programmers like to assign values to things, while relational DBs like to do CRUD operations on records. (This is sometimes called the "object-relational impedance mismatch" but I've always found this term silly - needlessly jargon-laden and scoped overly narrowly to the OO paradigm.)

There's clearly some kind of "isomorphism" or translation between the two models, but they're not quite the same.

Is this what ORMs are about? Translating between the programmer model of data and the relational DB model?

> Is this what ORMs are about? Translating between the programmer model of data and the relational DB model?

Pretty much, AFAICS.

With varying degrees of success.

Such tools existed and were popular in 1990s: DBase, Clipper, FoxPro.

They worked pretty well in their domain: data entry and report generation, with lightweight transaction processing and general computation.

Then happened the internet and client-server architectures, and these do not map as neatly onto local, single-user, single-transaction tables.

> Such tools existed and were popular in 1990s: DBase, Clipper, FoxPro.

Also Crystal Reports, Paradox...

> They worked pretty well in their domain: data entry and report generation, with lightweight transaction processing and general computation.

Having bought Ashton Tate, Borland got DBase and Interbase in addition to Paradox, and built data access components into the VCL class library (in effect "almost-first-class citizens" of the language), which IMO made Delphi the natural and superior successor to those languages: Not just "lightweight", but fully advanced (i.e, ~C++-level) general computation. (And with transaction processing built into the RDBMS connection components.)

> Then happened the internet and client-server architectures, and these do not map as neatly onto local, single-user, single-transaction tables.

Weeelll... Seen the spate of recent posts on here about how SQLite is good enough for pretty much anything? :-) And arguably, that's where Delphi was at too, over twenty years ago: AFAICR, there was a "Fishbase" (facts about tropical fish) demo included with Delphi, which in one variant could be built as a standalone Web service / server.

Also, AFAICS, that's where Free Pascal / Lazarus is at now, only using SQLite / Firebird / MySQL / PostgreSQL (and lots of other DBMSes) in stead of DBase / Clipper / FoxPro / Crystal Reports / Paradox. (I've been planning to look into that a bit closer myself, but haven't got around to it. Procrastinating away too much of my time on Hacker News, I suppose. :-( )

Duh, forgot to mention: "Fishbase" was based on a Paradox (IIRC; could have been DBase) file.

Edit: Also, for FP/Lazarus that's probably SQLite and newer RDBMSes not "in stead of" but in addition to the old file formats and RDBMSes.

I have a ton of games, bought or downloaded from a variety of places, some even for different systems and some more than once (e.g. Steam or GOG or sometimes some Humble Bundle bundle giving a game i already had) and in some cases a different version. Most of them are on my external HDD.

One thing i want to do is to make a database of all of them since they're easily more than a thousand and i want it on my PC. I want to be able to have a title, description, tags, screenshots, overall category, target platform info, links to the setup/archive files, the versions i have, extras like wallpapers, ringtones, music or even perhaps any box pictures or ads if they are available (some sites like Zoom Platform do provide those with the games you buy), reviews, patches (official and unofficial), links to folders with any mods i might have, etc.

This sounds like something that back in the day would be perfect for dBase or Fox Pro, except for the part where i want it to be graphical (remember: i want screenshots, wallpapers, etc). But if those were made with a GUI in mind, i'd expect them to be perfect. So basically, i think Access would be it. But AFAIK Access is now dying, last time i checked the UI it seemed like a massive downgrade compared to what i remember from when i first saw it back in Access 97 days (look, some stuff really work better with nested windows / MDI applications - having a two field form or 5 field table take up the entire available application space when 90% of is empty space makes no sense when you could have a bunch of windows with those things visible at the same time). And it is limited on Windows and TBH it looks like a massive program anyway.

Some alternatives mentioned are too much "programming" and not enough "database-ing" IMO. Ideally i should only need to bother with code to cover for any missing functionality not provided by the GUI but most of the DB and UI design (for the forms, etc) and ideally most simple behavioral stuff should be done from the GUI without using any code.

But i don't think there is anything really like that and the modern computing world feels way too "incompatible" with what i have in mind - e.g. whenever i mention this to some friends of mine they start thinking in terms of wiring together stuff like SQLite (or even worse, PostgreSQL), Python/PHP/whatever, some web-based framework, etc and other "mass-of-unrelated-software-held-together-by-duct-tape" solutions, when what i really want is a completely self contained program with no external dependencies (aside from the basic stuff for showing a GUI, etc, i mean not requiring stuff like setting up a PostgreSQL DB), no servers, etc, just a binary/EXE, saving the DB somewhere on the filesystem (ideally in a single file like Access so it is easy to copy/backup/pass around), being able to interact with the rest of the OS (remember that bit about keeping track of setup programs / archives /etc ? I'd like being able to run those directly from the DB GUI), etc.

Well, one of the 28378423 things i want to work on at some point in the future. Hopefully those life extension studies that are posted on HN now and then will eventually move on from rats :-P

I'd be curious to see with other options exist, but the one that I know is Filemaker Pro. I'm only familiar with the stand alone version, and while expensive, to me it sounds exactly like what you're talking about wanting.

I've never used it (and the fact that you need to fill a form to download a demo means that chances are i'll never use it :-P) but judging from the Wikipedia page it looks like it might have been close at some point but somewhere around late 90s/early 2000s it succumbed to the need to excuse its pricepoint and got too overbloated and enterprise-y (i mean, apparently even in 1997 it would act as an FTP server...? :-P).

What i have in mind is more like something between dBase and VB (VB3 at most), but with a fully integrated database that can understand graphics (ie. in addition to data types like text, number, etc it also has graphics), a more GUI driven workflow for basic stuff (e.g. instead of placing a button in a form and doubleclicking it to enter in a code editor where i type something like "form2.show" to show a second form, instead i have a dialog appear that provides some common tasks like "show a dialog, add/revert/delete/etc record, open external file, etc" with "runs script" being an option for when others wont do), being able to either infer table structure from form fields or automatically create forms from tables, etc.

Also completely self contained, no need to install separate "database drivers", or run any sort of server (even locally), just unzip some archive somewhere and run the program from there.

One of my favorite programs ever was built on DBase in the mid-90s.

Installing or moving the entire application and database to a new PC was as easy as "drag folder onto flash drive" -> "drag folder off of flash drive".

Instant loading, tiny file sizes, but it became too complicated for users when 64-bit windows pushed 16-bit usage into VMs.

Another relevant software category is the statistical analysis languages, including SAS, Stata and SPSS.

Old-school SAS included only two data types (floats and character strings), but allowed for SQL and sequential data-steps to live together. Persistence was baked in. The floats could be used to represent dates, datetimes and other formats. I particularly appreciated being able to use macros to define a data-step view to split the follow-up for an individual from a table. Such a view could then be collapsed using SQL. More recently, R tools such as dplyr have brought together data-frames and relational operations. However, I miss the sequential coding in SAS, using macros as higher-level tools to define the logic, including corner cases.

For strictly typed records, I have always wanted to spend more time with SML# [0] this allows for record updating, with close ties to SQL -- an under-appreciated version of SML.

[0] https://github.com/smlsharp/smlsharp

> In any case, I don't see any trend where a general-purpose programming language will include DB SQL as 1st-class.

Which is a pity I'd say. The new languages you mention would benefit most from such a feature. I believe they won't do it not because it's not useful, but because it's difficult to make it right (efficient, safe, natural, scalable) and then maintain forever.

There are many benefits of having this functionality working out of the box (and a few disadvantages, obviously). Many (if not most) apps are just CRUD apps with some added functionality. But a standard way of connecting the language to a database is still missing. The great success of the ActiveRecord back in the day shows that this is something many developers would benefit from (it was/is good, but still not ideal). And I don't believe patching the situation with a multitude of incompatible ORMs solves anything.

Rust has rather sophisticated macros, which let you do stuff like this outside the core language implementation, which is IMO very much where such things belong.

E.g. a linq clone in rust can look like this: https://github.com/StardustDL/Linq-in-Rust

    linq!(from p in 1..100, where p <= &5, orderby -p, select p * 2).collect();

> In any case, I don't see any trend where a general-purpose programming language will include DB SQL as 1st-class. Even the recent languages like Rust and Zig don't have basic SQLite3 db persistence as convenient built-in syntax. If anyone proposed to add such db syntax, they would most likely reject it.

What poor old BottomFeeder missed was that with a good object library / framework, you can get so close that it almost doesn't matter. I tried to convince him to even try Delphi, with its marvellous TDataSet descendants in the VCL... But AFAIK he never even downloaded the free version I pointed him to, whatever it may have been called back then.

The K language of kdb also count, but being proprietary and having a fairly impenetrable and alien syntax haven't helped it branching out of the niche where it is very successfully.

Isn't Q the language, and Kx the company ?

I have to use it at work, helping quants try to actually maintain production code rather than just vomit horrible one-time scripts they patch together in an endless stream of layers on top of layers.

This thing should be banned. Everything is one-letter, it's impossible to Google, it takes the opposite decision of every imaginable convention, I have yet to see someone who can read his own stuff 2 weeks later. Notwithstanding free tools to query force-expire every 3 months (QPad grrr) and as you said it's so closed they have to take webdevs like me to just help them maintain it all eventually.

It's risky for the bank (and we're not a small one :s), it's expensive for the programmer, and it's misery for the quants (but they all feel like geniuses spending weeks on simple stuff on kdb, so their misery is only in people looking at them wasting away their brain like that).

You are right. As far as I understand Q is K plus the additional sql-like syntax.

And yes, it is nigh-unreadable. I have yet to learn it, but I think there is value in succinctness at least for throwaway scripts.

Yeah but there s low value in throwaway, and it's not succint, it's 1-letter. ONE lol, for every imaginable labguage keyword.

For instance to parse a binary encoded dictionary from a table column: -9!'columnName(I remember it because it s the first time I spent a day on something so useless yet so indispensible yet so fucked up). I challenge you to google it.

He likes succint hihihi.

If you use k a lot, you start to automatically recognize idioms/stanzas; it just becomes automatic to recognize a group of chars. And then it becomes readable. The throwaway nature, in my experience and opinion, comes not from it being unreadable (as many people who never used languages like this seem to think), but rather from the fact that the code you are are changing is so short that thinking it up and typing it in (by combining idioms) is faster than placing your cursor and actually editing existing code. Also you are on a repl or editor that can run code on the cursor, so interactive development even makes that case stronger: before you know it, you have produced a new version of a function without actually even looking at the old one.

For a nice intro, read Nils' excellent book: https://www.t3x.org/klong/book.html it even has an idioms table in the back.

Why is it still used then? Are there just no good alternatives? Or legacy reasons?

On the things it does well - being a memory mapped column store, it hardly had any competition a few years ago (clickhouse might be getting there these days, perhaps)

And if your fluent in K, one-of scripts and queries are significantly shorter and easier to get right.

But it is not good as a general purpose language in a commercial setting - not because of the language itself which is fine if a bit spartan - but because it is hard to find people who are willing and able to work with it.

Much like its predecessor APL, it’s a tool for thought more than a tool for implementation.

Yeah I personally really like array langs so I was just wondering why other people don't.

I think many people would work in it, but the job market for it is limited and driven by who knows who as far as I noticed. I hope Shakti will make things better: so far it's nice and definitely it's pricepoint is a lot better. Not sure where it will go.

How do you know the price point is better? It's not on the website. Are you a customer?

> but they all feel like geniuses spending weeks on simple stuff on kdb

They are not experienced in the environment then? It is not that hard. Sure it is spartan as the tooling is not what you expect in 2021, but this seems a bit over the top?

> I have yet to see someone who can read his own stuff 2 weeks

http://nsl.com/ can, years later even (as can I but he has an impressive portfolio which makes it obvious).

The "V" programming language has these features built in. https://github.com/vlang/v/blob/master/doc/docs.md#orm

Does this doc misuse present tense or is it actually implemented? V's author has done this a bunch, so it's tough to take a doc at face value.

I just copy+pasted the example given, it compiles and runs

> Examples where database syntax (i.e. SQL syntax) is 1st-class without noisy syntax of function calls, without command strings in quotes

There more examples which I think qualify but don't quite fit into your categories:

* The E language runs all code in "Vats", each of which is a single threaded compartment with transparent persistence.

* Taking inspiration from E, the Waterken server did this for Java, but required annotating mutable fields in a certain way so the persistence layer could track them.

* FoxPro doesn't neatly fit into your categories.

How about LINQ, in particular LINQ2SQL?

Racket has a first class SQL query DSL. [0]

With some kinda dynamic naming system, you could probably load SQLite table schemas at runtime and provide bindings to columns automatically. Or maybe that's best done once at the compile step. Either is possible with racket :-)

[0]: https://docs.racket-lang.org/sql/index.html

Although it's not a relational database, MUMPS is another example of a language where there is nothing special whatsoever about manipulating the database compared to manipulating the same data but stored in a variable in RAM.

Didn't pascal have a way to persist/reload records. Not SQL-like but still something.

Also it's interesting you mention cobol. It really was a cool feature.

Not as any kind of standard, or even a popular extension.

Microsoft BASIC implementations for DOS however had ISAM in some of the editions (PDS and VB for DOS) which were similar: you'd declare a struct type, and then open a file with that type as the record type.

Pascal's records are basically like C's structs: you can use them to make a database (and many did) but there isn't any real support from the language, though some implementations may have had their own extensions since the original Wirth Pascal had no support for files at all and the standard had some very limited support.

I wrote CRUD apps that did this in Turbo Pascal back in the days of MS-DOS. I copied the idea of screen fields that I had seen in DBASE II, and make a set of field editors for each record type, I could put together a database pretty darned quick back in the day.