Hacker News Reader

I hope that app isn't actually running on a server somewhere for each commit indefinitely, and that it's moreso intended for serverless setups, like the Netlift edge functions they mentioned...

We could do with some more consciousness for energy & compute resources in this industry, as decoupled it may be from the real world, clicking the button to deploy an EC2 instance somewhere does use real power and will contribute to hardware wear.

It's serverless functions. So they should launch those instances on the fly when a request hits that url and that instance will shutdown after some inactivity. Same with all the 50+ edge locations they provide. They only launch an instance instance in particular region, when user hits that region.

I'd spawn a new DB for that instance too. It can still go a long way to give you a look at how your application worked in the past.

Am I mistaken in thinking that’s totally ridiculous?

Let’s charitably say you can run your app with a 1GB cut down snapshot of production.

You’re going to do what, save a snapshot of the database for every commit? Every schema migration?

…and then provision that up in a running database server… which is notoriously not container friendly…

? It doesn’t sound very plausible to me.

Our database is ~1TB, and the app runs with maybe a 10GB cut down version of it, which is a pain in the ass to deal with, just for example, for local dev environments.

I agree with the parent post; it’s cute, but I can’t imagine historical per commit deployments being useful for me, honestly.

> Let’s charitably say you can run your app with a 1GB cut down snapshot of production.

Plenty of apps can run from an empty database, if the data storage is focused on resources you, the user, have created for yourself.

But I imagine that people with their heads in the future are imagining you'll use one of these new database startups (the names escape me) to manage snapshots, seeding from anonymised production data, etc.

Historical deployments would be useful to me because our app rarely has database schema changes in the hot path.

Most of our effort is in user experience and post processing.

It would be nice to compare the experience of todays version with one from 6 months or 1 year ago for certain components.

We do already have a historical design storyboard but seeing it with live data would be more useful.

What do you need production data for?

Just fire up a fresh DB with some sensible default fixtures.

Or pause an integration test mid-way.

Or call some factories from the REPL.

The last thing in the world I want to do development with is production data. Production data is the data of last resort when nothing else can repro.

You can use a seeder script to generate fake data. If you are deploying something that is not production you shouldn't use real data, especially if it contains personal data.

There are ways to version your database schema. Then you just need to make sure your code version is tied to your database version.

You can't restart all of your web services atomically. What happens if two web services are running two different versions for a couple seconds?

If you want to get super fancy, always compatible database migrations. Have both versions live, migrate to the new schema in 3 steps:

1. Old version

2. Old version/new version both live.

3. New version live after step 2 is confirmed as fully done.

Right, I was just pointing out the flaw in the parent commenter's statement. You can't do "code version is tied to your database version" - that's just not a thing.

> The post mentions "The open source Deno runtime shows how clean and productive a modern, batteries-included, programming environment can be" so it sounds like they expect you'll be running database backed apps and not only static sites.

It suggests you'll be running a datastore, not necessarily an SQL database (the most overrated technology in existence IMO, especially for web apps where essentially none of its strong points are relevant). Storing old data as-is and migrating on read is definitely doable, and you can keep backwards compatibility to day 1 that way relatively easily. I worked on a system much like the one from "An oral history of Bank Python" that did exactly that, and had been doing so on a large scale for around a decade. Having a better-integrated datastore that can present multiple views of the same data is another way to achieve that, if you want to keep the migration out of the "application" code.

More practically: for example, I write my web service(s) with event sourced architecture. Now any worker, or any service sending a command, can operate against the storage. Regardless of version.

Other architectural patterns offer the same kinds of options. Clean architecture or hexagonal architecture limits the surface (coupling) between your versioned logic and the data so much, that they can version independent.

Alas, most web development uses some ORM as the center of the app. Active record, for example. An architecture known for its immense and tight coupling to the database. An architecture that has as major downside that businesslogic cannot evolve separate from the data store.

Your parent commentor very likely is, unaware, running into this downside and projects that experience to all of web development.

> More practically

> event sourced architecture

You must be joking. Please tell me you're joking.

I'm not joking. You are missing (or deliberately leaving out) context:

I'm not saying event sourcing is a more pragmatic (practical) solution. I'm giving an example from practice.

It really is a lot more practical. Most webapp devs using SQL databases spend a bunch of time engineering some notional failover into their app that's completely undermined by using a non-HA datastore, a bunch more time mapping data to and from square tables where it doesn't really fit, and end up with a system that's still last-write-wins at the user-facing level. And the ones where it isn't aren't because they used those SQL transactions, but because they wrote their own versioning layer on top.

So what’s a good web app datastore in your opinion, if SQL is highly overrated?

> Storing old data as-is

What does this mean? Can you give an example?

> So what’s a good web app datastore in your opinion, if SQL is highly overrated?

I've used Cassandra very successfully, but there are plenty of options.

> > Storing old data as-is

> What does this mean? Can you give an example?

I mean not migrating the stored version of old data (note I'm not suggesting "no schema" or anything like that; what I'm opposing is the idea that you have a single global mutable version of the schema and mutate historical records in place to match that). E.g. if you started by storing users with a single nationality field and then realised that actually the same user can have multiple nationalities, while you'd change the DTO representation of a user that you read into, you wouldn't rewrite the on-disk representation of previously stored users.

Cassandra is horribly inflexible.. you cannot use where conditions in any fields other than the PK (assuming you're not going to use ALLOW FILTERING for all your queries) and if your data isn't going to grow beyond baby size where allow filtering is okay, then you probably don't need a whole distributed java app running. Just do sqlite or newer local file storage dbs etc.

SQL has earned its place due to the expressiveness and flexibility .. you may hold your opinions but you'll hardly find anyone who would take 'SQL is overrated' seriously.

It means instead of migrating the database, you just leave it as it is. New code will expect to see both the old and new record formats.