Hacker News Reader

Comment by loeber | Hacker News Reader

In 2025, pretending that a CSV can be a reasonable alternative to a database because it is "smaller" is just wild. Totally unconscionable.

View on HN

I use CSV files to run multiple sites with 40,000+ pages each. Close to 1mil pages total

Super fast

Can’t hack me because those CSV files are stored elsewhere and only pulled on build

Free, ultra fast, no latency. Every alternative I’ve tried is slower and eventually costs money.

CSV files stored on GitHub/vercel/netlify/cloudflare pages can scale to millions of rows for free if divided properly

Can't argue with what works, but...

All these benefits also apply to SQLite, but SQLite is also typed, indexed, and works with tons of tools and libraries.

It can even be stored as a static file on various serving options mentioned above. Even better, it can be served on a per-page basis, so you can download just the index to the client, who can query for specific chunks of the database, further reducing the bandwidth required to serve.

Just to be pedantic, SQLite is not really typed. I'd call them type-hints, like in Python. Their (bad IMHO) arguments for it: https://www.sqlite.org/flextypegood.html

https://www.sqlite.org/stricttables.html

> Just to be pedantic, SQLite is not really typed. I'd call them type-hints, like in Python

Someone already chimed in for SQLite, so worth mentioning that Python is hard typed, just dynamic. Everyone has seen TypeError; you'll get that even without hints. It becomes particularly obvious when using Cython, the dynamic part is gone and you have to type your stuff manually. Type hints are indeed hints, but for your IDE, or mypy, or you (for clarity).

It's a bit like saying C++ isn't typed because you can use "auto".

Don’t you think it’s better in this dimension than CSV though? It seems to me like it’s a strictly better improvement than the other option discussed.

A sibling comment posted a blind link whose contents address this, but (for the benefit of people who aren't likely to follow such links), recent versions of SQLite support STRICT tables which are rigidly typed, if you have a meed tor that instead of the default loose type affinity system.

TBH this is why I've never messed with SQLite.

If I want to bother with a SQL database, I at least want the benefit of the physical layer compressing data to the declared types and PostgreSQL scales down surprisingly well to lower-resource (by 2025 standards) environments.

How exactly do you anticipate using Postgres on client? Or are you ignoring the problem statement and saying it’s better to run a backend?

https://pglite.dev/

Not sure why this was downvoted, but I’d be very interested in learning how well does pglite compares to SQLite (pros and cons of each, maturity, etc)

Interesting. TIL

It sounds like you use CSVs to build static websites, not store or update any dynamic data. That's not even remotely comparable.

The way you write this makes it sound like your websites are pulling from the CSV per request. However, you're building static websites and uploading it to a CDN. I don't think SQL is needed here and CSV makes life way easier, but you can swap your CSV with any other storage device in this strategy and it would work the same.

So... SQLite with less features basically.

Every file format is SQLite with fewer features.

Unless it's Apache Arrow or Parquet.

For both fun and profit I’ve used the Parquet extension for SQLite to have the “Yes” answer to the question of “SQLite or Parquet?”

jfc is there anything Apache doesn't have a software for?

Is this a static website? If yes, what do you use to build?

In 2020 Tailscale used a JSON file.

https://tailscale.com/blog/an-unlikely-database-migration

If you continue reading, you'll see that they were forced to ditch JSON for a proper key-value database.

I know. Now see how far JSON got them.

So why wouldn't you just use a text format to persist a personal website a handful of people might use?

I created one of the SQLite drivers, but why would you bring in a dependency that might not be available in a decade unless you really need it? (SQLite will be there in 2035, but maybe not the current Go drivers)

It's self-restriction, like driving a car not using the rear view mirror. Or using "while" loops always instead of "for" loops.

It's great for an extra challenge. Or for writing good literature.

You didn't really answer the dependency argument though.

Until the data for a static website becomes large enough to make JSON parsing a bottleneck, where is the problem?

I know, it's not generally suitable to store data for quick access of arbitrary pieces without parsing the whole file.

But if you use it at build time anyway (that's how I read the argument), it's pretty likely that you never will reach this bottleneck that makes you require any DBMS. Your site is static, you don't need to serve any database requests.

There is also huge overhead in powering static websites by a full-blown DBMS, in the worst case serving predictable requests without caching.

So many websites are powered by MySQL while essentially being static... and there are often unnecessarily complicated layers of caching to allow that.

But I'm not arguing against these layers per se (the end result is the same), it's just that, if your ecosystem is already built on JSON as data storage, it might be completely unneeded to pull in another dependency.

Not the same as restricting syntax within one programming language.

> SQLite will be there in 2035, but maybe not the current Go drivers

Go binaries are statically linked, unless you expect the elf/pe format to not exist in 2035 your binary will still run just the same.

And if not well there will be an SQLite driver in 2035 and other than 5 lines of init code I don’t interact with the SQLite drover but rather the SQL abstraction in golang.

And if it’s such an issue then directly target the sqlite C api which will also still be there in 2035.

Not so sure about this. At scale, sure, but how many apps are out there that perform basic CRUD for a few thousand records max and don't need the various benefits and guarantees a DB provides?

I assume parent's dispair is about CSV's amount of traps and parsing quirks.

I'd also be hard pressed to find any real reason to chose CSV over JSONL for instance. Parsing is fast and utterly standard, it's predictible and if your data is really simple JSONL files will be super simple.

At it's simplest, the difference between a CSV line and a JSON array is 4 characters.

If you ignore size as a benefit, CSV files still have a lot of value:

    - It's plain text
    - It's super easy to diff
    - It's a natural fit for saving it in a git repo
    - It's searchable using standard tools (grep, etc.)
    - It's easy to backup and restore
    - You don't need to worry about it getting corrupt
    - There are many tools designed to read it to produce X types of outputs

A few months ago I wrote my own CLI driven CSV based income and expense tracker at https://github.com/nickjj/plutus. It helps me do quartly taxes in a few minutes and I can get an indepth look at my finances on demand in 1 command.

My computer built in 2014 can parse 100,000 CSV rows in 560ms which is already 10x more items than I really have. I also spent close to zero effort trying to optimize the script for speed. It's a zero dependency single file Python script using "human idiomatic" code.

Overall I'm very pleased in the decision to use a single CSV file instead of a database.

Reply on news.ycombinator.com