Hacker News Reader

Comment by brap | Hacker News Reader

I don't mean to be a partypooter, but I watched the full Datasette pitch video and there's something I probably just don't get.

Isn't this really just a SQL GUI? Like basically any other SQL admin panel out there (minus the the writes)?

What's the distinctive feature here? The extensions?

View on HN

I think of it more like MS Access but a sane backend of sqlite and python. There are thousands and thousands of critical business processes cludged together in Excel and Access--datasette could be a much better choice for those use cases. Something both devs and business people can use.

Yeah Access is a really interesting comparison (Datasette has quite a way to go on that front).

I find it baffling that Microsoft haven't invested more in Access. The world needs a truly great desktop/mobile database solution! Excel isn't enough.

Regular human beings should be able to point a full database at their own problems.

Totally agree, so many things people get strong feelings about customizing workflows--note taking, todo lists, personal document management, inventory of goods, etc.--are really just a sqlite database with some nice custom views and interfaces. I could definitely see a future where datasette or similar tools can replace some of that stuff.

Access is probably caught in a weird spot internally at MS. If they put effort into it then it just removes some of the need to sell proper SQL server or azure cloud database tech. Better to just limp it along then start internal wars with bigger organizations/products.

And the great thing about those tools is that Datasette doesn't need to replace them - SQLite becomes the integration layer, so you can use any tool you like that provides a neat UI to storing data in SQLite, then use Datasette itself directly against that same database when you need to run your own SQL or integrate with other JSON apps or run custom plugins.

Check out Grist in the ‘Access with sane backend’ space. SQLite, open source and fantastic UX https://www.getgrist.com/ and https://github.com/gristlabs/grist-core

I use and love both Datasette and Grist - they’re complementary.

I was constantly thinking about MS Access while watching the introductory video. I loved MS Access in the 90s, and this being based on SQLite and Python makes it really great.

The bigger pro is the fact that you can export the data as JSON, which basically means that you have a server for your SQLite file which other applications can query against, without needing a full blown database server like MariaDB or Postgres while you still have the possibility to explore the data manually.

So for small projects this seems to be a really good tool.

This is a perfectly reasonable question. I've been thinking about this for five years and I still don't have a great, snappy answer to this.

I can answer it in several paragraphs, but I really would like to be able to answer it in a single sentence some day!

So here goes with the several paragraph version...

The problem that Datasette solves better than anything else (a big claim, which I'm happy to be challenged on) is publishing structured data online.

Let's say you have data you want to share online - every global power plant, or the full history of the US congress. How do you do it?

Some options:

- Publish some CSV files somewhere - on a website, on GitHub, to S3

- Build a custom application for it, often using Django or Rails or similar.

- Put it in a Google Sheet - that's how we solved this problem at the Guardian years ago, see https://simonwillison.net/2018/Aug/19/instantly-publish-data...

Datasette was originally created to take on this problem. I realized that SQLite is the perfect platform for this: it's fast, robust and crucially can be deployed anywhere that can host a dynamic web application (if you're publishing read-only data you don't need to worry about backups and replication and suchlike).

Here's every global power plant: https://global-power-plants.datasettes.com/global-power-plan...

And US congressional legislators: https://congress-legislators.datasettes.com/legislators - I use that one in the Datasette tutorial: https://datasette.io/tutorials/explore

If you're comfortable with the command-line, I challenge you to find a quicker way to publish data online than this:

    sqlite-utils insert manatees.db locations \
      Manatee_Carcass_Recovery_Locations_in_Florida.csv --csv -d
    sqlite-utils transform manatees.db locations \
      --rename LAT latitude \
      --rename LONG_ longitude \
      --drop created_user \
      --drop last_edited_user \
      --drop X \
      --drop Y \
      --drop STATE \
      --drop OBJECTID \
      --pk FIELDID

    datasette publish vercel manatees.db \
      --project datasette-manatees --install datasette-cluster-map

That's using the datasette-publish-vercel plugin, but Datasette can also publish to Fly, Google Cloud Run, Heroku and more using additional plugins: https://docs.datasette.io/en/stable/publish.html

For more on the sqlite-utils bits see https://datasette.io/tutorials/clean-data

So that's publishing. But Datasette has grown far beyond that in the five years I've been working on it.

I find myself turning to it any time I have any data I want to poke at and start exploring. That's the data journalism angle - "find stories in data".

In terms of commercial applications, I have a strong hunch that if I can help journalists find stories in their data, I can help everyone else find stories in their data as well.

Another key detail here is the plugins.

WordPress is a good CMS... with 10,000+ plugins that mean you can point it at any content publishing problem you can think of. As a result, it runs a double-digit percentage of the web now.

The most ambitious version of Datasette looks like that.

I want to build an open source EDA (Exploratory Data Analysis) and publishing tool that has thousands of plugins that mean you can use it to solve any data exploration, analysis, visualization or publishing problem.

It's at 127 plugins so far, so there's still a long way to go - but it's a great start! https://datasette.io/plugins

Attempting to turn the above into single sentences is hard, because there are a lot of different angles to it - but here are a few attempts:

Datasette is the fastest way to publish data online as an interactive, searchable database.

Datasette is WordPress for data: an extensible open source platform with plugins for exploring, analyzing, visualizing, and publishing data.

Heh...

"I can answer it in several paragraphs, but I really would like to be able to answer it in a single sentence some day!" ...

"Datasette is WordPress for data: an extensible open source platform with plugins for exploring, analyzing, visualizing, and publishing data."

Confession: I posted my multi-paragraph comment into Claude and asked it for some ideas. It didn't come up with exactly that, but what it said helped me get there: https://gist.github.com/simonw/a160a49d39446aa53870ed1ca43d7...

This is a great explanation. And to me, what sets Datasette apart from a generic SQL UI, is that Datasette excels at publishing _specific_ and curated datasets and allowing interactive exploration in a way that plain CSVs just don't offer.

Reply on news.ycombinator.com