SQLite-HTML: A SQLite extension for querying, manipulating, and creating HTML | Hacker News Reader

SQLite-HTML: A SQLite extension for querying, manipulating, and creating HTML

github.com

130 pointsthunderbong16 comments

Hey, author here, happy to answer questions! A few other recent posts/tools that you may be interested in:

- sqlite-lines discussion from a few days ago: https://news.ycombinator.com/item?id=32288165

- htmlq, Rust CLI for (like jq but for html): https://github.com/mgdm/htmlq

- The Go library that sqlite-html uses for making runtime-loadable SQLite extensions https://github.com/riyaz-ali/sqlite

- sqlean, a ton of other helpful SQLite extensions (in C): https://github.com/nalgeon/sqlean

The bottom of the README links to a sqlite-http repo but that's just a 404. Is that a private repo?

Ah yes, my bad, gonna publish that soon! Will be an extension for things like `select http_get_body('https://google.com')`

Maybe I'm being dumb, but what's the upside to generating/modifying HTML in SQL as compared to javascript?

totally not dumb - in general there isn't much upside. Maybe there's a usecase in server-side rendering HTML where you already use SQLite somewhere, or in a CLI tool if you're using sqlite3. But I doubt sqlite-html will be as fast as other tools. The querying functions are much more useful than the generating ones, IMO

Not an extension, but I wanted to add that sqlite-utils is an incredible Python library for working with SQLite dbs. Great for squashing several lines of SQL into a one-liner.

https://sqlite-utils.datasette.io/en/stable/index.html

Don't miss the Observable notebook which includes interactive demos of the new functions: https://observablehq.com/@asg017/introducing-sqlite-html

I'd love to query HTML in the browser with SQL. I'd really love to join data across websites in the browser.

I'm working full-time on exactly that, hope to launch soon.

Can you say more? It sounds interebut why would anyone want or need this?

I see a million use cases: no-code data scraping, analytics, feeds, notifications etc. Just think of the web as a giant database with URLs as table names and CSS selectors as column names. I started to work on it about half a year ago after I got tired of writing Python scripts every time I needed to join subsets of various online databases.

Very interesting proposition.

Is there any further info and any similar work on this subject?

> Just think of the web as a giant database with URLs as table names and CSS selectors as column names

Is this a reinvention of YQL? https://en.wikipedia.org/wiki/Yahoo!_Query_Language

This is amazing. I've been exactly thinking recently about the state of computing we're in: we keep using programming languages to build full solutions, which barely derive from something that already does the same but with a different color. How many times do we need to reinvent a forum ? Synchronous and asynchronous communication ? Social networks ?

I want to have a better, synchronized feed reader. I can install one of the many choices of full-stack solutions, with integrated clients, open protocols, etc. Or I can use my existing IMAP server to store it all, and have it already synchronized for free. Not only is the protocol open, but it is already widely known and is generic enough that it can be used for many usecases.

I want my RSS feeds in my IMAP. I want my Pleroma discussions in my IMAP. I want my HN discussions in my IMAP. Building upon existing solutions and interconnect them rather than taking the programming language and create something from scratch.

I'd love to take a test drive whenever you need an alpha tester. My Gmail username is the same as my HN username.

Let’s not forgot the impact “SQL on Rails” had on the industry a few years back /s https://youtu.be/0_PK1eDQyVg

Reply on news.ycombinator.com