Hacker News Reader

Comment by mg | Hacker News Reader

I wonder how much of a factor the "One DB, one file" concept is in SQLite's success.

I always loved that an SQLite DB is just a single file. People say "Who cares how many files the DB is?". But in practice, I always found it to be very convenient. It makes the concept of a project easy to grasp when the data store is simply a file.

I recently realized that a Django project can also be done in a single file. That made me like Django even more.

I have the feeling that this type of logic and simplicty is indeed a factor in the long term success of a software project.

View on HN

I think it's the same reason Windows feels easier than Linux to manage software in, or why Docker has the extremely fast and wide adoption it enjoys, or why so many of us hate working with modern web stack.

Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide).

Docker, obviously, because it eliminates all the bullshit of installing software - again, especially painful on Linux systems - and gives you a single file that drives it all. The platform itself may be system-wide, but the thing you care about: individual software - is contained (literally) so it doesn't spill out. Easy to reason about.

Modern web stack is a mess of tooling, some of which system-wide (so people will reach for Docker to contain it all!), with huge version churn and lots of moving pieces. Source code of your app doesn't feel like a major component of it; it's useless and unparseable without chains of finicky tooling. That's in contrast to "vanilla JS", old-school experience, where source files were the only thing that mattered. No transpilation, no build chains. You could fire a site up straight from your hard drive, or FTP/SCP it to a file host, and It Would Just Work.

Same with SQLite. To this day I don't like RDBMSes, because they're a system-level platform designed for admins, while all I care about is the RDB part. My database. SQLite maps conceptually to how I think about it - one database, one file (+/- WAL). One well-defined place my data lives, that I can move around and between machines using regular file operation tools.

Etc.

Files over apps, always.

EDIT:

This also makes me very eager to try doing something with RedBean[0] - a webserver in a single Actually Portable Executable[1] that's also a ZIP file into which you put all the data you want it to serve. A turn-key website in a single file!

[0] - https://redbean.dev/

[1] - https://justine.lol/ape.html

> Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide).

This hasn't always been true. The portable executables are a nice user convenience for certain apps, but by no means all, and Windows is the platform for which "DLL hell" was coined. They even got briefly worse with the "GAC" for .NET Framework, although it rapidly became apparent what the problem with that was.

These days they really want you to use .appx and ship via the Microsoft Store, so of course nobody does that and all Windows software is (a) nice single executables (b) sprawling DLL monsters (c) javascript in a box (Teams!) or (d) games.

Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF.

> Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF.

Isn't this already done better by jart's APE I mentioned earlier[0]? Though I imagine you could statically link SQLite to your APE and stuff a .sqlite DB into the ZIP file at the end, to use that for static storage. Or read-write storage if you also link zlib (though I'm not sure if you can portably overwrite your own executable on the hard drive).

[0] - https://justine.lol/ape.html

> I'm not sure if you can portably overwrite your own executable on the hard drive

You definitely cannot do this on Windows.

No, but you can

1. Write the new executable.

2. Rename the running executable.

3. Rename the new executable to the original name of the running executable.

4. Launch the new executable.

5. Exit the original executable.

6. Arrange for the new executable to delete the renamed original executable once the original has exited.

This is admittedly a bit fiddly, requires care if you need to preserve attributes and ACLs of the original file, and has obvious issues in scenarios where multiple instances of the original executable may be running concurrently, but it does work.

You also have to be careful to consider the ramifications of a system crash or snapshot backup in the middle of this process: ensuring the new executable is safely flushed to disk before it replaces the original is, under normal circumstances[1], easy (FlushFileBuffers); ensuring that one of the two executables always exists under the original name is hard (the Win32 ReplaceFiles API sounds like it should help here, but it doesn't).

The cleanest solution to the latter problem I've come up with is to use the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations registry key as a backstop to ensure the move (3) happens, if necessary, as soon as the crashed/restored system is booted, but this requires local admin rights (or wanton disregard for system security, i.e., changing the ACL on this registry key to allow ordinary users to write to it, and therefore any file on the system on every reboot).

[1] Here "normal circumstances" exclude cases where hardware lies[2] or users explicitly disable buffer flushing[3].

[2] https://devblogs.microsoft.com/oldnewthing/20100909-00/?p=12...

https://devblogs.microsoft.com/oldnewthing/20170510-00/?p=95...

[3] https://devblogs.microsoft.com/oldnewthing/20130416-00/?p=46...

This is one of the reasons I love Go, it can not only build to a single statically linked library, but you can embed files your app needs like html templates into that same binary. Deployment is copying that binary to where it needs to go.

Django in one file? How?

https://coltrane.readthedocs.io/en/latest/#other-minimal-dja... has a list of a few options, but the most recent is https://github.com/pauloxnet/uDjango.

You just need a wsgi.py file:

    import os
    import django
    from django.core.wsgi import get_wsgi_application
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi')
    application = get_wsgi_application()

    ROOT_URLCONF = 'mysite.wsgi'
    SECRET_KEY = 'hello'

    def index(request):
        return django.http.HttpResponse('This is the homepage')

    def cats(request):
        return django.http.HttpResponse('This is the cats page')

    urlpatterns = [
        django.urls.path('', index),
        django.urls.path('cats', cats),
    ]

Put that file in /var/www/mysite/mysite/wsgi.py, point your webserver to that file and you are good to go. In Apache you do it like this:

    ServerName mysite.local
    WSGIPythonPath /var/www/mysite
    <VirtualHost *:80>
        WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py
        <Directory /var/www/mysite/mysite>
            <Files wsgi.py>
                Require all granted
            </Files>
        </Directory>
    </VirtualHost>

> import django

Not really in one file, is it?

If you somehow managed to shove Python and Django into https://redbean.dev/2.0.html, then it might just work as bona fide one file.

One file in your project.

'apt install sqlite3' will install more than one file too. But the DB that is in your project is just one file.

Reply on news.ycombinator.com