Hacker News Reader

Comment by thristian | Hacker News Reader

One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file, which isn't the most helpful result.

Since SQLite aims to be a generic way for applications to store data, it'd be nice if its "open database" and "create database" functions allowed you to supply, say, a 4- or 8-byte magic value that would be stored at a fixed location in the file's header so tools could distinguish your SQLite-based file-format from everybody else's, without having to load the thing up and examine the schema.

View on HN

There are currently 24 contiguous bytes of unused space in the SQLite header. If need be, and if SQLite catches on for use as a portable image format, I will be willing to allocate some or all of those 24 bytes to an identifier string for file(1).

This is surely one of the greatest responses I've ever seen on the internet. What was the intention with the unused space originally? Did you foresee this moment coming?

When designing a file format, it is always a good idea to plan for enhancements. There were originally 36 bytes of unused space in the 100-byte header of the SQLite 3.0.0 file format, back in 2004 - bytes set aside specifically to deal with unforeseen needs. Over the years, 12 of those bytes have been allocated to various improvements. 24 bytes remain. (I'd prefer not to use them up all at once or on a whim, obviously.)

Thank heaven for people like you.

First, thanks so much for making what is one of the most awesome database tools around.

If you would be willing, what's your opinion on using SQLite as an image format like this? The "When to Use" page specifically sites application file formats as a good use, but would your header byte allocation be done begrudgingly or contentedly?

Who are you to be able to speak on behalf of the SQLite developers? Anyone could be behind that username.

What an awesome response.

I'm not convinced this is actually a good idea overall, but...

There's no reason `file` couldn't be made to recognize special cases of SQLite databases. It certainly handles far uglier cases already.

And SQLite already provides a place to store meta-data about a table: http://www.sqlite.org/pragma.html#pragma_user_version Though the bummer is user_version is only 32 bits.

>One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file

If the format catches on, they wont. For one, it would have it's own fixed file extension, and second, file(1) will be updated to perform and extra check even if something initially looks like a common SQLite database file.

There's already a similar situation for files like ODT, which are actually just a zip file with specific contents. But maybe the zip format has more flexibility to stick magic values in than sqlite does.

why not reserve a specially named table with the format identifier in there that file(1) could poke into and take a look at?

Reply on news.ycombinator.com