Dropbox-as-a-Database | Hacker News Reader

Dropbox-as-a-Database

blog.opalang.org

84 pointscedrics32 comments

It is a fun exercise to do these kinds of things. Obviously, they are very inefficient. But that is not the point! Perhaps there's some residual value related to some Dropbox feature (such as automatic replication) that makes this useful . But that is countered by getting your account rate-limited by Dropbox.

I wonder, is there some way to create a key-value store out of any physical or virtual object that can store state? I know there is a lot of work done with quantum computing to store state in quarks. I can see that has value.

But, what is the most inefficient data storage mechanism you can come up with? Here are a few I can think of:

Instagram-as-a-database - You issue a request to save your data, we encode it using some visual encoding scheme, take a picture, and then upload it somewhere. You get returned a URL, and can then decode it or update it.

MTurk-as-a-database - The possibilities are endless here. Someone could write down your request on a piece of paper and then you could send another request where they type it back.

Twilio-as-a-database - Encode your data as an audio stream. Store it in random voicemail boxes. Retrieve it later by calling up, listening to your voicemail, decoding the messages.

quantum computing to store state in quarks

Atoms or electrons, but not quarks.

Humanity as a database: after infiltrating a religious order mandate arranged marriages encoding your input data in predicted genetic mutations of offspring. Ensure the religious order is hermetic so no outside data appears and that its teaching require all members to appear at a certain location at a certain time. Retrieve your data 1000 years later by appearing at the preordained spot for the Holy Query Ceremony, and examining the eye/hair colours of those gathered. Good error correction protocols are required. (This might well already be part of a Neal Stephenson novel).

A Minecraft game would make a hilarious database. The fact that it's probably pretty trivial to CRUD it programmatically outside the Minecraft client by manipulating save files is a great bonus.

Start using the DB. If you get bored enough you can examine the data physically in-game.

Why not go full-hilarity and not touch the save files? The DB API simply performs clicks, keydowns, and mouse movements on the game client, so you see the character moving on the screen. I guess you would have to initially read the save file though, to know what you start with. Would be hilariously slow.

You could start with the Flat world generator...

I've used .json files in place of a database before on a read-only site. It worked very well. I do believe that we over-engineer some times and use databases when flat files will do just fine.

Using sqlite as the "this is just an idea" phase positions you better for scale-out/build-up if it becomes necessary. Helps keep your head straight but doesn't slow you down.

I say this as someone who worships Crockford for inventing JSON.

I've seen too many projects get fucked hard because they used flatfiles.

Edit: Seriously though, managing synchronization/atomic changes to flatfiles manually is a Bad Idea (TM).

It's somewhere up there with, "I have a performance problem. I'll use pthreads!" Now you have 1,02-SEGMENTATION FAULT problems.

Like I said, in my case there were no changes, I only read from them. So I let the server cache the hell out of them and the entire site was fast because of it.

You're not alone. Guess which famous tech website uses flat files as a database? Hint: orange header, heirarchical comments...

Is this true? Is there some post about this?

It seems rather generous to call this a database. It's files on a filesystem. The lack of any kind of transactional consistency guarantees (Dropbox is notoriously awful at this), and the lack of any sane mechanism for querying this "database" means that it's just "data", not a "database".

I think he calls it "database" because it is strongly integrated with the database syntax of the Opa language. You can declare and use this "dropbox-database" just like a Mongo one, independently of the notion of safe-ness of the database.

Consistent with the complaint leveled against my previous comment, I should think that this is really consistent with the DBMS syntax of the Opa language</pedantry>

Transactions are not a requirement for databases, and querying is the domain of the DBMS, not the database itself.

You're quite right. I think my complaint was that it reduces the definition of "database" to "anything that is used to store data", rather than the database being the json file(s) themselves, at which point we have degenerated to about the same terminological point where it would be reasonable to call saving CSV or MS-Access files into a Dropbox folder "Dropbox-as-a-Database", which seems an unhelpful label, if nothing else.

Actually I have the exact same idea like a few weeks ago. The potential for this idea is big. It can be widely used by small web apps (Chrome App). Instead of hosting a server and a database, every thing can be done and stored in the client, isn't that neat?

You should have a look at filepicker.io then

as yefrederic pointed out you could use filepicker.io to build static js applications. We handle all the server side logic for you.

Dropbox as a database is a actually great idea. It takes away scaling problems from the developer and puts the data control back to the user.

I'm actually working on NimbusBase, a product that allows you to write clientside javascript applications where there is no server code and data is stored inside of Dropbox. It also integrates with backbone.

Shameless plug: sign up for our beta! http://nimbusbase.com

Reminds me of the GMail filesystem: http://www.viksoe.dk/code/gmail.htm

Ha, I remember using this for backups before I had even heard of ftp.

"update: Heroku is down [...] the demo in now on dotCloud."

is this part of the app open-sourced? it's a great idea.

You can find the source code here : https://github.com/cedricss/server-monitor/

The list of commits is interesting: https://github.com/cedricss/server-monitor/commits/master

And also shows its really a one-day app!

http://server-monitor.herokuapp.com/ is open-source (MIT), and use the Opa framework (also open-source, MIT). The Dropbox-as-a-Database is integrated in Opa by default.

Your blog design is bad btw, popout at side overlays scroll bar in Windows/Chrome.

It's one of the design proposed by Google, blame them :)

Google isn't the one using it to demonstrate their work as a developer ;-)

Plus, i can barely read the text.

Reply on news.ycombinator.com