Hacker News Reader

Comment by peterwwillis | Hacker News Reader

if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i think that means basically implementing the equivalent of an 'intelligent' distributed parallel fault-tolerant file system. at that point i figure you could add "lots of glue" (heh, understatement?) to MemcacheDB, MemcacheD, MemcacheQ and some other crap and get something similar. but i could be crazy.

View on HN

http://ostatic.com/blog/a-bright-future-for-drizzle

I guess this is old news but this is basically a much moreand flexible form of what I was thinking of (I was only considering the infrastructure/fault-tolerant portion of an RDBMS). You could definitely design a well-scaling horizontal cluster using this design; even if the stock components don't support what you need you just modify stuff outside the microkernel.

The google appengine datastore is moving in the direction of being a full sql database, and I bet oracle have people working on it (because appengine, salesforce and aws is a threat to their business), so I don't think it's all that far-fetched.

Appengine's GQL is a very limited subset of SQL. The only queries which can be performed are those that can scale well, so it's doubtful that even simple things like JOINs will be supported in the future.

http://code.google.com/appengine/docs/python/datastore/gqlre...

Appengine datastore works by creating indexes for every query. In relational databases you make joins execute fast by creating indexes. I don't see why joins couldn't be added to appengine.

I believe there's a few reasons. one is that the google file system blocks that bigtable runs on and "reads" for your query could be on any number of machines. been a while since I read the google architecture papers but I would recommend it to all. it is fascinating how they implemented gfs, bigtable, chubby lock, etc. the work around is essentially to write your own code that does what a database does - something to the effect of getting a list of keys for the records you are interested in and using that to then scan just the area in your data structure you want.

It hasn't got anything to do with that. Creating an index on a join means that possibly hundreds of indexes need to be updated every time a you make a write - but that is where appengine would actually shine since those writes can be parallelized to hundreds of machines. It could easily make postgres/mysql/oracle redundant.

Reply on news.ycombinator.com