Vitess: Scaling MySQL databases for the web

I'm glad to see this released because it is great technology that showcases Go's strengths. Here's why they chose Go, in their own words:

"Go is miles ahead of C++ and Java in terms of expressibility and close in terms of performance. It is also relatively simple and has a straightforward interaction with Linux system calls."

It has always frustrating to say "We use Go at Google, I just can't say how." So I'm happy that now I can say that YouTube use Go in their core infrastructure, serving more than a billion requests per day with this service alone.

A quick glance at the sources shows immediately what Go offers per se in terms of readability and maintainability. Go strongly encourages, even forces at compile time, certain style constraints that results in such a delightful code to explore. Were it written in C, you'd have to deal first with arcane #whatever directives in .h and .c files and strange makefiles and such stuff.

I hope someday Go will get the acknowledgement it deserves as a real piece of genius. I really encourage recommend just to take a 15-min try at http://tour.golang.org.

Interesting comments (in mysql.go) :

    // Muahahaha
    func strlen(str *C.char) int {
            gstr := (*[1 << 30]byte)(unsafe.Pointer(str))
            length := 0
            for gstr[length] != 0 {
                    length++
            }
            return length
    }

BTW, is Google planning to make a proper mysql driver in go (implementing database/sql API and not unsafe) ?

This project looks awesome and really points out some of the major weaknesses with in mySQL/libmysql-

  * Impossible/difficult to gracefully handle connection/query timeouts.
  * Extremely poor query/row cache (the table's entire query cache is flushed anytime a write is made in the table)
  * Poor connection memory usage- Each open connection sucks up ~10MB of memory

How does this help "scale" databases?

It just looks like a connection pooler with a RPC interface?

Am I missing a feature?

At a glance this seems fantastic. I have an application where the database recently started to get overloaded at peak usage times.

It seems like this could be dropped in as a replacement with very little code re-factoring if the client library is similar to the native mysql client lib. (or an adapter could be written)

Does anybody know what would be involved in getting this working with PHP?

It is rather unclear to me whether or not Vtocc automagically takes care of sharding and replication.

What I don't understand is why you would use MySQL as your storage engine it seems to me you are stripping away most of the benefits of using MySQL by interfacing it with this library. Is there a compelling reason to not use google's own LevelDB?