Hacker News Reader

Comment by xtacy | Hacker News Reader

I don't understand how they could avoid locks. They say that each replica runs independently and hence that avoids locking; what about write updates to the same object from different clients? That would need locking of some form at some level.

They should also explicitly discuss the failure models for which they're guaranteeing Durability. If you're scaling to "web-scale", and running this db in a data-centre, then, a single failure would wipe out a rack of machines. Coordinated failures are not uncommon in data-centres. What about those?

View on HN

You avoid locks by serializing transactions at any site. Since you're not waiting on disk (in memory DB) and each partition runs on its own block of memory and has its own cpu/thread, you simply don't let two transactions on the same partition run concurrently.

See http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf for cases where you want to run two transactions in the same location concurrently in h-store (the academic precursor to VoltDB).

I see; what about cases where transactions involve objects at multiple servers? I guess I am confusing server specific locks vs client specific locks.

In that case there will need to be some kind of locking, which is probably the reason the whitepaper recommends avoiding transactions that span multiple servers: "For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. "

I had an idea for handling this situation that exploited the replication functionality. Basically, you prepare for a transaction by migrating the primary copy of all the objects involved to the same node. One would have to avoid deadlocks by sorting and serializing transaction-prep migration blocks. So transactions never span multiple servers, but you get this by possibly dramatically slowing down the time it takes to prepare for such transactions. (With the idea that the application designer is encouraged to avoid such transactions.)

If you do this for every transaction, you'll have more overhead than you'd have if you had just performed distributed consensus. But in principle you're right. It makes sense to put all the primary copies that are often used together on the same node. The problem is that it's hard to decide the ideal placement strategy. VoltDB puts this responsibility into the hands of the developers.

The point is to a) not have to implement distributed consensus and b) require the user to have to design the database for localized transactions if performance is desired.

It seems that it's single-threaded, so all writes will be serialized, hence no locking.

From their whitepaper:

"Conventional databases experience disk and user stalls within transactions. Rather than let the CPU be idle during the stalls, those DBMSs interleave SQL execution from multiple transactions during the waits so the CPU is always busy. This is what requires much of the complex latching and locking overhead.

VoltDB doesn’t experience user stalls (since transactions happen within stored procedures) or disk stalls (because VoltDB processes data in main memory). Therefore, it is able to eliminate the overhead associated with multi-threading (latching) and locking. Each VoltDB execution engine is single-threaded and contains a queue of transaction requests, which it executes sequentially—and exclusively—against its data. Elimination of stalls and the need for locking and latching overhead and allows typical VoltDB SQL operations to complete in microseconds.

For single-partition transactions, each VoltDB engine operates autonomously. For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. Many common applications such as order-fulfillment, software as a service (SaaS), Web 2.0 and trading systems have this property."

http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...

That still doesn't completely answer the question how it avoids race conditions and the likes without locking. Assume I'm executing multiple long-running transactions, each hitting and modifying a lot (millions) of different objects. How are race conditions avoided in this case ? One engine that distributes and coordinates work plans for the other engines sounds nice, but does this essentially mean that only one of these transactions is able to run at the same time, even when they could be running concurrently when using locking?

VoltDB is designed for short transactions that each touch one row, so you should expect pretty poor performance for long-running transactions.

It's Prevayler all over again!

http://www.prevayler.org/

(Not really. Prevayler uses a log.)

SQL doesn't scale didn't they get the memo...lol

Reply on news.ycombinator.com