DeepDB – High-performing transactional and analytic database | Hacker News Reader

DeepDB – High-performing transactional and analytic database

deep.is

34 pointsmisframer28 comments

So the team consists of ( http://deep.is/our-invention/ ):

  1 CEO
  1 CTO
  3 Founders
  1 Chairman
  1 VP Product Management
  1 VP Sales
  1 Controller
  1 Director of Engineering
  1 Architect
  1 Consultant
  2 Engineers

Yeah... sounds like exactly the company i would never buy from. A company which basically consistes of 80% "upper management" positions and barely any "real workers".. Especially when the website is only about marketing claims and very unspecific details. I can't even find downloads or documenation..

No, i'll stay away.

Those bios are a bit strange. They all manage to fill multiple paragraphs without actually saying anything relevant about themselves. Instead it's just strings of empty phrases like "I make magic happen" or "my DNA pushes me to always seek out the vanguard of technology". (Your DNA, really? Did one of your ancestors invent fire?)

To read the real bios, you need to click through to the people's LinkedIn profiles. This reveals that the founders come from companies like Oracle. I don't know if that explains anything.

CTO is a professor at UNH, Chief Scientist is from UNH and used to work at Virtual Iron which was acquired by Oracle, then he worked at Akiban until they were acquired. It appears some of their patents (http://www.google.com/patents/US20130290243, http://www.google.com/patents/US20130254208, http://www.google.com/patents/US20130226931) were assigned to Cloudtree Inc., which appears to have originally been something totally different but with what seems to be the same team: http://www.boston.com/business/technology/innoeco/2011/02/st...

I'm puzzled.

VirtualIron was a huge disaster of a company - they pivoted multiple times without ever finding market fit, then pissed off potential acquirers with their attitude, after which they laid everyone off and sold the remains to Oracle for a pittance ( I remember hearing it was ~$10m).

http://bits.blogs.nytimes.com/2009/05/22/with-virtual-iron-o...

Virtually outsourced engineering as well -- a separate "corporate office" and "engineering office" in two different states, with the business people and founders working at the corporate one.

Those kind of companies nearly always fail. Engineering feels like the engine room on the Titanic, C levels escaping with lucre while the boilers are stoked.

You need all of the senior folks to manage the two engineers and tell them what to do and subtly deflect criticism towards them when things go wrong. Is it not the case that the business folks are entitled to and have the right to the fruits of the engineers' hard work?

To be fair, all of their team except one come from technical backgrounds. There are three other engineers (Michael Jeffords - CTO's relative?, Vincent King and Richard Soczewinski) listed as employees on LinkedIn. That puts the split at 10:6, which looks healthy. I'd be more worried about the lack of information and the fairy tales on the product's website.

10:6 is healthy in your world? Ok.....

That does not mean 6 managers for 10 developers. That would be ridiculous.

If I can't download at least a demo, let alone the source it will take a lot of convincing to get me to use something.

Not to be too critical since I like database startups as much as the next guy, but there is almost nothing original in their approach to database engines. Most of their assertions about existing databases only apply to relatively weak open source databases; on the high-end databases don't work the way they are assuming at all, which makes me wonder how much they actually know about database engine internals generally. Architecturally, DeepDB goes into the same bucket as PostgreSQL in terms of capabilities and scalability, albeit a different design.

As a more technical nitpick, "high availability" does not mean fast restarts. It means never goes down. The design as described is not a high availability architecture.

Bonus observation: if you assert your database engines uses extensive fine-grained locking, don't show a chart with 24 cores at ~100% utilization and call it "efficient". This is what you would expect to see for a poorly designed and very inefficient implementation for this type of architecture. You can be burning all of your CPU and getting very little throughput. A credible presentation would have demonstrated that system throughput scales linearly with the number of cores (unlikely given the description of their internals).

Seems to be targeted as a replacement for InnoDB, slides with actual "meat" on them => https://portland2013.drupal.org/sites/default/files/slides/B...

At http://deep.is/deepdb-genesis-of-invention-iii/, the chief scientist of this company credits the following four "axioms" for his "General Theory of Information":

Axiom 1: Information is a sequence of Information (self-similar) that is segmented by consistent (well-formed) order.

Axiom 2: Segmented Information is addressable by the First where a sequence of Firsts is a Segment of Summarization.

Axiom 3: Information of a Segment is equal distance to the sequence of Information between non-associated Summarizations.

Axiom 4: Sequenced Information is in direction relation to former and later Information of which Patterns can be Matched.

I don't know what any of that means. Worse, it sounds to me like "cargo-cult" computer science.

I also don't know what any of it means, but it certainly looks impressive with all these uppercase letters.

I read I and II in hope that some of those terms would be defined. I came to the conclusion that they have reinvented the turbo encabulator.

> A NEW GENERAL PURPOSE DATABASE DESIGNED FOR BIG DATA AND THE CLOUD THAT PERFORMS SIMULTANEOUS TRANSACTIONS AND ANALYTICS … IN REAL-TIME, ON THE SAME DATA SET

Well they certainly have all the right buzzwords present in their pitch. Other than that there's very little to go on right now as to how much merit their product deserves.

See @buster's comment. 100 "managers" and 2 engineers, what can go wrong?

It amazes me that HN users would spend time commenting here before taking a few seconds to navigate a website. Here is an overview of technical information: http://deep.is/knowledge/deepdb-white-paper/

These are the interesting things that sticks out to me:

The understanding that storage system throughput is maximized by using sequential access patterns led to the creation of streaming, append-only transactional and indexing algorithms. Our approach is unique in that all database files (i.e. transactional state, indexes, and metadata) are streamed, append-only files.

...

The CASI Tree breaks from a traditional b-tree on-disk approach, eliminating update in-place operations on fixed size pages. Instead, the CASI Tree is an append-only Purely Functional Tree data structure.

...

A complete database audit trail is maintained when running in archival mode, making all previous database states available. These states may be queried in read-only mode, efficiently supporting read-only analytics of point-in-time transactional database states

This links to the wrong fractal tree paper for discussing write-read optimized data structures. The proper one is http://supertech.csail.mit.edu/papers/sbtree.pdf

This makes me wonder how much research this team really did. I'm not ready to claim they haven't done it, but it raises the question. I would like to see the actual data structure published somewhere.

One of the things I was left puzzled with was how could one efficiently leverage a transaction log and data organization for querying all in the same file without memory mapping and have this perform well. Maybe the implementation they are using is part of DeepDb special sauce? Also it seems the company is very young and so many of the things other posters are expecting such as a large engineering team, feature matrixes and downloadable demos arent in place yet and will be soon

Perhaps I am missing something, but color me skeptical... immutable on-disk structures essentially means that you have to perform run-time compactions on every query... unless your in-memory representation is the cache, in which case you have to worry about durability.

Those sentences don't make any sense. This whole thing looks like it comes out of some kind of vaporware start-up generator.

Experience has taught me to be naturally skeptical when a startup says they designed a database from the ground up. It seems like they put every buzzword of machine learning and database design into their website, and the staff seems to be primarily management people and only a handful of engineering staff.

But who knows, maybe in 5 years we'll be doing our business analytics with DeepDB, but most likely we'll be doing business analytics with a tool that doesn't seem this sketchy.

No download. No trial. No install doc. No example doc. No tutorial. Just lots and lots of assertions of greatness. That's not so deep.

Wow, a database actually designed for both big data and the cloud! Sounds like deep science to me!

I am having trouble locating any reports to reproducible benchmarks that back up their performance claims.

A lot of buzzwords, but little actual information. How fast is it? Is it equally fast for both OLTP and OLAP style work?

It seems to be optimising to remove the ETL step of transactional - analytical data migration, but you still need the analytical speed.

Reply on news.ycombinator.com