Hacker News Reader

Comment by fasteo | Hacker News Reader

They are impressive, but let's put some context:

  No guarantee of storage

  DalmatinerDB offers a 'best effort' on storing
  the metrics, the ingress transport is UDP and 
  there is no log for writes (there is the ZIL if 
  enabled in ZFS) or forced sync after each write.
  This means that if your network fails packets can 
  get lost, if your server crashes unwritten 
  data can be lost.

[1] https://github.com/dalmatinerdb/dalmatinerdb#no-guarantee-of...

View on HN

Actually let me put that paragraph fully in context with the reality.

UDP is no longer used (this is outdated sorry for that), the connection is TCP now as it turned out over all the performance was better.

Dataloss can still occur since DalmatinerDB keeps a cache (which other metric stores might also do). A lot of that can be mitigated by using N=2 (or 3) to store data in multiple nodes that will reduce the chance of dataloss significantly. Keeping caches isn't uncommon however, to ensure full consistency it requires a kind of transaction from that goes from client to server to client, which is 'really' costly and I am convinced not worth it for metrics, a few seconds of lost metrics doesn't warrant the cost of that.

The tests were run with TCP, they do not include non stored metrics.

Reply on news.ycombinator.com