Hacker News Reader

Comment by bk146 | Hacker News Reader

Curious what Datadog is going to build with this tech. Believe this company pitching themselves as OSS competitor to Datadog a few months ago.

View on HN

Based on the [press release](https://www.datadoghq.com/blog/datadog-acquires-quickwit/) it looks like they're hinting towards offering a 'self-hosted' model for customers that can't use pure SaaS solutions due to regulations:

> Organizations in financial services, insurance, healthcare, and other regulated industries must meet stringent data residency, privacy, and regulatory requirements while maintaining full visibility into their systems. This becomes challenging when logs need to remain at rest in customers’ environments or specific regions, hindering teams’ ability to attain seamless observability and insight. To help our customers meet these requirements without sacrificing visibility or introducing multiple logging tools, we are pleased to announce that Quickwit—a popular open source distributed search engine—is joining Datadog.

>it looks like they're hinting towards offering a 'self-hosted' model

That makes sense. Datadog has been pure SaaS the whole time, which is unusual. Buying a good db engine like Quickwit would be a smart head-start into the on-prem segment which is a natural expansion opportunity.

I've previously made the prediction that Datadog is the new Cisco - can expect lots of acquisitions to be made going forward.

We switched from Datadog to Grafana (do not recommend unless they got you over a barrel on pricing and you need to escape) and one nice thing Grafana gives you is the ability to self-host for local development so you can even run integration tests against your observability... an edge case need but if you need it you're glad it has it.

I work at Grafana. Can you say more about what specifically you don’t recommend?

Not OP, but looked at doing grafana self hosted for similar reasons. The tooling is too spread out across different installables, the common golden signals and other monitoring metrics have a high learning curve / cliff more like it, and there's not good enough documentation to cover the user from "I want to do a synthetics test on a service to see if it is alive and show the results of that test in a graph"...a journey like that involves 4+ different tools.

DD was just easier to use for everybody, has lots of useful baked-in things we liked to use (apdex scores), and was intuitive enough that non-devs could design their own dashboards. We also found it way easier to collect metrics, traces, and other things.

FWIW none of these things are insurmountable and I suspect you'll eventually reach parity. Datadog lost our business for two reasons 1) Lack of billing transparency and 2) an incompetent account rep who managed to piss off our finance department while also embarrassing our CTO. And to be clear, while we aren't a "whale" our spend was over $6M/yr with Datadog - and the CTO along with the rest of eng leadership were all huge DD fanboys and yet they still managed to burn that bridge to the point where we'll never go back.

I’ve historically been a pretty big fan of Grafana, I’ve advocated for the cloud solution at more than one company.

But it seems like business development has utterly hijacked the experience.

The flow you want out of the box is Prometheus, Loki, and Tempo with one button that drops you the config for grafana-agent (now alloy which seems good technically but brings a whole new config language with some truly insane discoverability problems) that makes graphs on screens, you build up from there.

But these days everything is some complicated co-sell, up-sell, click farming hedge maze through 90 kinds of cloud vendor rip-off half baked thing.

Graphs, logs, traces out of the box. Put all the works with Snowflake shit behind an icon. A small one.

A larger bill for enterprise customers.

arguably they acquired them so they dont build anything further with this tech, as it threatens renewals of their existing enterprise contracts

Pretty clear they want it to keep a moat on their side. Can't see Datadog continuing investing in this - it's a pretty direct competitor.

What happened to Vector the last opensource they bought? Are they still hired?

(disclaimer: Datadog employee)

I haven't kept super close tabs on it but last year we were hiring for a role to do tech lead stuff and OSS community building for Vector, and yes several of the original Vector employees still work here.

(Disclaimer: also datadog employee)

I joined Datadog after the Vector acquisition and now currently am the manager for the Community Open Source Engineering team that works on Vector open source.

Just confirming strongly what ripley12 said, as a person with direct involvement in OSS at Datadog.

I could only imagine they want to replace their existing infra with this as a potential cost saving measure.

And then I'm sure they'll pass the savings onto customers given the current crazy high prices.

The architecture of Husky seems fairly similar to quickwit. https://www.datadoghq.com/blog/engineering/introducing-husky... https://quickwit.io/docs/main-branch/overview/architecture

It’s an ourobouros of influences: the Husky creators left DD to make Warpstream, Warpstream inspired Quickwit, and now Quickwit is joining DD.

and Confluent acquired WarpStream

No. Quickwit was founded well before Warpstream and it did not inspire us.

The Husky blog post was released after we released a few versions of quickwit if I recall correctly. It was not an inspiration either.

As far as I know, the similarities are fortuitous.

they are planning to allow you to run your logs in your own datacenter/cloud and put something like a proxy there or being built into quickwit that your logs show up in the datadog UI

My guess is you will be billed per gig or something but not nearly the cost of shipping your logs to DD

Reply on news.ycombinator.com