maxthegeek1

We use HyperDX for our observability! We had been using google's observability suite before, because we're using GKE anyways, but HyperDX's search over traces is just waaaay better and I can't go back.

I'll take a look!

Hi z3ugma! Yah we'll add a README. To clarify the emailo package is not (currently) intended for consumption outside of Dittofeed, but it'd be great to document nonetheless!

Likewise we use typebox for everything in our project (open source customer.io alternative).

https://github.com/dittofeed/dittofeed/blob/main/packages/is...

I just wanted to share a piece I recently wrote on Dev.to about some of the work we've been doing at Dittofeed. Dittofeed is an open-source alternative to platforms like customer.io, Klaviyo, and Braze.

If you're interested in how to segment users by processing semi-structured event logs at scale, keep reading!

Article: https://dev.to/dittofeed-max/how-we-stopped-our-clickhouse-d... Dittofeed's primary repo: https://github.com/dittofeed/dittofeed

This article includes code samples, as well as a link to a git repo you can use to run tests and follow along. These code samples demonstrate a simplified user segmentation setup in ClickHouse.

Accompanying repo for the article: https://github.com/dittofeed/clickhouse-segments-tutorial

User segmentation is key to understanding your business or application’s users’ behaviors. For example, you might want to keep track of which users have gotten stuck in an onboarding flow or which users haven’t logged in within the last 30 days.

Here's a quick rundown of what I covered:

Diving into ClickHouse: Why we (and you) should use ClickHouse for your analytical workloads, and user segmentation in particular.

On User Segmentation: What are user segments, and how do we calculate them. We cover some basic pseudocode.

Tackling Idempotency: To avoid duplicate data messing up our calculations, we've made sure our setup is idempotent.

Scaling with Micro-Batching: Simulate stream processing by incrementally aggregating user events in small batches. This is really the core of the article!

"Event Time" vs. "Processing Time": When did an event happen and what are the different ways we can measure time? How do we accommodate different measures of time in our application?

I'd really love it if you could take a moment to check out the article and maybe even drop a star on the Dittofeed GitHub repo. Your support and feedback would mean a lot!

Catch you all soon!

Interesting that they seem to only intend this for uses cases outside of the "path of a user-interaction". Makes me thing they've prioritized devx/iteration, but that for a non-trivial number of uses cases you'll need eject to more low level infrastructure.

Having trouble finding blueprint's repo. The paper says it's open source. Would love to poke around, if you know where to find it!

Hi all. Have you used reverse ETL with customer engagement platforms / marketing automation in the past?

Would love to learn about your use case and pain points.

k3s is my preferred way to run k8s locally, for the purpose of developing helm charts locally. It's way more lightweight than minikube, and with k3d it's way easier to use as well.

It's handled on the Twilio side, but we have webhooks to listen in on twilio subscription changes!

Cockroachdb is a common solution here.

Hey, love what you guys have done with React Email <3

Hey, great idea! We should probably make our internal onboarding journeys public like you’re suggesting so that self hosting users can benefit from them :)

Great idea :)

Thanks for the kind words! Would love to have you in our Slack. We can help you get set up.

https://join.slack.com/t/dittofeed-community/shared_invite/z...

What alternatives are you looking at?

We plan to support plain SMTP as well, so that users can be self hosted end to end if they want.

Thank you! The main difference is that intend to support multiple communication channels include mobile push, sms etc

Great idea!

Hello! We're not an email service provider ourselves (where one would normally configure email domain/ip settings), but we allow users integrate their own ESP accounts. We currently support sendgrid, and in the near term future we plan to support AWS SES.

Wow this is a really cool idea. I feel like in the future we're going to have more of this kind of stuff, and move away from the idea that product engineering, and infrastructure engineering can be cleanly separated.

Infrastructural concerns are going to have to be embedded deeply into product development and vice versa.

Really cool. I wonder how much of this is going to make its way into GKE for us plebs.

Congrats on the launch! Elasticsearch/Splunk/Datadog, are expensive beasts. It's great to see an opensource alternative!

Thank you. :')

Great question. Hasn't been requested yet, but I'm sure we'll need to support arm at some point.

It's honestly gratifying to see that we landed on a similar product space independently. I read it as a strong signal that there's a real need for the product. I've been impressed by how much ground they've been able to cover, and how quickly they're building.

Beyond that, I think our focuses are somewhat different, as our current roadmap is focused on serving growth engineers. Still happy to see them working in the space.

Unfortunately, we don't plan on supporting caprover. We do plan on supporting render, and providing a helm chart in the near future.

Currently, we're just hosting temporal ourselves in our k8s cluster. However, I've been working with their team, and we'll likely be migrating our temporal cluster over to their cloud in the near term future.

Thank you, and you raise some great points.

Re-"developer as the customer", the mismatch between who makes the buying decisions for this kind of tech (CMO's etc.), and the devs who do much of the heavy lifting to make it work, is a real challenge.

Also, your point re the economies of scale of self-hosting vs using saas is valid. For small to mid sized orgs, using a cloud offering can be more economical.

However, we've observed that larger orgs often migrate off of saas to use open-source or build software in-house. This occurs for a number of reasons:

- They will have more engineering resources to allocate, in this case to marketing / growth.

- At their scale, the fixed costs of allocating engineers to implementing solutions are often exceeded by the variable costs of saas products, which commonly have volume based pricing.

- They often have more unique requirements that are not served by any particular saas product, and closed source saas is not extensible.

Some recent examples of this:

- Several large orgs are migrating off datadog in favor of open source observability tooling.

- Airbnb recently implemented the equivalent of Dittofeed internally (they responded in this thread).

Still, you raise legitimate concerns, and we're still figuring things out. Would love to get in touch, to better understand your perspective if you have time!

Thanks! Definitely, migration guides are coming. In the meantime, feel free to ping us on our community slack and we'll walk you through a migration process.