chtefi

Aiven, Confluent, Redpanda, StreamNative and Ververica established the Streamhouse Working Group and published an open, vendor-neutral definition of Streamhouse.

WDYT?

I work at Conduktor, so I agree with the technical building blocks being described, to form a coherent real-time architecture in a business. But right now, this feels mostly like an attempt to claim a category early rather than anything else: it's still very high-level and does not describe a concrete reference architecture.

Is this to package Kafka/streaming + Iceberg/lakehouse + catalog/governance packaged under a new name for the AI era?

I don't like seeing people market "mcp is dead" and always say the same thing, talking about context and how CLIs are better, so wrote this.

What do you think, what have you saw in terms of usage/adoption/issues?

Working on https://kapturekafka.dev, a desktop app for Kafka protocol inspection. Think Wireshark or Fiddler, but native for Kafka.

Useful to debug local Kafka apps against any cluster, intercepts the traffic, decodes the protocol. You see interesting (and weird) things when you look at the protocol. Still early, though already useful for local debugging when you know what you want.

I run a lot of small background services on my Mac (thanks Claude/Codex), and Homebrew services that I barely look at. It felt surprisingly hard to know what was actually happening on my machine and investigate/fix quickly, so I built Launchdeck: a fast TUI for launchd jobs and Homebrew services.

Mitigations mentioned:

- Make the validator read-only for the agent. Mount it as read-only in the container, or hash your eval scripts at startup and verify before each run. If the agent can write to anything in its evaluation path, it can (will) game it.

- Log the full trajectory, not just the output. Every tool call, file diff, reasoning step. Then run a second agent over the trace with no knowledge of the KPI: it only knows what honest execution looks like and will use its internal honest alignment to assess it.

- Write system prompts like job descriptions, not optimization targets. Name a reviewer. Give the agent permission to fail ("if you can't hit the target, explain why").

- Walk your own prompts: what's the metric, what can the agent write, and can it reach the metric by modifying the measurement instead of doing the work? If yes, close that path.

Wanted to build a curated list of truly free, publicly accessible real-time datasets and streaming sources to have real-life data, and see their shape quickly in the browser (WebSocket, SSE). Also led me to build kafka-connect-websocket (https://github.com/conduktor/kafka-connect-websocket) to publish into Kafka directly.

Full disaggregation of compute and storage is the right direction. Let storage handle replication, it's getting good, global, low latency, cheaper (like with S3 Express). Kafka becomes a smart data ingester and router: it moves bytes, enforces ordering, does minimal buffering. That's it. Do one thing well.

You get a system simpler to operate, to scale, and more flexible; data could be consumed outside of Kafka itself (in a batch way typically), without duplicating the data, that's a big win.

A bit specific but Nasa has some public Kafka: https://gcn.nasa.gov/docs/client

Conduktor | Senior Java Backend Engineer & Senior Product Manager | London | Full-time | Hybrid Conduktor is a data platform that sits on top of any data streaming technology (Kafka), ensuring companies across the world maximise the value of their data.

We’re hiring for two positions:

1— Senior Java Backend Engineer: We're assembling a team to build the most powerful Kafka proxy for enterprises. It's a critical part of data infrastructures. If you’re a Senior or Staff Engineer with a passion for low-level Java, threading wizardry, and networking in distributed systems, you have a chance to shape the future of data streaming at Conduktor. UK / London (on-site).

Apply here: https://jobs.lever.co/conduktor/9b11cf2b-906e-49d7-baa1-6420...

2— Senior Product Manager: We’re seeking a strategic, customer-centric Senior Product Manager to partner with Product & Engineering leadership. The focus will be on enhancing Conduktor’s capabilities in data security and observability. Ideal candidates will have experience in enterprise software, big data platforms, or observability products, and be excited about real-time streaming data technologies. A technical background is mandatory (or we won’t even look, sorry). This is a hybrid role based in London, with the team coming onsite 3 days a week.

Apply here: https://jobs.lever.co/conduktor/e4c1a6c9-29c0-48c1-aa31-4569...

Conduktor | Senior Java Backend Engineer & Senior Product Manager | London | Full-time | Hybrid

Conduktor is a platform that sits on top of any data streaming technology (Kafka), ensuring companies across the world maximise the value of their data.

We’re hiring for two exciting positions.

1— Senior Java Backend Engineer: We’re looking for an experienced Java developer with deep knowledge of Kafka. Production experience is critical; we are serious about a production mindset! This role involves a deep dive into Kafka’s low-level protocol and networking stacks. We DON’T do Spring, at all. FP experience is appreciated. This is a hybrid role based in London, with the team coming onsite 3 times a week.

2— Senior Product Manager: We’re seeking a strategic, customer-centric Senior Product Manager to partner with Product & Engineering leadership. The focus will be on enhancing Conduktor’s capabilities in data security and observability. Ideal candidates will have experience in enterprise software, big data platforms, or observability products, and be excited about real-time streaming data technologies. A technical background is mandatory (or we won’t even look, sorry). This is a hybrid role based in London, with the team coming onsite 3 times a week.

Apply below:

- Senior Java Backend Engineer: https://jobs.lever.co/conduktor/9b11cf2b-906e-49d7-baa1-6420...

- Senior Product Manager: https://jobs.lever.co/conduktor/e4c1a6c9-29c0-48c1-aa31-4569...

Excited to announce Conduktor 2.0!

We build Conduktor to improve adoption and collaborative use of Apache Kafka across teams and organizations. We are driven to make Kafka a better and safer place, allowing the business to thrive in real-time use cases and adapt proactively to build a better customer experience.

Please share your experience with Kafka, your love and/or your hate!

Sharing some experiences about co-founding a startup and how important it is to have a co-founder (at least one).

Conduktor Inc | Many Roles | Full-Remote | Full-Time | https://conduktor.io/jobs

Conduktor is helping companies operate their data by making their Apache Kafka journey easier. We’re an international team (mostly in Europe) looking for highly motivated, tech/data-oriented people. Today, we are a developer tool, tomorrow, we'll be something way bigger! We're looking for people around the CEST timezone:

* Experienced Scala/ZIO & Kafka developers

* Typescript & React developers

* Product Manager to lead our products

* Developer Advocate if you know your Apache Kafka stuff to help us working with our community!

* Customer Engineers to help our customers working with Conduktor & Kafka!

Take your chance: jobs@conduktor.io

Conduktor Inc. | Scala/Kafka Software Engineer Roles | EU/US/World | Full-Time | FULL REMOTE

Conduktor, a desktop application, a developer tool, exists to help developers, operators, QA, enterprises, working with Apache Kafka to ensure their system and data are safe. We want to make working with Apache Kafka as easy as possible and streamline all the processes thanks to the different softwares we want to build around it. We don't host any Apache Kafka ourselves, our customers have their own solution. We are building smart and beautiful interfaces for our users to "forget" about the internal slippery details of Apache Kafka, and let them trust us.

We're a small team for now (founders, a few engineers, a few sales), constantly looking for new people to help us. We're currently hiring mostly in EU timezone, but are also open to US/Canada (note: most of us are french-speaking, hence a french culture).

Our stack: Kotlin, Scala, ZIO, React/TypeScript, Google Cloud, and you must know Apache Kafka

- We don't host any Apache Kafka cluster, this is not a Data Engineer or DevOps position. - We're using ZIO in Scala, we are adept of FP principles and love it.

Kafka & Scala Software Engineer | Full Remote

If you're interested, apply on our AngelList jobs page, here: https://angel.co/company/conduktor/jobs