Confluent S-1
sec.gov
sec.gov
> Software developers, including those within our customers’ IT departments, are often familiar with our underlying technology and value proposition. We rely on their continued adoption of our offering to evangelize on our behalf within their organizations and increase reach and mindshare within the developer community.
> Actions that we have taken in the past or may take in the future with respect to Apache Kafka or our community license, including the development and growth of our proprietary offering, may be perceived negatively by the developer community and harm our reputation.
They know that there's a fine line between making enough open source to be seen as 'really' open source software. The 'open core' model is sometimes abused where they intentionally cripple the open-source version to prod people towards the closed one.
Also confirms what I have heard from a lot of smaller developer-tools companies - 'traditional' marketing and sales doesn't really work, and they go to pretty big efforts to get support from the bottom up, instead of taking the CIO to a fancy conference.
Going to the landing page of any enterprise B2B Saas company is always bizarre. You can tell nobody cares if the copy makes sense because these types of products get sold in person.
They also contribute to Kafka and manages various language specific drivers.
They sell proprietary Apache Kafka add-ons for enterprise such as authentication/authorisation RBAC. The open source Kafka release supports auth/authz but not consistently between the suite of tools. They also have a monitoring UI etc.
They release the commercial add-ons under the Confluent platform distribution.
They heavily push ksqldb which is a sql layer on top of Kafka that competes with https://lenses.io/. Confluent market it as a DB but that’s pushing the creative marketing.
They provide a hosted Kafka service.
The hosted service is likely more attractive to SMEs. It also has competition with the SME market with AWS providing managed Kafka. Confluents service does have extras on top of AWS like tiered storage behind a non open source license but nothing is stopping any one building there own tiered storage. I can see people using Confluent’s cloud but don’t see it being big.
Large corporations will (have) generally hosted/run their own Kafka.
I like Kafka but can’t see how Confluent can be profitable.
Cloudera is an example, they couldn’t make the service business work, I feel Confluent will have the same issues.
Elastic search have issues with AWS eating some of the market. Confluent have the exact same issues.
The Kafka is your nerve center narrative, that is great from Confluents perspective financially but it’s no different than the old enterprise service bus as the nerve center for everything. There’s a reason we don’t see that anymore.
I've seen what some corporate customers pay for an enterprise license. It's kind of insane.
"As of March 31, 2021, we had 561 customers with $100,000 or greater in annual recurring revenue, or ARR, across a wide range of industries, compared to 374 such customers as of March 31, 2020, representing year-over-year growth of 50%. As of March 31, 2021, we had 60 customers with $1.0 million or greater in ARR, compared to 33 such customers as of March 31, 2020, representing year-over-year growth of 82%."
PrivateLink allows for shared VPC and hypothetically a shared k8s. So the cost structure can be better on the Confluent side. Of course scaling down instance types would make a huge difference, but I don't think that's supported currently. The cheaper option is multi-tenant Kafka clusters.
In general, I think VPC peering is a dead end for use cases where you are connecting two companies networks together. It's not great operationally and puts a lot of security / filtering burden on each party.
PrivateLink is a much better option.
My company uses Kafka extensively. We hosted it ourselves but after a few Kafka meltdowns we decided to go with a hosted solution. We compared AWS's offering to Confluent and at the time found that while Confluent had all of the nice tooling, we'd already built our own tooling to do the same thing, and rebuilding our entire infrastructure was a non-starter. So we went with AWS. It was a drop-in replacement for our own Kafka and it hasn't blown up on us since.
If we where starting fresh, Confluent would be more viable. I can't help but wish that the offerings that are part of Confluent where part of Kafka though. Kafka's lack of the tools that Confluent offers are imo what's holding it back from totally taking over the world.
It feels obvious to me that AWS will roll-out S3 backed storage and managed Kafka Connect at some point, their recent IAM / ACL integration points to a pretty active MSK team.
I completely agree in the past there were some pretty big gaps - that's why we built kPow - my feeling is those gaps are narrowing pretty significantly.
Let me know how you go with kPow! If you want a guided tour just let me know. :)
ps. we're releasing a major bump to the UI later today..
Can Kafka replace a database? We have an architect within our company that heavily pushes using Kafka for everything (as a database, a caching layer, message bus etc.,)
It excels at write-heavy workloads and, if you're already using Kafka for streaming or message passing then you can use it as a key-value store in order to avoid the extra operational burden of dealing with a distinct service for caching or storage.
But if you need a data warehouse or just a relational query engine for data at rest, or if your data volume is not too large and you're not already using kafka, you're probably better off with a regular DB.
Their cloud growth is good but their total growth (~50%) although nice isn’t so big for a company doing 236m in revenue but spending ~400m to do that at 50% growth.
Still I like their product and will keep an eye on this one.
Splunk acquired Streamlio the people bind Pulsar in 2019 so you have to ask with the Confluent IPO what are Splunk doing with Pulsar? It is looking like a missed opportunity at this point.
Pulsar is a direct replacement for Kafka. Pulsar even supports the Kafka wire protocol.
IMHO their biggest competitors are the clouders themselves (with pubsub, SQS, etc..), MQTT, ZeroMQ, and the open source Kafka (which is surprisingly easy to setup).
For me, the best idea they had was the Kafka schema registry (https://www.confluent.io/product/confluent-platform/data-com...) : it's too easy to inject bytes into kafka, but for using that stuff you need to know the format (and no, dunping a JSON string is not a good subtitute for a schema). Instead of guessing, and pulling your hair when there are changes, they provide a way to insert the data with the correct types and format.
(disclaimer I don't have shares in confluent, nor interest in buying some)
That's possible. Or perhaps AWS MSK is eating some of their lunch; I hear it's pretty good.
> open source Kafka (which is surprisingly easy to setup)
I've found the hard part isn't day 1 (you do get there quite easily these days), but day 20 when something breaks in an unexpected way. Like when we shut down a cluster to fix issues with the underlying machines and one broker only kept its config, but not its data directory. When restarting the cluster, the one broker without data was the first to be online, since it didn't have to load tens of gb from disk. That lone broker made itself a cluster of one, discovered all our topics, made itself leader for all partitions of all topics ... and when the other brokers came online and joined the cluster, they discovered the no-data leader and promptly started to sync their state to the leader's by deleting the messages they still held, resulting in three empty brokers, 100% data loss. This was a wayy pre-production system and we were happy to break it in interesting ways, but the ease with which we got it into such a state really made us treat it with a lot of respect ever since (that wasn't the only such occurrence, either). Kafka is a very powerful piece of software and big deal for the use cases it's built for, but it has some very thorny parts, and if you can have someone run it for you, you probably should give that some thought.
(disclaimer: I work for Aiven, we also provide Apache Kafka as a service)
It showed a lot in the industry that Elastic didn't have nearly that much to offer than what everyone thought once Amazon stepped into the ring and the same is apparent with Confluent.
The same similar situation seems to be emerging with their managed Kafka service. Once you're on AWS there will be incentive to move to their Firehose platforms.
Very few customers actually should be running a Kafka cluster at all and the bulk of those would be better off running Pulsar to reduce their pipeline storage costs significantly.
It'll up to 3 years for the full affect to be felt because of time for new projects, rewrite, end of life on the ES version that was last open source.
Everyone I know when they read that blog post was like, "Alright, moving forward we're not vendor locking ourselves into the license with Elastic."
On top of this most companies were going with Elastic for the enterprise features but those are now becoming free with AWS's version.