IBM to acquire Confluent
confluent.io
confluent.io
Ok, so does anyone remember 'Watson'? It was the chatgpt before chatgpt. they built it in house. Why didn't they compete with OpenAI like Google and Anthropic are doing, with in-house tools? They have a mature PowerPC (Power9+? now?)setup, lots of talent to make ML/LLMs work and lots of existing investment in datacenters and getting GPU-intense workloads going.
I don't disagree that this acquisition is good strategy, I'm just fascinated (Schadenfreude?) to witness the demise of confluent now. I think economists should study this, it might help avert larger problems.
It really is probably that strangest company in tech which you think could be mysterious and intriguing. But no one cares. It’s like no one wants to look behind the boring suit and see wtf. From my low point on that bell curve I can’t see how they are even solvent.
I'll believe that when I see it. They had a decade headstart with all of this, and yeah, could have been at the forefront. But they're not, and because of the organization itself, they're unlikely to have a shot at even getting close to there. Seems they know this themselves too, as they're targeting the lower end of the market now with their Granite models, rather than shooting for the stars and missing, like they've done countless of times before.
Edit: you also seem to be giving too much credence to Watson. Watson was actually mostly a marketing tool designed to win in Jeopardy and nothing else. It was constructed specifically to compete in that use-case and was nowhere near to the architecture of a general transformer which is capable of figuring out meta-patterns within language and structurally understanding language. You can read about Watson's design and architecture here if you're curious: https://www.cs.cornell.edu/courses/cs4740/2011sp/papers/AIMa...
Well, in Confluent's case I'm not so sure that's true given that their CEO is also the company founder as well as one of the original authors of Apache Kafka.
IBM has a ton of Enterprise software, backed by a bunch of consultants hiding in boring businesses/governments.
They also do a ton of outsourcing work where they will be big enterprise IT support desk and various other functions. In fact, that side has gotten so big, IBM now has more employees in India in then any other country.
Leadership in IBM also thought that Watson was like what what OAI/Anthropic/Google are doing now. It wasn't. Watson was essentially a ML pipeline over-optimized on Jeopardy, which is why it failed in literally every other domain.
Outside of Jeopardy, Watson was just a brand.
For my package on one VERSION alone: https://gitlab.com/redhat/centos-stream/src/kernel/centos-st...
I dont know if you were trying to be funny, or simply dont understand how much change really goes on.
Athough i think they just di/dont know how to adapt these to market that isnt a enterprise behemoth , rather than develop/price it so more devs can take a hold and experiment.
I do. I remember going to a chat once where they wanted to get people on-board in using it. It was 90 minutes of hot air. They "showed" how Watson worked and how to implement things, and I think every single person in the room knew they were full of it. Imagine we were all engineers and there were no questions at the end.
Comparing Watson to LLMs is like comparing a rock to an AIM-9 Sidewinder.
I do remember they tried to sell it - at least in the meeting I went - as a general purpose chatbot.
I did try briefly to understand how to use it, but the documentation was horrendous (As in, "totally devoid of any technical information")
Unfortunately, the way it solved fuzzy was 'engineer the problem to fit Watson, then engineer the output to be usable.'
Which required every project to be a huge custom implementation lift. Similar to early Palantir.
> Unfortunately, the way it solved fuzzy was 'engineer the problem to fit Watson, then engineer the output to be usable.'
I'm going to review my understanding of fuzzy optimization because this last line doesn't fit the bill in it.
In the Watson era, all the front- and back-ends had to be custom engineered per use case. Read, huge IBM services implementation projects that the company bungled more often than not.
Which is where the Palantir comparison is apt (and differs). Palantir understood their product was the product, and implementation was a necessary evil, to be engineered away ASAP.
To IBM, implementation revenue was the only reason to have a product.
Well this is _not_ what they wanted to sell in that talk.
But the implementation shown was über vanilla, and once I got home the documentation was close to un existent (Or, at least, not even trying to be what the docs for such a technology should be).
- the pure consultancy is another company now - the IBM portfolio of software "products" are being packaged in ways that emphasize professional services and elaborate licensing schemes (rather than turnkey software)
> Everything will make sense when you realize that IBM is a consulting company.
This and.
The 'and' being that consulting companies, in their DNA, build solutions for their customers.
Which is a very different business than building products for all users.
Not least because the former is guided by understanding a customer's requirements, while the later is having a strong intuition (backed up by market fit) about what all users want.
I'm pretty sure there might not be a full end user capable (in the sense of design-build-iterate) product team in IBM at this point.
Mostly because I don't think they've any middle/upper management that can think that way. They've got the engineers!
Same thing happened with their cloud offering. They laughed at AWS, then tried to catch up, then missed and pivoted to “hybrid” (cloud and local).
I worked with IBMers. The main priority for a lot of them is to ensure continuous employment for themselves and their buddies. They'd add unnecessary complexity to a product to stretch out the development for another couple of years. And they work at leisure pace for tech. Actual 9 to 5, many coffee breaks. They can't compete.
Ultra-based. We should all be so lucky.
we are achieving true egalitarianism here, everyone can be a slave regardless of color, creed, or origin
The fast-paced startup is really the hack, combining the energy of youth with the ego-mania of their founders. Ask yourself, is it healthy?
Anyway, IBM's customers tend to be other fortune 100s and governments- basically other similar organizations, and my experience was that we took care of them pretty well. The products were not pretty (no Steve Jobs-like person to enforce beauty), and rather complex due to all the enterprise requirements. But they were quite high quality, particularly the hardware.
https://www.idsa.org/profile/sassano/
Well there's a whole group, but Camillo is the guy I worked with when I was there.
Found my dream job :-)
When I pointed out that this was a stupid way to do it, they openly told me that they just wanted to sell DB2.
They were only honest because I was working for an IBM Business Partner. They wouldn't disclose it to the customer.
I worked with some pretty talented and dedicated people at IBM. The “hop on a 2am call to put out a fire because they happened to check their email and they owed the person on pager duty a beer” kind of people.
That company was a red tape rats nest, but that’s management’s fault. And you get lazy people or shit departmental culture at various points in nearly every company, but painting a tens-of-thousands strong workforce with that brush is ridiculous.
If someone is telling you to work more than 40 hours a week in a salaried position, and they're not paying out the nose, you're being scammed.
and no job i've had considered 9-5 40 hours after a 1 hour lunch break
But yes, "out the nose" is qualified by your particular situation. For me, that might be 2-3x my normal salary, which would mean I could take breaks for a few years or retire sooner.
I do not get the unified industry delusion about "why X company has a bad product". It is usually either one of two things: comfort or ego. Everyone knows that but do not want to say it out loud.
I have seen these happen time and time again. Companies that are cash cow, do not care to do a better job. There is no incentive to do a better job. Moreover, the recurring thing is that if I did something different, I wouldn't have been this much successful in the first place.
The rest of the smart consultants walk on eggshells. They hint at stuff but never want to bite the hand that feeds them because the clients would rather fire you than be challenged.
It is not an IBM thing; it's generic business thing to some degree. I really have to call this a delusion. Good consultants submit generic reports that just tell them what they want to hear. It is not you; it is the economy. Stupid consultants that are well-meaning tell them they should be the best on competitor intel. Do you not think some stupid person did not approach IBM to do what Oracle or AWS is doing? Of course, they did, and they were fired immediately.
The best consultants are less of a consultant and more of a therapist.
After doing only four-month projects for the entire year, this year's realization was that nobody in the industry wants to do better. Everyone is in their place because of ego or a perceived sense of success. Or because of a grand conspiracy theory. IBM has a significant number of government contracts, so they are set for life because the vast majority government IT systems are pigeonholed into IBM systems. The acquisition is to tell the shareholders that we are so successful that we can literally buy companies. We do not even care to do things. Whatever the new thing is, we will buy it at some point.
At one point we worked with a large energy company that was basically sold something LLM-like (large-scale indexing and searching/querying of documents) in 2016 or so. IBM had a team of 90 people doing full-time data ingestion for something like 26,000 documents. We got asked to do a counter-product in two weeks, which was literally just a TF-IDF search and some smarts around ingesting different types of documents. Both solutions performed approximately equally, except one cost something in the order of $185m and one cost $40k. Watson continued running for about a year until an external data science contractor realised they could query Watson for highly confidential board meeting notes, and it would provide full previews into the documents. The project was shuttered shortly after.
Alas, nobody gets fired for hiring IBM.
Okay, but why can't IBM enter the LLM business reviving the Watson brand?
I'd wager most people around here would have more use for Granite, however.
Until 3/4 years go I was in healthcare for 15 years, a good bunch of them being partners with IBM in radiology imaging solutions. I've been in their IBM La Gaude (former) research/presentations lab a couple times and I've seen a lot of their Watson product come and go, without much success. I have to say that I've seen a couple that were very interesting, but were mostly statistical, with no AI/LLM/... involvement.
And don't talk me about Softlayer/Bluemix. Or their private cloud racks that I cannot even remember their name...
--
0: https://pharmaphorum.com/news/ibm-sells-off-large-parts-of-watson-health-businessIBM lives off huge multi-year contract deals with their customers, each are multi-multi-million dollars worth. IBM has many of these contracts, maybe ~2000 of them around the planet, including your own government wherever it is that you live. This is ALL that matters to IBM. ALL. That. Matters.
These huge contracts get renegotiated at every X years. IBM renewal salespeople are tough and rough, in particular the ones on the renewal teams, and they spend every minute of every hour in between renewals grooming the decision makers, sponsors, champions and stakeholders (and their families) within these big corporations. Every time you see an IBM logo at a sports event (and there are many IBM-sponsored events), that's not IBM marketing to you the ad-viewer. They are there for grooming their stakeholders, who fight hard to be in the best IBM sponsored-seats at those venues, and in the glamorous pre and after party, celebs included. IBM also sponsors other stuff, even special programs at universities. Who go to these universities? Oh, you bet, the stakeholder's kids, who get the IBM-treatment and IBM-scholarship at those places.
But the grooming is not enough. The renewal is not usually at risk - who has the balls to uninstall IBM out of a large corp? What is at risk is IBM's growth, which is fueled by price increases at every renewal point not the sale of new software or new clients - there are no new clients for IBM anywhere anymore! These price increases need to happen, not just because of inflation but because of the stock price and bonuses that keep the renewal army and management going strong, since this is a who-knows-who business. To justify the price increase internally at those huge client corps (not to the stakeholder but to their bosses, boards, users, etc) IBM needs to throw a bone into these negotiations. The bone is whatever acquisition you see they make: Red Hat, Hashicorp... Or developments like Watson. Or whatever. They are only interested in acquiring products or entering markets that can be thrown at those renewal negotiations, with very few exceptions. Why Confluent? Well, because they probably did their research and decided that existing Confluent licenses can be applied to one (yeah, one) or many renewal contracts as growth fuel for at least 1-to-N iterations of renewals.
Renewal contracts correspond anywhere from 60% to 95% of IBM's revenue, depending on how you account for the the consulting arm and "new" (software/hw sales/subscriptions). I particularly have not seen lots of companies hiring IBM consultants "just because we love IBM consultants and their rates", so consulting at a site is always tied to the renewal somehow, even if billed separately or not billed at all. Same for new sw sales, if a company wants something IBM has on their catalog from their own whim and will, then that will just probably be packed into the next renewal because that's stakeholder leverage for justifying the renewal's increase base rate. Remember, a lot of IBM's mainframes are not even sold, they are just rentals.
Most IBM investment into research programs, new tech (quantum computing!) etc are there just to help the renewals and secure a new Govt deal here and there. How? Well, maybe the increase in the renewal for the, ie, State of Illinois contract gets a bone thrown in for a new "Quantum Research Center (by IBM)" at some U of I campus or tech park that the now visionary Governor will happily cut the ribbon, photo op and do the speech. Oh wait! I swear I made this up as an example, but this one is actually true, lol:
https://newsroom.ibm.com/2024-12-12-ibm-and-state-of-illinoi...
You get the drill?
IBM bought a company whose product we'd been using for a while, and had a perpetual license for. A few years after the purchase, IBM tried to slip a clause into a support renewal that said we were "voluntarily" agreeing to revoke the perpetual license and move to a yearly per-seat license. Note: this was in a contract with the government, for support, not for the product itself. They then tried to come after us for seat licenses costs. Our lawyers ripped them apart, as you can't add clauses about licensing for software to a services contract, and we immediately tore out the product and never paid IBM another dime.
I tell this story not to be all "cool story, bro", but to point out that IBM does focus on renewal growth, but they're not geniuses...they're just greedy assholes who sometimes push for growth in really stupid ways.
IBM has been around for over a hundred years, maybe they know a thing or two about running a software business :-)
My main takeaway from IBM's longevity is just how astonishingly long big companys' death rattles can be, not how great IBM are at running software businesses.
For me it was the death of IBM's preeminence in IT. When I started there a job at IBM was prestigious,a job for life. More than once I was told that we had a lengthy backlog of inventions and technological wonders that could be wheeled out of the plant if competitors ever nipped too closely at IBM's heels.
At that time IBM had never made a single person redundant - anywhere in the world. The company had an incredible sophisticated internal HR platform that did elaborate succession planning and considered training and promotion as major workforce factors - there was little need to think much about recruitment because jobs for life. IBM could win any deal, maybe needing only to discount a little if things were very competitive.
It's impossible to imagine now what a lofty position the company held. It's not unfair to say that, if not dead, the IBM of old is no longer with us.
I think the pace of progress and innovation has, for better or worse, meant that companies can no longer count on successfully evolving only from the inside through re-training and promotions over the average employee's entire career arc (let's say 30 years).
The reality is that too many people who seek out jobs in huge companies like IBM are not looking to constantly re-invent themselves and learn new things and keep pushing themselves into new areas every 5-10 years (or less), which is table stakes now for tech companies that want to stay relevant.
If your average zoomer had the ability to get a job for life that paid comparably well by a company that would look after them, I don't think loyalty would be an issue.
The problem is today, sticking with a company typically means below market reward, which is particularly acute given the ongoing cost of living crises affecting the west.
the final end-state of the company, like a glorious star turning into a black hole
Investopedia says[0] they make 60% of their profit from "Software" but how much of that is "providing cloud solutions" and similar software-adjacent consulting exercises?
[0]: https://www.investopedia.com/how-ibm-makes-money-4798528
For Red Hat, there's no longer an official "public" distribution of RHEL, but apart from that they seemingly have been left alone and able to continue to develop their own products. But that's only my POV as a user of OSS Red Hat products at home and of RHEL and OpenShift at work.
GTK is still alive. It seems like Cosmic desktop with GTK apps will be a reasonable path forward. Of course there's KDE and QT, but I mean as an alternative to those.
Yes, two decades: https://adtmag.com/articles/2003/08/04/solaris-gets-a-gnome-...
Slow and boring is a pretty nice place to be.
https://www.centos.org/centos-stream/
And Fedora is still the upstream of RHEL, nothing changed there.
CentOS Stream employs a rolling-release model, which is much less stable than RHEL.
The previous main selling point of CentOS was bug-for-bug compatibility with RHEL. Red Hat is just killing the distro by moving their focus to a non-existent market. Enthusiasts will choose RHEL, while enterprises would choose the more stable RHEL, which Red Hat could earn money from, or alternatives like Alma or Rocky.
CentOS Stream has major versions and EOL dates, and thus is not a rolling release. It functions as the RHEL major version branch and follows the RHEL compatibility rules, so it's the same major version stability as RHEL.
While you may have considered bug-for-bug compatibility the main feature, it was a major point of frustration for many users and the maintainers. That model means you can't fix any bugs or accept contributions from the community. CentOS finally fixed both problems by moving to the Stream model.
The culture makes the company. Everyone on the lower rungs of the org chart knows this, because it's what they live and breathe every day. A positive, supportive workplace culture with clear goals and relative autonomy is a thing of beauty. You routinely find people doing more work than they really have to because they believe in the mission, or their peers, or the work is just fun. People join the company (and stay) because they WANT to not because they have to.
Past a certain company size, upper management NEVER sees this. They are always looking outward: strategy, customers, marketing, competition. Never in. They've been trained to give great motivational speeches that instill a sense of company pride and motivation for about 30 seconds. After that, employee morale is HR's job.
I have worked in a company that got acquired while it was profitable. The culture change was slow but dramatic. We went from a fun, dynamic culture with lots of teamwork and supportive management, to one step or two above Office Space. As far as the acquiring company was concerned, everything we were doing didn't matter, even if it worked. We had to conform to their systems and processes, or find new jobs. Most of us eventually did the latter.
Somehow Red Hat seems to be a notable exception. Although IBM owns Red Hat, they seem to have mostly left it alone instead of absorbing it. The name "IBM" doesn't even appear on redhat.com. Because I'm an outsider, I can't say whether IBM meddled in Red Hat's HR or management, but I would guess not.
1. https://www.cio.com/article/4084855/ibm-to-cut-thousands-of-...
2. https://www.newsobserver.com/news/business/article312796900....
HashiCorp also changed their licenses to non-open-source licenses, but again I think this was technically pre-acquisition (I think as they were gearing up to be a more attractive target for an exit).
Client side state encryption was one of the things which HashiCorp always gatekeeped for HashiCorp Cloud and never implemented in the Open Souce / Sourece Available versions.
A common conspiracy theory, but not true.
j/k Love ghostty!
An "exit" from the public market?
Hopefully mitchellh will write a book about Hashicorp some time. Would be fascinating to read the inside take.
Red Hat has far more autonomy. We are not structured the same.
On the HR side — many good people are leaving; new hires have to be on-site for 3 days and located in 4 "strategic" locations in the US.
What do you mean by that, like "centos/stream" (aka https://www.centos.org/download/ ) ?
The previous main selling point of CentOS was bug-for-bug compatibility with RHEL. Red Hat is just killing the distro by moving their focus to a non-existent market. Enthusiasts will choose RHEL, while enterprises would choose the more stable RHEL, which Red Hat could earn money from, or alternatives like Alma or Rocky.
While you may have considered bug-for-bug compatibility the main feature, it was a major point of frustration for many users and the maintainers. That model means you can't fix any bugs or accept contributions from the community. CentOS finally fixed both problems by moving to the Stream model.
From my perspective, as someone who is deeply suspicious of IBM in general, that's a plus.
https://newsroom.ibm.com/2025-12-08-ibm-to-acquire-confluent...
I don't understand how this acquisition is relevant for AI.
AI is just the lastest buzzword. Everyone has it, because they have to. Don't look behind the curtain.
/s
I've been pretty happy with RP performance/cost/functionality wise. It isn't Kafka though, it's a proprietary C++ rewrite that aims for 100% compatibility. This hasn't been an issue in the 2+ years since we migrated prod, but YMMV.
RedPanda was a huge win for us. Confluent never made sense to us since we were always so cost conscious but the complexity/risk of managing a critical part of our infra was always something I worried about. RedPanda was able to handle both for us - cheaper than Kafka hosting vendors with significantly better performance. We were pretty early customers but was a huge win for us.
The stuff IBM is doing on Quantum Computing is serious cutting-edge science and engineering for instance. The R&D they are doing on semiconductors on their 2nm and sub-2nm processes is also impressive and hardcore tech. They are doing a bunch of progress on post-quantum cryptography and homomorphic encryption. They've fallen behind now, but they were also quite strong on pre-LLM NLP for a couple of decades, it was not all fluff.
Yes they have an awful enterprise culture and they are not focused on building excellent products. But what they offer fits the needs of many organizations, and a lot of the things they are doing on R&D are no joke.
> a lot of the things they are doing on R&D are no joke
Sounds a _lot_ like Microsoft too...
That said the GP is spot on for this sort of acquisition we know what will happen and has nothing to do with 2nm research division.
Agreed, like others, small startup I was with, we were acquired years ago and first advice from IBMers who'd been acquired was that IBM is like 1000 smaller companies.
This. Employees in the various sub-companies and divisions usually don't even know who most of the executive leadership is outside their little world. There is no cohesive "IBM" anymore, and I don't think there has been for a very long time.
Then laser eye surgery, magnetic storage, relational databases, UPC barcodes, DES, FFT, RISC, ...
yeah, almost nothing. /s
disclaimer: I work for IBM Research and I love every second of it.
Just look at this comment section as Q.E.D.
I won't even comment on the rest... You just proved my point.
But they don't have production. How can they develop successfully without running silicon in a fab?
Some companies like Amazon buy companies and let them run almost independently - IMDB for example, Zappos, Twitch, Whole Foods, Zoox, Audible.
https://en.wikipedia.org/wiki/Android_(operating_system)#His...
Android Inc. was founded in Palo Alto, California, in October 2003 by Andy Rubin and Chris White
Google acquired the company in July of [2005] for at least $50 million
It was ad-supported of course, but it's definitely not similar to IBM acquisitions
> ...and internal systems they have to use, whose sole purpose is to make sure nobody does anything
It felt like they basically tacked on the email functionality to to Notes to sell it, but it always seemed kinda ok to me.
Let's revisit this when it's Reply to Reply to Reply to Reply :)
I saw at least one large company that migrated from Notes to exchange and they got the email/calendaring bit done quite easily and were still running notes servers for line of business applications years later.
https://www.reddit.com/r/etymology/comments/8shw5r/what_is_t...
A lot of the successful projects at the original company are now dead.
It's also weird being in IBM, because if your "contract" ends they put you on the bench. Then you basically have to job hunt within IBM, and if you can't find anything within a month or so you are out. It's super weird.
This is standard operating procedure at most consulting/professional services firms.
GTRI locally hires a lot of non-students to work in its various labs. Its labs then pitch ideas to private companies and the DoD. Sometimes they're solicited directly if the lab is well-known and has a track record of delivering good research-oriented results. They research and build prototypes around various capabilities: robotics, avionics, even classified stuff.
They're always pitching, because contracts end or fall through, and that's the source of everyone's payroll. The labs can even be competitive with one another, and the individual researchers might spend time split between labs.
Academics as a service.
Yeesh. Which level of hell is that?
It's a fact that a publicly traded company is beholden to Wall Street and any time such a company would use their earnings for R&D the P/E and margins go down (i.e. spending more money to earn the same) and this is considered a negative signal at Wall Street and the company gets punished in the market.
So the only way a company can spend their earnings is to pay dividends or buy assets such as other companies, which then must be squeezed for margins.
More here:
https://www.cringely.com/2015/06/03/autodesks-john-walker-ex...
Dilbert’s company buys an “artsy” startup (represented by a chap with a goatee and a ponytail).
Dilbert comments something like “We get your energy and skill, and we provide … an endless supply of 3-ring binders.”
To which the chap replies “I hear that if your name goes into a binder, you lose your soul.”
They seem to primarily benefit from kickbacks in the form of both leasing and technical contracts for things like opening offices in a location for tax benefits or to promote local economy.
Then they see how far they can cut back their end of the contract after the first few months (e.g. Maybe we agreed to have 500 employees in an office, but since nobody is allowed in, we think we can get away with 100 employees.) Then this turns into trolling about how the contract never defines what in office means so can we offshore… Too much undefined confusion, so I guess we get to break the contract but keep what the mayor paid us… Then they just shut down the office and move on to the next location.
It seems like the local government must be in on these schemes for leasing. Otherwise this wouldn’t be going on for decades as it has been.
The other part of business, technical contracts, is similar except instead of leasing it’s providing some sort of infrastructure coverage for something big. It starts off with good faith fulfilling the contract. Then a few months later it’s like well we have a US military contract that demands US employees but US employees are too expensive. What if we offshore but all the traffic is technically going through a single US employee’s computer which is what the contract technically demands.
Then it turns into well we have offshore people working on this anyway, why not just give them direct access and we’ll have a US person overseeing them. Lay everyone else off.
Then they see how long they can get away with this until someone gets mad. Then they take one step back to see how close to the technical contract they can get while threatening to abandon the whole thing at the same time.
Along with this sort of atmosphere and attitude for the law, it seems we see them constantly doing everything possible to constantly fire old people or anyone else that has legally protected status. So you’ll get statistical analytics on ways to fire protected people based around the constant performance reviews with statistics being used to see how close groups of protected people can be removed without statistically breaking the law. Whatever that algorithm is.
That plays into just straight up cutting people, but it also goes into a lot of other subsystems of skirting the law, like if old people can’t relocate as easily then hopping offices and forcing people to relocate 5000 miles is a way they can be eliminated. Part of this might be moving people onto new teams and then saying that team has to be in office for some made up reason, and then firing them for not relocating or using some made up metric like badging timestamps to get them, or some other technicality like leaving for lunch 5 minutes early despite being a salaried employee which is reported as hourly because of tax trolling.
I don’t know how IBM still exists because from my perspective it’s pretty clear they’re breaking or at best on razor thin gray line on ice on just about every possible law you could break.
I've heard chatter from our engineering leadership that IBM is trying to push some silly initiatives, but we've been able to prioritize the right work so far.
I also get more equity (one time award + employee stock purchase plan) than I did previously, and with how IBM stock has been performing lately this has been a net positive for me.
FWIW I have heard that IBM used to force their management style on acquisitions in years past, so perhaps this is a fairly recent shift towards a less hands-on approach.
Definitely wasn't like that for Red Hat. We had a CFO with an IBM past which was a really nice guy and never ever felt like he was parachutes from IBM.
Now after 6 years legal, HR and finance will move to IBM starting next January; but my perspective from engineering is that after the acquisition it's been and remains business as usual.
I have no idea how it was for Hashicorp.
Outside IBM land, Meta runs on a CentOS Stream fork.
You might live/work in a bubble. It's used everywhere in large enterprise.
The worst is when your sales team (and all of its super valuable institutional knowledge of your specific market) are cut, and all your management is laid off so that the new corp's managers (who have embedded themselves into the corporate bureaucracy like a trichinosis worm) can treat all your teams as free headcount.
Soon, your company, which was acquired for growth, can't do anything and turns into an albatross around the new corp's neck. So the layoffs begin.
It reminds me of another IBM IT rule: they wanted your chat history (and email) older than two years to be all deleted for legal liability reasons. It was important to save your sametime chat history (an XML file) and export your email periodically if you wanted to keep this stuff.
This was actually better than Slack in one way- you could grep the files for things, and not have to rely on search within the tool.
Hint: by all means possible, make sure you are not the owner of (or manager of the person who owns) any assets beyond your personal laptop. If, for example, you end up being the owner of all the development and test servers of the original company, then it will become your responsibility to ensure that each OS (of each LPAR of each VM) is security compliant, is running the end-point asset manager, and has up to date OS patches, that the DASD is encrypted, and you must periodically show physical proof that the asset still exists and indicate where it's located- photos of assets tags or whatever. It will be your responsibility to dispose of the asset (with all associated paperwork) at the end of its life.
It helps if such machines are not actually on the 9. network, or are behind an internal firewall (then they don't care about the security compliance as much).
I still "own" (i.e. I'm the sole user with a root access and can install OS of my choosing) an old machine from the days before everything moved to a cloud and guess no one from IT has got to decommission it yet. I'm have no idea where it is located (besides knowing which office it is assigned to), never saw it, no way in hell am going to attach any tags and waste my time to install enterprise spyware on it or manually encrypt it's data. Do engineers do that for development servers on your job? If yes, name and shame!
It was right before I left that we got our own Jira instance. This was all around the time of the Red Hat acquisition. I remember the announcement b/c we used SuSE for everything IIRC.
So if they somehow can get past initial device deployment/user account logon, and get other resources IE; slack....well that speaks to how difficult/pointless it would be to get proper VPN/Jira access.
Sometimes a team unilaterally decide to change the process, info is sent to a random number of mailbox/managers who may fail to pass the info. Some entire teams just put themselves in away status 24/7 and do not respond to direct messages.
So yes I can believe his story. Sometimes in these kind of companies you just don't know who and how to ask for something and you just hope someone knows someone who might know.
Have a table level seqno as monotonically increasing number stamped for every mutation. When a subscriber connects it asks for rows > Subscriber's seqno-last-handled.
The second you approach any kind of scale, this falls apart and/or you end up with a more expensive and worse version of Kafka.
I was surprised how far sqlite goes with some sharding on modern SSDs for those in-between scale services/saas
Kafka already solves this problem and gives me message durability, near infinite scale out, sharding, delivery guarantees, etc out of the box. I do not care to develop, reshard databases or production-alize this myself.
My main point is, I have zero interest in creating novel solutions to a solved problem. It just artificially increases the complexity of my work and the learning curve for contributors.
Not everything needs to be big and complicated.
(SELECT * from EVENTS where TIMESTAMP > LAST_TS LIMIT 50) for example
But yeah, for a lot of implementations you don't need streaming. But for pull based apps you design your architecture differently, some things are a lot easier than it is with DB, some things are harder.
anyone have an idea on how streamnative is doing? we're considering them for managed pulsar and unfortunately nobody else is in the game
we keep pushing Pulsar and moving things - https://streamnative.io/blog/ursa-wins-vldb-2025-best-indust...
When looking for alternatives to Kafka these are the most promising options I found, not counting RabbitMQ which needs no introduction.
Pulsar seems to be 'kafka done better'. The version of Kafka that Confluent used internally (Kora) seems closer to Pular as well. Pulsar has a lot of features that Kafka doesn't have, like per-message acknowledgement, similar to RabbitMQ. And it has protocol support for RabbitMQ and Kafka, so can be a drop-in replacement.
Redpanda seems like a great re-implementation of Kafka.
I'm hoping this boosts Pulsar's status and helps get some traction for StreamNative. It seems like the best technical solution for events and messaging. It just needs a bit more market adoption to make an easier choice for enterprise, in my opinion. This might be that moment.
*(yes I know about NATS)
Now AWS is eating Confluent's lunch with it's managed Kafka. AWS MSK is still a little rough like all AWS services but it's still cheaper and a no-brainer if you are already on AWS.
Since when is streaming event logs AI? Am I taking crazy pills?
Confluent stock soars 29% as IBM announces $11B acquisition deal
https://www.cnbc.com/2025/12/08/ibm-confluent-deal-data.html
A nod's as good as a wink to a blind bat!
Nudge, nudge, know what I mean?
And here is a picture of Ben Lorica 罗瑞卡 interviewing Jay Kreps and other industry leaders at The Hive back on the evening of 25 February 2015. I believe they were talking about strategies for implementing Lambda Architecture.
All of which is to say: I have been a big fan of both companies for a long, long time. While today I am at employed at Redpanda Data, a direct competitor of Confluent, I hope to set aside any "team"-based bias to provide a sober and honest appraisal.
First, IBM has been shrinking. They were at 345,000 employees as of their 2020 Annual Report. But the COVID-19 pandemic was only one of many setbacks the company faced when Arvind Krishna took the helm as CEO. By December 2024 the employee base shrank to 270,000 — a drop of nearly 22%.
IBM revenue in 2020: $73.6B.
IBM revenue in 2024: $62.75B — a less-precipitous drop of 15%.
Revenue per employee over that period rose from $213k to $232k.
Confluent on its own? $400k.
And to compare: Amazon earns $580k per employee. Microsoft generates over $1M per. Nvidia? $4M-$5M.
And now, in November, they announced thousands of more layoffs. No one seems safe, regardless of job title. Those cut include positions in "artificial intelligence, marketing, software engineering and cloud technology."
Next, IBM has had a mixed record as a steward of acquisitions. Red Hat has doubled in revenues since their 2019 acquisition. For a while its headcount continued to grow, as much as 19,000 by 2023. But then it was forced into layoffs by parent IBM in April of that year, and then each year since, even while it remains one of the highest margin businesses in their portfolio.
SoftLayer — "IBM Cloud Classic" — also suffered significant layoffs in early 2025, with offshoring sending jobs to India.
DataStax had layoffs in 2023-2024, even before its acquisition was announced. Maybe they were "trimming the fat" to get into a shape to be acquired.
As a person with a long career in marketing, I know that many of the first roles to be jettisoned at a newly-acquired company tend to be in go-to-market organizations. Sales, Marketing, Developer Relations, Documentation, Training, Community, Customer Service. These tend to be seen as "nice to haves" by upper management. But their loss guts organizations and hollows out user-facing teams and open source communities.
My hope is that Confluent is spared as much of the pain and turmoil as possible. That, like Red Hat, it is run autonomously as much as possible.
[Crossposted from LinkedIn here, where you can see the photo mentioned: https://www.linkedin.com/feed/update/urn:li:activity:7404052...]
The spin-off also explains the drop in revenue. In 2021, the first results reported excluding Kyndryl, IBM had revenues of $57.4B [1]. Since 2021, revenue grew by ~9% to reach $62.75B in 2024.
[0] https://www.kyndryl.com/us/en/about-us/news/2021/11/2021-11-... [1] https://newsroom.ibm.com/2022-1-24-IBM-RELEASES-FOURTH-QUART...
I can't think of a better place for longevity of open source projects than Apache (maybe I'm out of the loop?).
Compare it to the Linux Foundation where everything is a single commercial vendor sponsored project. At lease Apache requires independent governance and a diverse ecosystem before the project graduates.
Am I missing something with the Apache Foundation?
But you can also use Kafka as pub-sub system (duplicate messages to each subscribers)
Messages are not deleted after consumption (configurable)
IMO they were simultaneously worse situated for near-real-time stream processing and for S3-esque cloud storage, areas Kafka and Confluent excelled.
I'll start.
kevstev wrote just above about Kafka being written to run on spinning disks (HDDs), while Redpanda was written to take advantage of the latest hardware (local NVMe SSDs). He has some great insights.
As well, Apache Kafka was written in Java, back in an era when you were weren't quite sure what operating system you might be running on. For example, when Azure first launched they had a Windows NT-based system called Windows Azure. Most everyone else had already decided to roll Linux. Microsoft refused to budge on Linux until 2014, and didn't release its own Azure Linux until 2020.
Once everyone decided to roll Linux, the "write once run everywhere" promise of Java was obviated. But because you were still locked into a Java Virtual Machine (JVM) your application couldn't optimize itself to the underlying hardware and operating system you were running on.
Redpanda, for example, is written in C++ on top of the Seastar framework (seastar.io). The same framework at the heart of ScyllaDB. This engine is a thread-per-core shared-nothing architecture that allows Redpanda to optimize performance for hardware utilization in ways that a Java app can only dream of. CPU utilization, memory usage, IO throughput. It's all just better performance on Redpanda.
It means that you're actually getting better utility out of the servers you deploy. Less wasted / fallow CPU cycles — so better price-performance. Faster writes. Lower p99 latencies. It's just... better.
Now, I am biased. I work at Redpanda now. But I've been a big fan of Kafka since 2015. I am still bullish on data streaming. I just think that Apache Kafka, as a Java-based platform, needs some serious rearchitecture,
Even Confluent doesn't use vanilla Kafka. They rewrote their own engine, Kora. They claim it is 10x faster. Or 30x faster. Depending on what you're measuring.
1. https://www.confluent.io/confluent-cloud/kora/
2. https://www.confluent.io/blog/10x-apache-kafka-elasticity/
Kind of like how people use docker for evrything, when what you really should be doing is learn how to package software.
Agree on the Kafka thing though. I've seen so many devs trip over Kafka topics, partitions and offsets when their throughput is low enough that RabbitMQ would do fine.
The people distributing software should shut them damn up about how the rest of the system it runs in is configured. (But not you, your job is packaging full systems.)
That said, it seems to me that this is becoming less of a problem.
So you are stuck with some really terrible tradeoffs- Go with Confluent Cloud, pay a fortune, and still likely have some issues to deal with. Or you could go with Confluent Platform, still have to pay people to operate it, while Confluent the company focuses most of their attention on Cloud and still charges you a fortune. Or you could just go completely OS and forgo anything Confluent and risk being really up the river when something inevitably breaks, or you have to learn the hard way that librdkafka has poor support for a lot of the shiny features discussed in the release notes.
Redpanda has surpassed them from a technical quality perspective, but Kafka has them beat on the ecosystem and the sheer inertia of moving from one platform to another. Kafka for example was built in a time of spinning rust hard disks, and expects to be run on general purpose compute nodes, where Redpanda will actually look at your hardware and optimize the number of threads its spawns for the box it is on- assuming it is going to be the only real app running there, which is true for anything but a toy deployment.
This is my experience from running platform teams and being head of messaging at multiple companies.
Not a drop in replacement, but worth looking at.
Sigh.
I used ZMQ to connect nodes and the worker nodes would connect to an indexer/coordinator node that effectively did a `SELECT FROM ORDER BY ASC`.
It's easier than you may think and the bits here ended up with probably < 1000 SLOC all told.
- Coordinator node ingests from a SQL table
- There is a discriminator key for each row in the table for ordering by stacking into an in-memory list-of-lists
- Worker nodes are started with _n_ threads
- Each thread sends a "ready" message to the coordinator and coordinator replies with a "work" message
- On each cycle, the coordinator advances the pointer on the list, locks the list, and marks the first item in the child list as "pending"
- When worker thread finishes, it sends a "completed" message to the coordinator and coordinator replies with another "work" message
- Coordinator unlocks the list the work item originated from and dequeues the finished item.
- When it reaches the end of the list, it cycles to the beginning of the list and starts over, skipping over any child lists marked as locked (has a pending work item)
Dead simple design, extremely robust, very high throughput, very easy to scale workers both horizontally (more nodes) and vertically (more threads). ZMQ made it easy to connect the remote threads to the centralized coordinator. It was effectively "self balancing" because the workers would only re-queue their thread once it finished work. Very easy to manage, but did not have hot failovers since we kept the materialized, "2D" work queue in memory. Though very rarely did we have issues with this.
Generally I say, "Message queues are for tasks, Kafka is for data." But in the latter case, if your data volume is not huge, a message queue for async ETL will do just fine and give better guarantees as FIFO goes.
In essence, Kafka is a very specialized version of much more general-purpose message queues, which should be your default starting point. It's similar to replacing a SQL RDBMS with some kind of special NoSQL system - if you need it, okay, but otherwise the general-purpose default is usually the better option.
> Ah yes, and every consumer should just do this in a while (true) loop as producers write to it. Very efficient and simple with no possibility of lock contention or hot spots. Genius, really.
https://docs.streamnative.io/cloud/build/kafka-clients/kafka...
IBM will likely give Confluent employees a large pay package, and then let them go after the merger.
edit: btw, it's typical for any acquisition/merger
IBM is a really big and diverse company, in a way fundamentally different from most other big tech. In a sense, it is completely incoherent to refer to them as a singular entity.
My opinions are my own. I worked at IBM like a decade ago in a role where I could see the radically different motivations of divisions.
Some redundant departments (HR, finance, accounting and the like) will be downsized after the acquisition.
Engineering and product will be unaffected in the short term, but in a year or two the IBM culture will start to seep in, and that would be a good time for tenured employees to start planning their exits. That's also when lock-up agreements will expire and the existing leadership of Confluent will depart and be replaced by IBM execs.