IBM swallows Red Hat storage products
blocksandfiles.com
blocksandfiles.com
- Redhat kneecaps centos because some mid manager was worried he might not get this years bonus
- Redhat kneecaps Ansible by making ansible-core and forcing everyone to re-install community plugins they were using from EPEL, or buy the shiny Redhat version of ansible.
- Redhat decides to turn Ceph into yet another garbage storage product with the same exorbitant support fees and contract payments that were the original motivation for an open source storage solution in the first place.
All this is being done under the guise that "oh its not proprietary, its enterprise grade now!" like the very same devsecops engineers and SRE's who made this shit work in the first place arent wholly capable of running it in production without forking over most of their $round of investment to the same company that turned softlayer into a swirling vortex of DDoS gangs spam relays and double extortion rings.
to date it feels like all IBM wants to do with Redhat is turn it into IBM so if thtas all these juggernaut dinosaurs can do im quietly waiting to see Microsoft acquire Ubuntu and quietly turn it into a teams plugin the same way they acquired github and turned it into an outage response simulator for developers.
The ACD (or what I still call "Ansible") includes everything that was traditionally part of Ansible, and then some, just bundled up into one big install instead of requiring you to install ansible-core then pick and choose collections to install on top.
This covers about 99.9% of all use cases (for Ansible for automation work) still, and for anything else, you can add your own roles or install an additional community collection or two.
I understand why collections got split out due to development needs, but the switch over to them has been pretty crappy from a user perspective.
But it was clear later when I met from people from Lotus there was tension between the Lotus division which IBM had bought. The cultures didn't mesh well and they resented everything being IBMified. Though from the IBM side it seemed they really were all in on Notes. Really pushing it hard. Lotus's "smart suite" of desktop application not as much.
Another friends company got bought by IBM a couple years ago. It seems they didn't really want it as they let half the people go (core people, not just overhead).
Its kinda sad really. It seems its better after an acquisition to let the company keep doing what it was doing, with minimal changes (excepting removing the corporate overhead stuff).
I'm pretty sure that if not for this we would have moved away in the early 2000s. Instead we waited until the situation became so untenable that we had no choice to invest the millions and years to make it happen. I remember Notes constantly crashing when someone would send me a 'funny' GIF on Sametime, and of course it would take all my draft emails with it because the autosave never seemed to do what it was supposed to.
The only thing I did like about it was collapsible sections in emails. And the fact that you could write really advanced 'agents' for it. I had one that would remove the "return read receipt" flag and insert a button in the email so I could send it whenever I did wish to send it instead.
Anyway, when IBM took over Lotus we were still full-on using it and I was really hoping they'd clean up the product and make it functional. Of course none of that happened and they just let it continue on its path to extinction. I have no idea why they would buy something way after its prime and then just let it wither away.
Oh yeah and the notes mail databases. When there was a new version we'd get tons of questions like "why does my mail still look like the old app but notes itself is new??". Well, click on the upgrade your database. Ooops now it got corrupted completely well good luck bye :D
Seriously why did the email UI look and behaviour have to be embedded inside each user's mail database??? This makes totally no sense and breaks pretty much every software design concept. As well as create tons of vulnerabilities for malware to embed itself. Data and code don't belong in the same place. Though luckily Notes got deprecated before that really became a thing.
>The only thing I did like about it was collapsible sections in emails. And the fact that you could write really advanced 'agents' for it. I had one that would remove the "return read receipt" flag and insert a button in the email so I could send it whenever I did wish to send it instead.
Thats pretty good.
I see notes as a basically a NoSQL database attached to a primitive gui and a very bad scripting language (lotus script). You could make a little application and deploy it to a team pretty easily. IBM had a bunch of little Notes databases/applications that could be used by the dept to track project and other things. I haven't seen much that replaces its sort of simplicity and ease of use. I guess the web, but it seems it takes more to stand up a custom app these days.
I have no idea how you would export the unstructured data that lives in those databases..
True. This is 15 years of frustration talking.
I'm surprised you didn't have the stability problems. So so so many times have I been looking at a standard Windows "OK" popup box with so much java traceback in it that it literally filled the entire screen. Some periods this happened literally every 2 days or so. What helped stability extremely was installing the separate Sametime client and disabling the sametime functionality in Notes itself. However our admins frowned on that for some reason (never found out why). So I always had to get it from dubious sources.
> I see notes as a basically a NoSQL database attached to a primitive gui and a very bad scripting language (lotus script). You could make a little application and deploy it to a team pretty easily.
Too easily, yes. It was too handy. I wonder if this was intentional because these little databases had a way of becoming indispensable over time. Leading to Notes being set in stone as a business requirement.
> IBM had a bunch of little Notes databases/applications that could be used by the dept to track project and other things. I haven't seen much that replaces its sort of simplicity and ease of use. I guess the web, but it seems it takes more to stand up a custom app these days.
Yep.. MS Access roughly had the same and it was actually structured as a decent relational database, not some unstructured mess. But file based, a bit like sqlite with a forms and scripting tool built in. So it was also easy to migrate to SQL Server once an app became so big that it needed a real server. MS even had (has?) a wizard for this. It was really piss-easy to make a little tool in it and its scripting language was easier and known by more people (being VBA which most Excel experts know). It was even quite possible to make decently performing fairly complex apps in it. Not sure how it stands now as I haven't used it in years, but from what I've seen with some quick looks throughout the years, Access has not really changed at all since 2005 or so.
However, Access did not reach this ubiquity and the reason is that MS left it out of the cheapest office editions. Which meant most standard users never had it on their PC and they would have to request it and provide a business reason for the cost.
Notes on the other hand was always there and available, ready for people to tinker with to their heart's desire and create a whole shadow IT world :) Very smart move from Lotus, I have to say.
> I have no idea how you would export the unstructured data that lives in those databases..
Oh don't start... It was terrible. None of it was documented because said "handy Joe"s were usually not experienced developers and had almost always left the company years ago.
Also, most of these had been unmaintained for many years and thus not really fitting business requirements anymore. So lots of ugly workarounds were used, and the business side tried to use the project to inject new business requirements. Which in some cases made sense but in other ways would derail the project completely.
In the end most of them were converted to web using a low-code/zero-code platform.
What they do now is dabble in a bit of consultancy, a bit of datacenter ops, a bit of software and some servers here and there. And of course all the support on their legacy mainframe stuff. But when they have something that really stands out like the Thinkpad line they divest it.
It's almost as if they don't want to be noticed or something. I don't know.
I never thought their RedHat acquisition was a good thing for RedHat, though I never thought RedHat was a very good company to begin with. When Microsoft acquires Ubuntu (and I'm surprised this hasn't happened yet) they have more good stuff to mess up. Which I'm sure they will. And it seems like Canonical in anticipation of the acquisition has already started going the wrong way themselves.
Is Ceph still a viable alternative in hardware and operational cost and reliability to HDFS to store hundreds of PBs of data, if not EBs? I kept hearing people recommend Ceph over HDFS, but was unable to find examples of large deployment of Ceph for HDFS-like workload.
Reading the histories of both companies, it seems to me that Redhat was always IBM. The acquisition was a marriage of separated twins more than anything else, the dominating culture in both organizations is largely sales and consulting driven, with public strategy not quite identical but strongly rhyming. The technology aspects (while perhaps once important) were far from the prime focus of either organization long prior to the merger. I think of IBM's acquisition more as a desire to access marketing expertise and established sales channels than anything to do with technologies, most of RH's open source technology is after all mostly just an advanced form of disguised marketing
From what I know Red Hat devs are also very open in other areas of OS research and development, from the new io_uring, eBPF, etc. IBM is trying to stamp it out, but it's still there.
We manage this POS for some clients, and it's the worst distribution you can have. Stuff you can "just install" under any other popular distro is often silosed off into special packages you need to buy, despise it being OSS and "just available" on EPEL.
> - Redhat kneecaps centos because some mid manager was worried he might not get this years bonus
Red Hat paid (pays, probably) bonuses quarterly, and they were (are, probably) based on stock performance. They are not and never were based on cost saving measures.
The reality is that, probably, CentOS should have never become part of the "Red Hat family". The Chinese firewall concept didn't really work internally, the vast majority of RH engineers ran Fedora anyway, and it was basically just giving CentOS better build infrastructure plus giving customers a way to buy their way out of nonsense situations. The Red Hat support agreement always said "if you're not running RHEL, we'll close your case", which frequently meant customers who ran CentOS in dev/qa/preprod and RHEL in production who were forced to install another RHEL system (which RH made no money on anyway, since they never charged license costs, only subscription fees for updates) just to reproduce an issue so the sosreport didn't say "CentOS Linux ...".
In general, the same engineers who worked bugs (or were escalated to cases) on RHEL were often part of the community of upstream maintainers anyway, and RH's policy was "upstream first, and if it needs an emergency downstream bandaid, take it out once upstream accepts a patch and backport it properly", but it was easier for customers to get attention on problems if they could back it with dollars.
Rocky, Alma, and OEL (however you feel about that) exist anyway and basically picked up where they left off. Stream probably always should have been a thing. It was (probably is) an enormous amount of work to branch, re-sync, and generally re-sync again any given Fedora snapshot to a new RHEL release, and getting it into shape was (probably is) a nightmare, because the difference between the code which is currently running on any given RHEL release and what upstream is is huge. sysvinit -> upstart/sysv -> systemd, for example.
- Redhat kneecaps Ansible by making ansible-core and forcing everyone to re-install community plugins they were using from EPEL, or buy the shiny Redhat version of ansible.
The Ansible leadership was generally terrible about open source philosophy regardless. This is nothing new.
- Redhat decides to turn Ceph into yet another garbage storage product with the same exorbitant support fees and contract payments that were the original motivation for an open source storage solution in the first place.
Open source Ceph still exists. I don't see the difference here. It's just moving it from one independent company to another independent company, at least nominally.
The systemic problem with IBM acquiring Red Hat was in a lack of good answers for many of the questions employees had (policies on open source, WFH, etc), the fact that the purchase was not communicated even to management until news articles broke, a six month cliff for "business critical" employees, and following that, the fact that IBM fundamentally believed that they could do what everyone always thought Red Hat was doing and drive the market, so they were constantly pouring money and engineers into projects that had no chance to succeed in the market rather than adopting and fostering upcoming solutions, and *that* drove engineers out of the company very rapidly.
Instead of being a fork, it’s upstream.
>to date it feels like all IBM wants to do with Redhat is turn it into IBM so if thtas all these juggernaut dinosaurs can do im quietly waiting to see Microsoft acquire Ubuntu and quietly turn it into a teams plugin the same way they acquired github and turned it into an outage response simulator for developers.
RedHat was eating into their business so they bought it.
> https://ubuntu.com/ceph/docs/supported-ceph-versions
> For instance, it is possible to deploy Ceph Quincy on Ubuntu 20.04 LTS via the UCA or on Ubuntu 22.04 LTS via the regular Ubuntu package archive. See Ceph and the UCA for details.
That is the key take-away for me that I hope remains true and does not go the route of Sequent. Should that be the case I also hope the Ceph Foundation keep Ceph alive and independent from IBM perhaps by forking it in the same manor that was done with MySQL (MariaDB and others) when Oracle acquired it.
Open Source commitment, upstream first, and foundation continue as before.
I am a member of the team, and while we are certainly adjusting to the news, we are also pretty excited. Red Hat has been generous to Ceph, but it could not take Ceph beyond platform storage for OpenStack and Kubernetes — we want to reach for the full “Linux of Storage” original vision.
Ah, here we go... https://blog.noobaa.com/blog/the-noobaa-brand-brilliant-foun...
"We did in fact have high hopes for words like “Nubes”, amazingly the latin word for cloud, also a play on words with “newb” reflecting our introduction of a new architecture for reliably retaining data.
So then it’s time to lock in your domain name. GoDaddy had nubes.com on sale for $800, of course wayyy beyond the budget of this self-funded entrepreneur (and BTW still available. Fortunately GoDaddy’s algorithm sensed our cheapness, suggested alternative “NooBaa.com” for $3.99 per year, and the mouse went 'click!
But....NooBaa didn’t mean anything. Nobody spelled it right. Everybody asked us what it means. We invented creative facts about African tribes. Then we observed that it’s attention-getting. It’s intriguing."
It's baaaaaad.
Technical founders are really a special bunch.
It's almost the real life equivalent of Tobias Funke's "A Nu Start".
Now we have some RedHat employees getting the news they are being transferred to an IBM area. I wonder how that's viewed from the RedHat side.
Shocking absolutely no one. Every acquisition comes with the promise of "X will let Y operate exactly like it always has been, and we'll never steer their direction" and it has never been true.
You align software to where it works best. IBM is better suited to develop and sell Software Defined Storage than Red Hat.
I saw some people furious because of personal prejudices against IBM but many seem to be looking at this opportunistically.
It wasn't clear which products those would be and when, though.
It seems 15 years ago I was asking ?why aren't we running this in vanilla Tomcat?
I know, I know, it's like "Beetlejuice", you say "Integration Integration Integration" and five salesmen appear to sell you something that provides a bad UI in front of configuration files you'll eventually hand-edit anyway.
Most of the big banks
Ceph is fine, but its a toy.
You can slam hooks into everything, and the raid replacement (which is probably only allow on IBM tin, the bastards.) is brilliant. (rebuilding stuff doesn't take 24 hours)
Want to have a global filsystem, no problem, want to mange wrote clashes? no problem, everything is based on rules. Want to have a storage array in the US, but a copy in the EU, no problems. just set your rules and you're away.
and its all posix and really rather scalable.
Because?
Thats the easy part. allowing concurrent access to the same block, managing write clashes, decent cacheing are hard. Even harder when you are doing it with posix semantics.
The main issue with Ceph is that the filesystem is an addon and required a huge amount of resources to make it work properly. (search out the ceph plan for gitlab, before they were bought out by a company that insisted on bringing in sysadmins to sort infra out)
the big and largest elephant is that cephfs is still not really "stable" by that I mean supported fully by a large support company. This means that if something goes wrong, you need to figure it out yourself.
The opposite. Ceph started out with the underlying load balancing algorithm (May 2004), then with the plan for the filesystem metadata management (Nov 2004). The block storage was created to support the filesystem, virtual block devices were an afterthought to enable private clouds, object storage was an afterthought in a world where S3 got popular.
See https://www.ssrc.ucsc.edu/person/sage.html for the chronological order.
Of course, stabilizing the most complex part of the stack, depending on almost everything else already being stable (everything except RBD and the S3-compatible gateway, which both are relatively slim layers on top of RADOS), takes the longest. Meanwhile, business users were very keen on getting the benefits of RBD for "private clouds" and the object storage, so the filesystem was on the back burner.
[Disclaimer: I worked on Ceph pre-RedHat]
Well, it'll happen, unfortunately. Everything that Big Blue buys will eventually be assimilated. Unfortunately the next phase is usually a few years of tightening budgets, then beyond that 'buying another thing to replace it'.
> Unfortunately the next phase is usually a few years of tightening budgets...
This happens because post-Blue Washing, execs want to quickly drive cash towards shareholders and bonuses towards themselves, and the easiest way to do that is to slash budgets. This translates into losing the best and brightest, and...
> ...then beyond that 'buying another thing to replace it'.
This doesn't happen at 1:1 product granularity as much as many think it does. It is more a big swap of strategic focus, which happens to include large, splashy purchases of other assets that gets lots of attention while existing steady revenue generators get metaphorically tossed aside if not budget-wise, then management focus-wise.
No way Red Hat is staying like some gilded ivory tower of open source purity. They are a brand name for selling services and licenses, so if Red Hat still looks after ... wait that was only 3 years ago? Well anyway if Red Hat is still recognizable after that acquisition in some non-IBM way then IBM is showing enormous restraint.
The previous 10 years at IBM were "buy company, attach Websphere to the front of their product names, sell, abandon product". Still better than Computer Associates software, but not much better.
IBM has left us alone. I don’t get paid by IBM, my benefits are Red Hat, My CEO is Red Hat’s CEO and funny enough when we work with IBM we lead the calls because IBM is letting us do what we do best.
I could sense storage was changing. We had product delays, we kept pushing things out to future releases to expand ODF to our managed services. Evrything was always next quarter. Now it makes sense.
I wouldn’t be surprised if Jboss is next.
It's sad, because there's a lot of genius level people that work on Ceph. But the project as a whole is not high enough quality to be reliable.
I imagine most people that work on ceph (eg, the code, not the rest of the org) will want to move to greener pastures soon. Anyone that wants to work for IBM pretty much already did.
https://www.redhat.com/en/blog/red-hat-storage-strategy-upda...
It’s always cheaper on the bill of materials. Who is going to build and maintain the complex infrastructure. It would be interesting to see the TCO over 5 years.
In our company it certainly is not cheaper. But what they do is ridiculous. They just moved from physical datacenters to AWS instances. Every "server" instance is running 24/7 independent of load, and you have to fill in an excel of 10 tabs to get permission to fire up an instance. In fact, the only thing they changed is the physical rack location tab to an AWS zone one :P
So now we are "in the cloud" but we have almost none of the benefits said cloud offers. And we're using it in such an inefficient way that we're wasting money by the millions. And we kept all the internal overhead we already had when we managed the datacenters itself (that's what that excel is for).
Once we actually start dynamically spinning up instances based on load and shutting them down when not needed, then yeah I could see it being cheaper than having a bunch of metal in a rack somewhere. But I guess we won't get there in the next 10 years or so. We'll probably be all-in on cloud by the time everyone has already gone on to edge computing or whatever the next big thing is, lol.