Splunk IP suit against Cribl
splunk.com
splunk.com
They are mad because Cribl is good at transforming data before it ingested by Splunk, so as to reduce the amount of data that is indexed. Period.
Splunk ONLY RECENTLY released “Ingest Actions” to filter data post-ingest (to avoid indexing) for their SaaS product — something that has always been a mainstay of their on-premise “Enterprise” product. Their ONLY suggestion to filter data that we didn’t care to index in early 2021? Cribl. There’s literally no other reason for us to use Cribl.
I’ve been paying for Splunk since 2008 and can’t wait to get away from them. Their sales teams have decayed into unethical slimebags and I am trying everything in my power to not renew our contracts with them. This just sealed the deal.
Source: I cut checks to Splunk for $x,xxx,xxx yearly
Elasticsearch simply couldn’t handle key collisions. We have hundreds of various apps across 5-10 different languages and frameworks where a key name may be reused as either a string or a hash or an integer or an array. If we can’t freeform search (which Splunk is EXCELLENT at), we just need to be able to transform the data beforehand. Datadog plans to do so with their recent acquisition of Vector.
Instead, each of these systems have their own collectors and correlating from one to the other is hard. A canonical log line is so much more valuable than a metric collected every 60 seconds, and the former can derive the latter: https://stripe.com/blog/canonical-log-lines
Splunk should have been the lynchpin.
Shoving SignalFx down our mouths and trying to get us to create “metric” indices was the straw that broke my back.
I'm sure that's not 100% true but it felt like it. Trying to build Splunk on top of a modern IaC deployment methodology is a huuuuge lift.
Pretty lame not cloud native.
This is the question. If you’re looking for APM well you’ve got great options but for those using Splunk in the security space (SIEM & SOAR) you’re screwed.
There’s no better SIEM alternative that deals with logs at scale.
Splunk recently screwed a friends Fortune 50 company. They didn't pay a bill on time (renewal negotiations) and Splunk without even contacting them just left all the logs from one of their instances on the floor. They lost everything for literally an entire country.
I mean EVERYTHING.
NONE of our other vendors have EVER done that shit in my entire career. EVER. My word means nothing to them; they act like a pure private equity player now.
I think folks that use Splunk for basic search just don't fully comprehend how capable the product is for hunt-type operations when someone fluent in SPL is at the helm.
My frustrations with Splunk have been around their certification and training changes over the years. Used to be able to get a solid tool certificate and decent training materials all for free. It only hurts Splunk though as less people have experience with the tool it lessens their advantage. Makes me disappointed as I really do like the tools itself but literally everything else is terrible. I'd much rather deal with Elastic or go open source with Security Onion.
I don't side with Splunk on their actions but I can't stand customers who withhold payment as a renewal tactic and think that it's equally wrong.
My first few weeks at Splunk were very odd. They try to indoctrinate new hires with a barrage of "A-players" that continuously talked about how awesome Splunk was. Except... When I started Splunk was getting their ass kicked by cloud-first players that had recently come to market. Splunk's monolithic architecture wasn't well suited to be run as SaaS at the time and Splunk was burning cash and losing money on every customer that they suckered into moving away from their perpetual licenses into subscription hell. I left money on the table when I ran out the door less than 6 months later.
I'm curious what Splunk's long game is with this because they just told every F2000 that their bottom line is being chipped away by Cribl and friends. So if I'm an enterprising procurement department I'd be tossing Cribl or Rudderstack or whatever other data transformation preprocessor on the table alongside my renewal. Expand opportunity? If you put your ear to the tracks you can almost hear all of the account managers digging out missed quota excuses.
Splunk isn't innovative and hasn't been for a long time. Most of the employees saw the writing on the wall and went to Snowflake as soon as the opportunity presented itself. Splunk tried to capitalize on the security market by, basically, double charging customers for ES. Instead of delivering value it seems to be Splunk is just looking for ways to squeeze a few last drops of lemonade.
Splunkers have received over 1,020 patents to date
You could have made the same choice, but did not.
Their SaaS offering used to have said inline tier called IDM (Inputs Data Manager) where we were directed to configure filters during our POC… a key requirement for moving from Enterprise to SaaS because conf files aren’t managed the same. One month (to the day!) after we moved, they randomly decided to migrate us to a new “Victoria experience” where that tier suddenly disappeared without explanation. We filed support tickets asking 1) what happened? and 2) how do we filter things out now? and were directed to hire professional services because that was outside the scope of standard support!
The whole point of moving to SaaS was to not have to babysit our own clusters (small shop at the time), so spinning up a ton of infra in front of the freshly greenlit SaaS setup would have negated the productivity gains and financial pivot.
Ultimately, the entropy of hundreds of applications logging in disparate formats and namespaces outweighed our ability to sanitize each app within a reasonable amount of time, leading to unwanted data being indexed, ergo overages. Overages that our sales engineer originally assured us we could address by filtering things out with the snap of a finger. Bait and switch.
Ingest Actions were not available at the time, and were not functional (even in beta) until 10 months later.
If they do discount, even 5%, then it ripples across their accounts as a legal matter, esp at your scale. I was a buyer for some big companies, 8 digit, and the procurement office would only do a deal with MFN/MFC clause. They would also audit the supplier from time to time.
(This seems to be the repository in question, but it's been taken down: https://web.archive.org/web/20210104032001/https://github.co...)
On the other hand, the patent claims referenced in the lawsuit seem to me like great examples of software patents that ought to be struck down for being uselessly over-broad. For example, I would love to hear an argument as to how the "'433 Patent" wouldn't be infringed by running Wireshark in a Kubernetes pod. That meets every single one of the claimed elements that Splunk is claiming Cribl is infringing.
Presumably anyone with Wireshark could reverse it, so does it impart a significant advantage? Or is it just about control?
- Founder publishing a private protocol definition to help in building for it
- Sales staff sending account and prospect info to their new cribl email addresses before leaving Splunk
- Engineers leaving Splunk with technical specifications, such as their newer S2S protocol versions
The patent stuff is kind of whatever, but all three of those items would be enough to establish some very clear damages. Cribls an exciting new player but they can't take shortcuts like this, if the allegations are founded.
While I have jumped to competitors, I moved to roles that weren't in any form competition to my former team/role. That makes it easy, even if I would accidentally take things with me, I wouldn't be tempted to look at it, as there would be no point.
So yes, I take all my growth, knowledge and experience, but nothing that is really unique (say trade secrets) to old company would directly apply to my new role, so there has never been any problem. Once one is willing to jump to a competitor in a manner where you trade secret knowledge would benefit your role directly, one is creating a problem.
Non-compete clauses will try to limit the usefulness of the "in your mind" knowledge by restricting the domains in which you can work post-departure. It's my understanding that such clauses are generally held to be unenforceable except in an acquisition scenario.
Really egregious is taking the sales data. Business analytics around leads, customer satisfaction, pricing, etc are not the same as retaining general knowledge. If you left and remember the point of contact you had at a customer, that's allowed (barring non-solicitation agreements). If you leave and you take a list of customers, data that the business has generated about them, etc, that was never yours and it's not your knowledge. It's clearly the business's and there's usually dozens of people involved in the creation. That's clearly theft, especially since it was never yours to begin with.
I've been looking into Cribl and it seems their product has surpassed their competition as well but not in search, more in data summarization and log reduction, possibly before you ship it off to a more proper place like Splunk.
Splunk's cost makes it inaccessible to most people or companies. I mean, I work in infosec and I highly caution against Splunk because it is so amazing you will hate anything else but in security you need tons of otherwise rubbish data collected centrally sometimes and it will force you into a corner where you will say you can't afford to store that log you really should be storing. Better a crappy tool that can be used to find the logs you need than a nice tool that can only retain so much.
Cribl is supposed to help people reduce what they put i Splunk so they can keep using Splunk, it would have been nice if they partnered instead.
Graylog is another nice tool I like that is somewhat but only slightly similar to Cribl that was founded by a former Splunker out of frustration.
Last time I used it was almost a decade ago and it was rubbish, queries took 10-40 minutes to complete.
We were only sending a small subset of our logs to it so about 200+ GB a day. Our Linux box with spinning disks could grep the full set of logs much faster than querying Splunk, so I don't think anyone really used it.
Your queries or infrastructure were not optimized. It’s very fast when optimized.
It was Splunk managed and configured, so I would have thought it optimized, but I guess they made more money from it not being optimized.
If I remember right then we were throwing about 200+ GB at it a day.
It never got the love it deserved and I could absolutely believe that its Splunk cluster suffered as a result. RIP
But you have to learn to use it, if you don't give it an index and a sourcetype that will slow it down, and like ES leading wildcards slow things down. The fastest searches are simple terms like a word or an IP.
From the general responses it sounds like we got unlucky with a dud implementation.
I work as a Splunk integrator and here's what I often see:
1. Customer installs Splunk with a qualified Splunk or third-party architect team. The deployment works well.
2. Customer adds infrastructure to the deployment. Splunk slows down. License costs go up.
3. Customer chooses between outside help or DIY. DIY rarely works.
4. Customer now needs outside help. Now Splunk is very slow and expensive, and now it will cost a lot to tune it.
Splunk, the company, is in a tough spot for several reasons: rotating c-level cast, unpopular changes to license model, bad acquisitions. The product is still best in class but tough to keep optimized.
A firm with a competent IT team is unable to get splunk to work and only "outside help" can make the product work?
Given splunks license costs are tied to data ingested, how do you integrate new infrastructure to the deployment and not have license costs go up?
Way to sell us on Splunk?
We finally got rid of it a few years later, but for the entire time we had it, it was a constant "round hole square peg" problems. Each time the consultants assured us Splunk could do what we needed, each time it could not.
Just that it looks like most people here had a good experience and we had a bad one for some reason.
The guy we had help us tune our clusters after I rebuilt them all was also very good. Fortunately I'd done most everything by the books and we overkilled the nodes with hardware (we had some older hypervisor nodes lying around I stole for Splunk).
As an end user having used both to manage logs on a few dozen distributed applications I would never choose Splunk over Humio.
I'm with you. Splunk core - the indexing, automatic parsing, HA architecture, is unsurpassed. You can rebuild/duplicate parts of it but it's not going to come close to what Splunk can do, effortlessly, out of the box. I'm frustrated at the crud that Splunk has acquired which doesn't solve their customer's core problems. Splunk isn't well-rep in the network space. In my past I've worked for a huge tech company that was the darling of its day and Splunk business trajectory reminds me of that; we're within the start of the descent.
I read through the complaints in this thread, how it's slow, behemoth, hard to manage, copmlexities grow ... I've never experienced this problem. I've built and managed 3 Splunk clustered installations, in the 10sTB/day, and I will never use anything else. Sadly, that makes me only able to work for people able to afford the license :nervous laugh: So if you're made of money and want black car white glove data service, buy Splunk and hire people like me.
I want to take a CSV file and provide same functionality. Eg. Give user information on how many times each field occurs. For example, if it is a CSV file with cities, countries, continents, I want to aggregate and tell how many cities are in each country and how many countries are in each continent.
Is there an open source version of splunk I can modify? I tried logstash but it is not straight forward to work with. It still needs me to define schema everytime.
Thx!
https://github.com/grafana/loki might work for you. It’s not a drop in replacement for Splunk, FWIW.
They sent us an invoice for renewal in early August. I replied back (5 separate times) asking for the original contract (our ops department is tightening up on vendor management, didn't have it on file already); and we've heard nothing. Our service has continued to work despite not having paid (or signed a renewal), but we're switching to opsgenie.
> On March 24, 2017, a few months after his initial copying of Splunk’s source code, Mr. Sharp resigned from Splunk to co-found Cribl with Dritan Bitincka and Ledion Bitincka— both former software architects at Splunk.
Except that they didn't because initially the had created a company called diag.io that was focused on troubleshooting fault configurations.
Go Clint & Ledio!
With no details, hard to read this suit. Would need to know what evidence Splunk has that Clint Sharp stole source code. All the rest seems superfluous.
There are also various copyright claims on things in manuals, plus claims that they infringed numerous patents.
All in all, it sounds pretty bad, but lawsuits almost always do. I would wait to read the responses before coming to any conclusions.
Although Splunk provides HEC for third parties to use, Splunk maintains other aspects of its software as proprietary. One example of such proprietary software is the “S2S” protocol. S2S stands for “Splunk-to-Splunk,” and this is software that Splunk itself uses to send data to, or receive data from, Splunk Enterprise and other Splunk software and technologies. Splunk does not support use of S2S by third parties, does not publish S2S’s source code, and does not document S2S in a manner that facilitates third-party use of this protocol.
Mr. Sharp posted a derivation of Splunk’s proprietary and confidential S2S source code to his personal github webpage (a publicly accessible website for sharing source code). Mr. Sharp named this derived code “go-S2S.”
And it drains your bank account.
There's also Logz.io or you can use Elastic for an ES backend.
It's in beta and free. My plan is honestly to have my pricing be free for small amounts of data, and then 50% the price of Splunk for larger data sets. Just show me an invoice, and you'll pay half!
Unless splunk has a smoking gun it’s hard to really take their side here.