It’s a highly repeatable pattern. There should be a GitPrime for every industry.
It’s a highly repeatable pattern. There should be a GitPrime for every industry.
- This is only true if the security controls that your team, application, infrastructure has in place is matches the major cloud providers (i.e. Salesforce, Google, AWS, Microsoft). Even then, spreading your data around to 1,000 different SaaS vendors increases the surface area for attack/loss by 1000x.
"Trying to build a vertical analytics offering on top of OSS increases the level of difficulty by 100x"
- 100x is hyperbole, it significantly harder before OSS was focused on operations, but now there is an HA Helm chart, or even an K8s operator for most of the popular OSS components. It might still be slightly harder today, but organizations that want to pull insights from THEIR data often value the proprietary nature of that data.
If you look at his comment history, you'll notice he mentions Snowflake at every single opportunity. Snowflake raised $450M last October (~$1B valuation), so they have a nice warchest for strategic acquisitions.
[1] https://techcrunch.com/2019/02/19/google-acquires-cloud-migr...
[2] https://technical.ly/philly/2018/11/07/stitch-acquired-by-ta...
How do you measure 100x? How come it isn't 90x or 112x?
Does that mean what one engineer can do with a commercial database, you would need 100 people do the the same thing with OSS in the same time frame?
If you used a phrase like "difficulty on another level" or something descriptive like that instead, I'm sure people would be more interested to hear what you have to say.
- The pricing structures don't tend to be amenable to this operating model. Most of the newer SaaS ETL services like Fivetran and Alooma tend to structure their pricing discrimination/tiering to favor a higher volume of throughput through a low number of individual connectors/sources. At least the last time I was sourcing one of these solutions, trying to follow the above pattern (a high volume of individual connections with potentially low volume of events from each source) was almost impossible to negotiate without a substantial increase in cost for any given volume of events.
- The pricing structures also tend to be unfriendly to unpredictable and unknown event volume, particularly at the pre-launch phase. While that's not a problem if you're aiming towards the enterprise space or have investment capital to burn, it's a non-starter for people looking to bootstrap an idea that may or may not gain any market adoption. Industry- and domain-specific BI tools are inherently niche, and tend to have a market discovery phase as you align with that industry. Committing to minimal annual event volume for your data pipelines before you've even attempted to go to market pushes you towards rolling your own pipelines and away from using any of the tools like Fivetran that would have simplified the go to market process in the first place.
- Data security and legal compliance. It's quite possibly more of an issue with my current environment and client pool, which skews more towards the enterprise. But my current job is infinitely more difficult because leveraging SaaS-based ETL tools like Fivetran, Alooma, Funnel.io, Supermetrics, etc are all absolute non-starters. Between GDPR, the new California privacy laws, and all of the press on data leaks over the last few years, any solution that includes "sending internal data to external companies" or "providing credentials for internal data sources to external parties" triggers instant lawyer and IT security review. And that includes going all the way down the rabbit hole - if we try to use a vendor, then we need to know and audit who all they give access to, etc. Which basically means we have to roll our own pipelines and keep them internal or on a client's own cloud account. At least for a lot of interesting use cases for domain-specific BI tools, I could see that same caginess coming into play.
The security reviews when there are 3 parties are a pain. It can be overcome, the benefit of getting the data infrastructure automated just has to justify the extra sales effort.