OpenSSL is a famous example of an open source project that is massively depended on, including by billion dollar companies, and yet the company/project trying to maintain it was receiving almost no funding until an outcry a few years ago.
It might also be instructive to think about FreeBSD. It is much less popular than it used to be. Its code was used as part of the basis for the original Mac OS X release in 2001, but Apple's changes were made proprietary rather than contributed back to FreeBSD, and (arguably) as a result hardly anyone uses FreeBSD now, while millions use macOS. We can think of macOS as having been a competitor to FreeBSD in this sense.
Linux, on the other hand, requires contributions to be given back to the main project, and is doing much better.
Haven't heard that before. Do you have supporting data?
https://arstechnica.com/information-technology/2014/04/tech-...
We experimented with a lot of different business models, but the reality was ... no engineer was willing to pay us for something like "analytics" ... or "hosting" ... They were just going to keep using the code.
It's also incredibly difficult to track actual usage. There was no way to enforce the use of it on our CDN, so people were likely using our product, and we didn't know about it at all. When you open source, you just give up a lot of control. When you're end user are engineers, they won't pay unless it's something they can't do themselves, or aren't already using another service.
If you look at a company like SugarCRM (open source CRM product), businesses will pay. Granted, they aren't salesforce, but they have a legitimate business model that works for them. So while it can work, before making a decision, I would talk to your customers about what they would be willing to pay for, and how much.
The obvious one I can think of is where you sell (commodity) hardware + value-add software for a premium. Mobile apps is maybe another example which already suffers badly from cloning (without available source code), because the over-saturation makes success more or less luck.
From a business perspective, open-sourcing some code still make sense in many situations though. You may want to grow a community around it (Docker/Chef/Puppet/Ansible/Salt/etc), or gain market share by making it easy to install/try out (see previous, or PostgreSQL vs Oracle/MSSQL/IBM DB2), or recruit. In general, "business" is always going to underestimate the value of this and stray away, because it's taking a leap of faith (read: risk).
It gets interesting when pluses and minuses overlap. For example Arduino, would they have gotten as popular without being open source? Probably not. Are knock-offs cannibalizing their sales? A bit. Did that cause much loss? I don't know, I assume ARM boards and ESP8266 had more impact. So even though they're commodity hardware + software, the community part probably offsets the easy-to-clone pitfall.
Off-hand I can't think of any companies that have taken a successful proprietary product (code-base) and then open sourced it. If your main code-base and business around it are successful there's not much value to open sourcing it. It would be hard to convince any business that they should do something that increases risk. The "business" argument in favour of open source would be around contributors and the marketing effect - but if the product is already successful that's a weak argument.
In most circumstances software companies have open sourced proprietary code-bases because they were under threat and needed some sort of radical strategy. The canonical example is Netscape. There's really on a few examples - I don't think there's much evidence that they failed _because of_ open sourcing. The reality is they were under competitive pressure any way. Another big example is Sun though that's a nuanced story far beyond open source.
Then the last group is companies that are built as open source endeavours. Do they fail 'because' of open source more than proprietary software companies? It's a bit of a difficult to infer what would have happened if they weren't open source: how do you imagine Docker would be doing without being open source, VMWare is an example from an earlier age but the user audience and market dynamic are a bit different. My personal opinion would be that open source helps in the initial stages if your target user is technical [0], but that it does make it harder to commercialise and puts you under consistent risk of substitution or disintermediation. Community building isn't really dependant on being open source - look at GitHub, it serves the developer community who are pragmatists. The problem of commercialisation is that the economics aren't on your side and the customers buying behaviour is orientated on the proprietary model - users buy scarcity.
If you're interested there's quite a lot of good academic work on open source as a business model, searching for 'innovation and open source' should turn up some interesting papers. Like all business studies they all suffer from a small group of examples and a complex environment aka real life.
[0] In other words, if it's a normal user audience open source makes no difference. That's why line of business apps don't have any traction.
http://blog.balancedpayments.com/benefits-open-company-balan...
Whether or not that had anything to do with their demise, I couldn't yell you. Fantastic company though.
Sun was bought by Oracle, which is the opposite of open-sourcing the product.
Packer, Vagrant, Terraform...they are largely adopted and it makes Hashicorp in a strong position