Hacker News Reader

I work on one of those FLOSS projects that has several ostensibly-open-but-only-works-on-the-proprietary-instance sort of features baked into the core codebase. Some of them are dilutive, but some of them are actively subtractive, especially when they impact performance, become a blocking point during library upgrades, confuse our users, and generally make the codebase harder to reason about. This code is essentially dead code in the community offering, and we all know that dead code is not neutral, it's a liability.

Fortunately, our project is moving in the opposite direction than the one that the article describes: we have an independent steering committee & well-funded core team now, and lately have been actively trying to boot proprietary features out of the core offering. But it's a lot of work, both technically and socially, and we'd have much more time to spend on new features if it weren't for the dead weight we have to deal with.

More than one FLOSS developer has written of huge companies with thousands of programmers working for them practically crying and begging to have a feature added to software that serves only the company's interest.

Or even replying that they are busy with other things but if the company wants to pay for the change they can work on it, and company replying that that seems like "extortion", as if the developer is under moral obligation to fix Initrode's problems for free by the end of the week.

Yeah, our situation isn't quite that bad. The big company is happy to write the feature usually, but that doesn't mean the feature should be bolted into the main repo with its special dependencies dropped right into setup.py install_requires.

> "...huge companies with thousands of programmers working for them practically crying and begging to have a feature added to software..."

One clueless employee, almost certainly without the knowledge of his cow-orkers or managers let alone any kind of authorization to represent their employer when communicating externally, writes a dumb email to a FOSS project and suddenly the tech commentariat is all abuzz about "$BIGCORP (the whole thing) is 'practically crying and begging' to $FOSS_PROJECT for a feature"? Surely there must be a name for this fallacy and, if not, there should be. The organizational hivemind fallacy perhaps?

Many mega corps have a culture of bullying the vendors. That can easily spill over to a single interaction.

Yeah but the point is that megacorps have dedicated teams of specialists to do that kind of bullying and they do it with organizational consensus and coordination between relevant departments. No random individual engineer is ever going to be authorized to do that.

What I meant was that when an engineering group needs a fix from a FOSS developer, the natural thing to do is to bully and beg.

One reason beside bad culture is another aspect of bad culture. If they want a fix to a piece of FOSS, probably a library, they might have to beg their management for weeks or months, instead of just specifying that their project requires lib-from-overworked-dev-in-Nebraska-who-is-thinking-of-taking-up-alpaca-ranching-instead version 2.44 or later, which is what they can get if they beg, cry, and/or bully said alpaca contemplating developer.

I would be very interested in learning how to go about getting a well funded core team for an open source project (libreqos.io). We are increasingly popular, but still lack a business model. I too tend to reject most friction-creating and value-subtracting processes, but the top ramen is running low.

When you find out, please share. There's dozens of us.

I've been slow walking my FOSS releases because I have no idea how to squeeze some ramen out of it. Though my (dare I suggest) paradigm/category busting project was really hard (for me) to figure out, the end result is simplicity itself. And so therefore trivial to clone, fork, run away.

Possessing the business acumen of plankton, I've been swindled, plagiarized, and mooted a few times before. I don't need much more than ramen money. But not getting anything while others profit from my work, again, would really suck.

You should not do FOSS if you do not want to make a gift, with few strings attached --if any. Use a proprietary license instead :]

Interesting. Where can I read more about FOSS as gifts?

Happy to share, although I don't know how applicable the journey is to most.

The project started as nonprofit with both a business goal (sustain itself by selling online courses) and a philanthropic vision (essentially: increase education access by disrupting the degree market & also open sourcing the code).

A decade later, the nonprofit sold the entire online course business, its brand, and most of its staffing to a for-profit company. It kept the copyright to the open source code, and the remaining staff now use the proceeds of the deal as an endowment to pusue the original philanthropic mission, which includes being a sort of core team for that open source code.

Some details [0]. The online course business is edx.org, the buyer was 2U, the source code is Open edX [1] and the nonprofit started as edX Inc and was renamed to Axim [2] after the sale.

0. https://press.edx.org/2u-inc.-and-edx-complete-industry-rede...

1. http://github.com/openedx

2. http://axim.org

TL;DR: Start off as a nonprofit, build a valuable product, sell the product, and then use the money to continue the nonprofit. YMMV ;)

EDIT: Probably obvious, but I should note that this didn't all just "happen"--there were (and are) extremely passionate people on both sides of the sale and in the project's open source community who worked hard & advocated loudly for years, ensuring that both the nonprofit built a compelling product and the open source project thrived. Both of those were critical ingredients in the project getting to its current well-funded state.

I think strip mining is a good analogy. Like google has strip mined the internet and left a toxic pit behind, if companies have their way they'll do the same to any captive open source project, turing any public parts into nothing but minimum viable bait to try and get people to pay for something.

> left a toxic pit behind

Nonsense. If google scrapes your site, your site is not affected. Where is the toxic pit, exactly?

By becoming the de facto entry point into the internet, Google's existence has spawned a horde of SEO spam sites that dilute and push down real content on the internet.

You only see it if you use Google, and you are always free to not use it. They may be evil etc but this logic is like "I can't stop punching myself".

Not remotely true, you see it every time you go to a SEO optimized site, however you get there.

And how, may I ask, do you end up on all those SEO optimized sites?

> Google's existence has spawned a horde of SEO spam sites that dilute and push down real content on the internet

They don't push real content "down the internet" they push it down the google rankings.

And while Google search is not as great as it was, it remains a strong contender for the best way to find stuff on the internet.

I don't think the internet would be better off without it.

You can however make good arguments for Google News, "Instant Answers" in Search, and other products, taking (impressions) from the internet.(And Chrome, Chrome worries me.)

> They don't push real content "down the internet" they push it down the google rankings.

From experience, many people can truly not tell the difference.

> Like google has strip mines the internet and left a toxic pit behind

Can you clarify what you mean? I think there's a sense in which sometimes people say that google ruined the internet b/c (a) their prominence and their algorithm induced the creation of awful content farms, and (b) they pulled ad revenue away from some businesses that create content leaving us with a more impoverished web experience.

If you're making a different kind of assertion, please elaborate on it.

Wrt (a), I think site owners that had useful content weren't for the most part pushed to remove it, but were often pushed to pad it to the point that it feels less valuable to users (like recipe sites where every recipe is 12 paragraphs of prose before the actual recipe, IIUC), so I think this still fits in with the "dilution" framing, rather than removing valuable material in a way that excludes others from its use.

Wrt (b) I do think that when google displays content on the search page in a way that stops most users from continuing on to the page from which the information was sourced, that does seem effectively extractive. But I think that's distinct from the OSS project-capture problem.

> if companies have their way they'll do the same to any captive open source project, turing any public parts into nothing but minimum viable bait to try and get people to pay for something.

I actually think google is a good illustration of the full spectrum of OSS projects. They're definitely "getting their way" but:

- gson, guice, protobuf, snappy, the go language are all examples of projects that are very useful and pretty separate from any revenue-generating google product

- kubernetes, istio, etc are projects that can easily be used without touching any google revenue-generating project, but if you're using some of their revenue-generating projects, you may likely want something like this. Perhaps this is OSS but writing for your paying customer-base as a target audience?

- android, chrome, tensorflow are all about building little googleverse of customers in their orbit

I'm not GP and I personally think the strip-mining analogy is bad. But attempting to steelman GP's position based on other things I've read that I've pieced together:

I think an example is LLMs like ChatGPT. Basically consider a website that has good info on it. The LLM scrapes the site and regurgitates it's content to their user, and the original site never get a visit or recognition. The end result is the death of the site (much like strip mining results in).

Google didn’t break the internet. They failed to execute on social and rewarded shitty players with traffic.

Since the 80s or earlier, companies always sought to exert control over their audience. Things like Compuserv were early entrants, followed by Prodigy and eventually AOL and Microsoft’s entry - MSN.

Google made the network aspect of the internet better and ended up being a sort of gatekeeper. Facebook reinvented AOL, and a thousand copycats did variants of the same… Discord, Instagram, Twitter, Reddit.

Social wounded the internet. If anything “killed the internet”, it was Reddit and Twitter, cancer-like services both of which metastasized to destroy small-time internet properties, only to consolidate them into poorly managed, unprofitable companies.

I don't know if "killed" is the good word, but simply block google.com in your /etc/host and you will see how broken surfing on the web now is. Google is now technically owning the internet and is clearly using this position to its advantage, not to the advantage of its users.

I disagree, the analogy is weak. Strip mining is about harvesting naturally occurring resources in the most invasive way. The internet is not a natural resource. It was created by and for humans, from zero to unfathomable scale in thirty quick years. It is a reflection of ourselves, our systems, and our social and financial incentives. Google definitely deserves some blame as they certainly have helped shape those incentives, but it's not like they came in and started dynamiting some pristine wilderness, do you remember what search was like before Google came along?

I think strip mining is a decent analogy as well, so long as you include the fate of the participants. Take diamond mining as an example. The miners themselves tend to be dirt poor, and searched for stuff of value on exiting the mine, yet the outcome is big money for the corp controlling the mine.

> Like google has strip mined the internet and left a toxic pit behind

Sorry, what?

For a certain use-case, I'd say I agree. I would argue that the basic web browsing experience for the less technically minded is worse than it was in the past.

For example, I am still working with my partner to get her to recognize and ignore the top level garbage: sponsored links and obvious SEO bait* when she blasts off a "Best <kitchen widget> 2023" search.

*You know, those listicles offered up from a handful of different sites that tend to be a 'top ten' list of a selection of 15 or so different products that are mostly just 7 actual different products white labeled by some brand and sold on amazon through a referral link in the listicle.

SEO existed before Google.

Yeah, but back then we were all just cramming "Britney Spears" into our Meta Keywords tags; we weren't actually eroding the quality of our entire web properties to court a live-or-die relationship with a single traffic vendor.

Web is not limited like physical real estate and SEO spam does not take away some virtual land from useful content. If you think Google is useless and full of spam the best way to deal with it is to simply not use Google

Seriously, if there any organization that has done more to advance the web, IDK who it is.

The toxic pit in this case is the transformation that occurred "ecologically" to the way contributors and agents on the web interact with eachother. This was strongly affected by the major centers of gravity in the 2010s like google and facebook. Unfortunately, their stewardship was focused around leveraging centralization towards profit rather than towards making the web a better place for users. And so here we are today.

FTR, I think youtube is one (among a few) exceptions to this overall trend from google.

A bold claim considering the shitshow that is ContentID ? (To be fair to Google, they were stuck between a rock an a hard place...)

The way ContentID is put to use may be the best interpretation of copyright. You get to reuse someone's work and the original author gets paid. There are some false flags reported with it but it's hard to say how many are due to actual violations (when you legally license a song from me but you didn't know I illegally sampled Taylor Swift and so you go complain about YouTube when your video is demonitized). I heard the bigger issue is false DMCA takedowns by shady companies who don't participate in Content ID...

I though that false DMCA takedowns were punished quite heavily ??

Well if the victim is in another country or an average person without money for a lawyer what can be done?

Then it's that country that starts legal pursuits against them ? The average person only has to complain to the authorities, they don't need to sue the company, don't they ?

- YouTube is in US

- US company false DMCA's your video

- you are in Nigeria for example, whom do you complain?

- YouTube subject to US laws and must comply, while you can't do much

- YouTube probs doesn't have much incentive to investigate or sue false DMCA

If that becomes an issue, you can just stop using platforms like YouTube.

> To the contrary, building off the OSS ecosystem can make it healthier, if for no other reason than they are cultivating more engineers that know how to use these tools. "Extraction" is not the right metaphor.

It can, but it often does the opposite. "Extraction" may not be a metaphor that's quite true to the material. But it's very true to the attitude with which many companies operate, and that's because a lot of business culture was developed in extractive contexts and then applied elsewhere.

The strip mining imagery landed for me when focusing on the trend of open source communities adjacent to cloud platforms. Citus, ElasticSearch, Kubernetes, etc all feel like corporate goliaths forked and outcompeted David. No analogy is perfect, but I can see facets of why one might liken this to strip mining.

How does David vs. Goliath apply to any of these projects?

Kubernetes was developed, paid for and released by Google and loosely inspired by Borg.

ElasticSearch meanwhile wouldn't exist without Lucene which was again paid for and released by Excite in 1999, and who were very much not a David at the time.

PostgreSQL came out of Berkeley.. is the idea that David is the PostgreSQL Global Development Group (which includes Microsoft, Google and Amazon employees)? Or maybe it's EDB (https://www.enterprisedb.com/)?

I actually like the metaphor, although I agree it's flawed.

> "Extraction" in the literal sense of pulling something out, when dealing with physical material means that others cannot have what you've pulled out; it's gone.

For instance, look at the amount of brainpower wasted on using tech to steal our attention via adtech (and its supporting industries), large companies using OSS to increase their monopolies instead of giving back, small companies trying to build nothing of immediate value, but rather blitz-scale so they can get sold at valuations not reflecting their value.

Potential value extraction of OSS really depends on the license; the (A)GPL vs. MIT-style license debate has been going for decades now. There’s a reason none of the big corporations touch anything GPL, aside from Linux distro subdivisions that are never their direct money makers. And when you compare what actually reaches consumers, like Qt vs. GTK for an ancient but ongoing example, or basically every Android distro pre installed on a consumer phone, and the various modern MIT-licensed FE tools (React, Flutter, etc.)… the apparent levels of nefariousness/data mining/anti-competitiveness/price-gouging generally seem higher for the MIT-licensed (and especially corporate-controlled) products than (A)GPL products. (Yes, there is also definitely a trade off in terms of easy UX between these!)

Often these open-source projects have their own commercial offerings to support development, e.g. Elasticstack, MongoDB. And then AWS destroys that, without offering any contributions themselves. So it's no longer commercially viable for the original organization, and development suffers.

That's a net negative result.

I think open source users need to get more serious about using a more pro consumer license like the gnu gpl. If you're using less restricting licensing you're working for Amazon for free.

I agree but this blog post is hardly about that. It’s way less “companies bad, OSS good” as I expected it to be after reading your comment.

>A company (VC-backed or otherwise) that starts from OSS tools (operating system, languages, build tools, application frameworks, etc) to build their own offering doesn't (need to) remove that value in a way which excludes anyone else from enjoying it

They don't need to, until they realize people aren't upgrading to the added layers of offerings and then start removing that value. This is required because the base value tends to be substantial enough to swiftly encourage people to use it, and VC loves the growth this comes with.

Most company use of OSS isn't a freemium model, but instead them using it for their infrastructure in complements to their ultimate product. Think Google's use of Linux on their servers, or their use of gcc/clang to compile their search application.

Disagree on that. It is the freemium model that's used. Specifically enterprise tiers for the vast majority of OSS.

There is a crucial subsection of OSS used within companies but those often come with support contracts and branding which is what you're referring to with Google, but a far cry from "most".

No, like, how many companies are paying support contracts for Nginx, anything you npm install, anything their engineers 'brew install'. Some of these might have support contracts that you can buy, but that's not used in the vast majority of cases.

I really do think you're not seeing the forest through the trees here.

I’m launching something soon to try to reverse this and give back power to open source creators, around their earnings and, to bring a bit of order to an informal marketplace that really needs it. If you’re interested, join the wait list / launch email list and you’ll get to be among the first to use it: https://ash6wpkw.paperform.co/

> I find this analysis suffers for using metaphors about the physical world with software. Strip-mining is a loaded term

The analysis isn’t suffering from it but rather that is the analysis.

The authors whole point is that these foss communities and projects will die as a result

Which FLOSS projects did pyTorch and Tensorflow -- the libraries behind the hottest companies today -- strip mine? Both come from venture-backed companies.

And they have no dependencies? Despite being known for being hellish to install due to how badly they handle dependencies…