Hacker News Reader

Comment by jrochkind1 | Hacker News Reader

> Something important changed between the 90s and today. If you look at most open source projects now, the distribution of who’s doing the work versus who’s simply there is skewed dramatically: it’s common to see projects where 95% of the work is done by a nucleus of people, perhaps even a single developer, with a long tail of “contributors”

Any evidence this has actually changed between the 90s and now?

I have a strong suspicion that it's always been this way.

There are exceptional projects where the work is more distributed (is Linux an example?), but my suspicion is that they were exceptional in the 90s and still are, and that all along most projects have this "dramatic skew" as described.

For software of now, you can look at github. For software of the 90s, 2000s... I wonder if anyone has tried to research this?

I definitely don't see any reason to assume it was different without evidence.

View on HN

>Any evidence this has actually changed between the 90s and now?

Experience with the relevant communities (Gnome, KDE, and others, etc) in the 90s and 00s.

One reason for the change since then is that nowadays most things are more feature complete so low hanging fruits are not there (higher cost to entry for newcomers and fewer things to do if you're not "up there" as a developer), plus you need to "compete" or "collaborate" with a number of paid developers from companies like RedHat who handle most of the project and steer its direction wherever the company likes.

Plus FOSS just isn't as glamorous as it used to be:

The year of "Linux of the Desktop" (which as a vision meant mass adoption of Linux to win over Windows in desktops and laptops, not the "but Linux is fine for desktop use today, I've used it for years" or "but billions of Android phones use Linux behind the scenes" we settle'd for) never came.

"Big" companies stopped caring about Linux except as an server thing (e.g. we had Easel, Corel, and others competing for desktop Linux, even IBM and others had an interest, whereas VA Linux was one of the biggest IPO stories).

There are much less stories about the sexiness of FOSS or some FOSS future, and hackers changing everything working together, etc, in mainstream (not industry) media, etc.

Linux on the "fleet" desktop almost happened (some companies and schools do it), but then webapps got big and ChromeOS swept in. Now the only reason to make a non-ChromeOS "webtop" fleet is if your organization is explicitly Google-free for competitive/legal/political reasons.

Keep in mind that Linux is not the only free software project. Take a look at Blender (for example) which is turning the scene upside down.

And, on the last point: Was there ever any mainstream media coverage of FOSS?

You could call it the facebook effects nominally. The web is/was the key open technology of this era.

Collaborative efforts had a lot of successes. Wikipedia turned out to be a good encyclopedia, which seemed laughable/impossible initially. Linux, obviously. The www itself. Open, collaborative, organized outside of a corporate body. These were big, impressive wins. AOL and others who tried to compete with openness head on failed.

FB succeeded. Or rather, around that time a lot of these wins started being accumulated within a more standard corporate frame. FB controls who sees what and doesn't share. Twitter is a company that controls a protocol. Android is, philosophically and business-wise the anti-thesis to the goals you mentioned.

"Hackers changing everything working together" is now stuff that HR talks about... entirely coopted.

Fully agree with coldtea points.

As personal anecdote, I also transitioned from a period I still believed into "Linux of the Desktop" and strongly advocated it as such with M$ email signatures, back into Windows/macOS world, just keeping a survivor netbook still around.

FOSS dynamics and culture proved not to be the place to be, when trying to build business around desktop software or developer tools, GPL + commercial seem to be the only way that kind of works.

It used to be you could email a patch to someone and that patch would get reviewed and committed.

Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file.

Over time, this stuff filters out contributors and leaves a core of people who are comfortable with the structure of the engineering process for that specific project. It's not a bad thing that engineering standards are higher, but it's no longer a world of people 'scratching an itch' -- there's yak shaving to be done, and the number of people willing to shave that yak has always been smaller than the number of people who are not.

I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been.

> It used to be you could email a patch to someone and that patch would get reviewed and committed.

May I ask what communities you were involved in, and when?

Even formatting patches so they could be reliably applied was a nontrivial matter. It was not uncommon for them to be applied manually (and sometimes faultily). Sometimes you would mail patches to maintainers directly, other times you would share them on mailing lists, where they would undergo a bit of "social review" (works for me!) and would or would not be applied.

Some people just mailed entire modified files and expected maintainers to figure out the changes on their own.

As for "committed", many maintainers may not even have used source code control (notably Linus didn't use any), and I can't recall a single project with publicly accessible repositories in the 1990s as opposed to tarball releases.

> All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved [...]

Getting an account on any of these code forges in 2020 is almost certainly much easier than getting an e-mail account at all in the early 1990s for many people.

I don't agree with the author that contributing was easier in the 1990s. Generally, it involved a bit of a social process, trivial drive by fixes were not any more popular than today, there were coding standards to be observed (and not always written ones), in larger projects, you could expect to navigate some flame wars, and sometimes your contributions were rejected anyway because RMS decided to cancel your OS.

Very much mirrors my experience. Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues. No more mailing lists or bug trackers to sign up for and manage!

And it's also even got an integrated version control system, that 99.9% of projects then use, and it's one that makes it easy to version control a fork of your own as you work on any patches you might be making.

Progress!

> Very much mirrors my experience. Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues. No more mailing lists or bug trackers to sign up for and manage!

And no more need to become emotionally invested in the project, thus furthering its development into a proper community of users and contributors that propel it forward. :(

But what if I don't want to become emotionally invested in the project? It would still be beneficial for the project if it were easy for me to submit bug reports (and maybe bug fixes) anyway.

Emotional investment is always an option for those who want it.

And a lower barrier to entry might even increase the chances of it happening anyway, even for those that started out thinking they didn't care much.

> Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues.

Unless you value your privacy. If you take the necessary steps to approximate the level of "privacy in public"[1] that you get by default when dropping into someone's disconnected bugtracker or attaching a patch via email, then GitHub is pretty cumbersome to use. The article argues the case that GitHub is making life easier for people trying to build their personal brand, but not everybody wants that.

I don't use GitHub for myself and never have, but I used to make exceptions for projects that are hosted there[2]. Not anymore. There's enough toil involved trying to dodge the unwanted social hoopla that GitHub imposes on you, and the returns I've gotten out of "engaging with the community" are so, so low.

Nowadays if I report a bug or try to submit a patch and someone gestures towards GitHub and insists that it's not just the front door to the building but in fact the only way in, then I explain that the only circumstances where I use GitHub involves for-pay work, so unless there's someone I know I can send a bill to, then I'm perfectly content to just never go inside. But people in general seem to have a lot of trouble right now grappling with the notion that submitting a patch or a bug report is not something that I'm doing because it benefits me. (Imagine working as a plumber, showing up for a call wearing coveralls and carrying your toolbag, finding that you've arriving at a club, and then dealing with someone at the door who's trying to argue with you about their entry fee because, hey, this is a club, so they expect you to pay them.) The only thing people seem to understand anymore is the quest to become someone with influencer-level status, and that certainly has something to do with the culture referenced in the blog post. It would be one thing if it were just kids, but it's something that has infected plenty of old hats who were around before GitHub, which is pretty annoying.

1. https://www.colbyrussell.com/2019/02/15/what-happened-in-jan...

2. https://www.colbyrussell.com/2016/02/13/keeping-a-low-profil...

> It used to be you could email a patch to someone and that patch would get reviewed and committed.

Yes, but 1990 is also when this study came out: https://ftp.cs.wisc.edu/par-distr-sys/technical_papers/fuzz....

They find that a lot of UNIX tools could be easily crashed by generating random inputs. The quality of our most popular open source software has dramatically improved over the last 30 years.

Check the updated version of that paper, it hardly changed in 30 years.

https://www.researchgate.net/publication/348025458_The_Relev...

In my experience, bureaucracy is mostly uncorrelated with the willingness of a project to accept a PR from a new contributor.

Homebrew has very strict contribution requirements but has a huge number of contributors and is generally easy to contribute to. Meanwhile there are lots of little projects that go dead for lack of interest by the sole maintainer in reviewing and merging patches.

Also this:

Doesn't really make sense to me. How many projects have you actually had this problem with?

For some people, using Git and Github is a barrier to entry. For others, submitting a patch to a mailing list is a barrier to entry.

> dead for lack of interest by the sole maintainer in reviewing and merging patches

I think you mean "lack of action" rather than "lack of interest."

It's easy to be enthusiastic about developers doing lots of work for you (usually for free) and to cast them as "uninterested" if they don't comply quickly (or at all) with your requests.

I suspect many maintainers are "interested" in reviewing and merging patches but are simply unable to perform large amounts of work without pay, due to time and financial constraints.

You could re-frame 'bureaucratic process' as 'expectation setting'.

It also benefits the contributor to know what is expected of them. If you know that your contribution will be properly considered when it meets the outlined expectation, then it is easier to put in the effort to contribute.

Totally anecdotal, but I feel open source projects were much more community oriented in the 90s and the 00s. But after that, it seems they’re either driven by a company, or an individual. That would roughly conform to the article’s timeline of GitHub adoption.

Everything has more "individuals" now, since everyone can get a soapbox/megaphone, from YouTube to GitHub to Twitter to all the rest.

And it's possible to monetize that work now, with ads and micro-patrons, which is a huge drain on free and open source, mitigated only (partially) by the increased ability for open sourcerers to collaborate and that nowadays we're drowing in software and don't really need much more new software.

You don't need to join a team to get noticed (and on the ugly side, even though having a team helps, it's better for the Personal Brand to make that team invisible)

Counter-anecdote: most of the small open-source projects I use only have a small number of active contributors because there is a small number of people who actively try to contribute.

Projects that stop accepting PRs tend to be pronounced "dead" after just a few months, but there isn't always someone able/willing to fork it and keep the project going.

Maybe there are more projects to contribute to than before, so each project gets a smaller % of the "potential contributor" pool.

By "community-oriented", do you mean that the distribution of commits was such that more people were making more commits, instead of "95% of the work is done by a nucleus of people"?

I'm not sure if those are the same thing, I'm just talking about the distribution of work.

Another component, I think, is the difference between being able to work on whatever you want vs. having to follow a roadmap dictated by project leadership. Other commenters have touched on this (pointing out the decline of community orientation, rise of corporate oversight, or the fading glamour, for example), but no one has identified it directly.

The article points out that we collectively re-examined Brook's rule, going from "adding more engineers to a project makes it ship later" to "if you need to get more work done, go attract more contributors." I think the evidence shows it's more nuanced.

If you have a large amount of potential tasks and a large pool of potential contributors, and little strong preference for which tasks get done (OSS in the 90's maybe) then highly self-motivated people can pick up work they find interesting or which are useful for them. You can then avoid the exponential explosion of communication/coordination overhead by empowering those individual contributors to make their own decisions about how that work should be done. This is how OSS can scale to hundreds of contributors.

Once you introduce some sort of top-down planning that dictates which tasks are important, the blessed approaches, strict coding patterns etc. then you demotivate a large number of contributors who don't share the top-down priorities and you have to pay the exponential overhead cost of distributed coordination to enforce the top-down priorities.

Users of software care about their own workflows, their pet-features, and the bugs they hit. Some are willing to put up the work to patch in those features or fix those bugs and upstream them. Fewer still are people who are invested enough to dedicate themselves to a project as a part-time job.