The Cathedral and the Bizarre | Hacker News Reader

The Cathedral and the Bizarre

marktarver.com

107 pointsbronsa124 comments

It is perhaps useful to understand where the author is coming from.

The author is very critical of open source on the whole. He has a series of articles detailing criticisms of open source software.

http://marktarver.com/problems.html (The Problems of Open Source) http://marktarver.com/open.html (In Praise of Closed Source)

He is the inventor of the Qi (later Shen) language (essentially a Lisp dialect with a configurable type system). He experimented with taking it open source. He was very underwhelmed by the results and has made later revisions closed source again. In particular as far as I can tell he felt he was losing creative control of the project and getting little to nothing back for it in terms of contributions and features (and in general very unimpressed with people's reaction to the language and the wider programming community's capability of understanding the language).

He has also had at least one argument on Reddit about this (see the post by Mark_Tarver here https://www.reddit.com/r/programming/comments/57h33q/arguing...).

This is one article in a long series of events and thoughts for him that have played out over the past decade.

HN makes an appearance:

> This is the time when reddit made an appearance. I blessed the Shen subreddit because I thought that reddit was some polite ladies bookclub - obviously showing my ignorance of it. It is nothing of the sort of course and some of its most vicious sites like Coontown - the reddit for racists only recently closed - shows what is so often like. Reddit is a shit hole; and to a lesser degree so are SlashDot, and Hacker News.

> Unfortunately some members of the Shen group, imbibing the mindshare poison, gave much power away to the opinions on reddit.

> The OS fanatics relished their power to bully and shame and people used to mail me and say 'You need to say something because XYZ has said such and such on reddit'. I did on occasion call such people 'complete prats', but really this was whack-a-mole because the supply of arseholes was unending.

It sounds like this guy has some sort of axe to grind, but he's wrong. His argument rests on the claim that "most open source code is poor or unusable."

When most people refer to Open Source Software, they're referring to serious projects like the Linux kernel, Apache, PostreSQL, Firefox, etc. They're not referring to random crap on Github.

The serious projects are actually better than their closed-source counterparts, both in quality, and in their ability to be extended and customized. Customization is not going to be cheap or easy, but it will at least be possible.

The fact that I wrote a toy renderer in Rust five years ago and haven't touched it since then doesn't change this. The fact that there are 100 or 1000 times as many toy projects as there are serious projects doesn't change this either.

Ironically, he anticipated this objection, which many people beyond the parent poster are making, and answered it in the paragraph that contains:

"... Open source users will admit that a lot of open source is buggy abandonware. However they argue that this really does not matter since some small significant fraction is really quite good and that's the stuff we should use. ..."

Finding and dissecting the paragraph that contains the above sentence is left as an exercise to the reader.

As far as I can tell, he makes a weak argument by analogy

> This lesser-known fact is masked by the isolated points fallacy. The isolated points fallacy consists in taking the high scoring points on a graph and plotting your line on the basis of them. Hence open source champions wheel out the standard examples of success - Open Office, Wordpress and Red Hat Linux (we'll look at why some of these have succeeded later) - ignoring the vast sea of floating half submerged buggy and abandoned projects (over 120,000) that litter Github. It is the sort of technique Mugabe would have used for TV. If you're accused of starving the country, wheel out a handful of well nourished kids for people to see. 'Look, our country is fine; see how healthy these kids are'. Out in the slums the less fortunate die of cholera.

I don't buy his argument and I think he's confused. Lots of people have released code on github (as have I) so that other people can use it. Lots of it isn't used, so it looks abandoned. Maybe it is. So what? Why is that considered a failure of Open Source?

I think he's applying closed source's rules. A closed source project is a failure if no one uses (ie, buys) it, since it represents a development cost that wasn't compensated. Not so with open source. I don't suffer if no one uses my project, and the existence of my repositories doesn't detract from anyone else's work.

I agree. When I first started programming I put a program close to hello world on Github. I suppose that's counted as abandoned?

Also, I'm sure everyone's seen a closed source project that went nowhere?

A closed source project that went nowhere is less likely to be seen, as it often isn't released somewhere, I think.

I agree completely. I meant I expect many people have watched closed source projects die before they're released or be abandoned after release, but that is harder to measure in aggregate because it's much less likely to be seen.

So the author is wrong to compare all software on Github to closed source software he's heard about. Those "measurements" are completely different.

His point remains reasonable, though: If all you did was dump the resulting artifacts of a learning or self-edification exercise online, you didn't make a product like a Firefox or MySQL. And to the extent that people use your code, it's most likely as part of a similar journey to yours, and studying that code is not hugely different from studying books and whitepapers, even if the code is easier to leverage. But people were doing this before "open source" became a thing, and in a manner more befitting the role of such code: as part of a text file documenting the core concepts and posted on a BBS or Usenet, so that you could program the same thing. It's the kind of thing that would be a blogpost now. The actual code? Usually only the really deep, underlying algorithms need that spelled out.

It's only when you make a full system that the end user can get value out of that you get into the scenario he outlines. Somewhere in the middle in between the simple code dump and Firefox, open source tends to sink into a quagmire where it looks like a product but it's not supported like a product. And the only good answers then seem to be "go small" (impose LOC limits and hard constraints on features you ship so that maintenance drifts towards stability) or "commoditize" (become part of someone's product in a way that gets you paid).

Except he's arguing the fact that there's a bunch of useless code on Github means open source is bad. I argue that for the purpose of comparing the results of open and closed source development it's reasonable to restrict to software with a real user base.

> A closed source project is a failure if no one uses (ie, buys) it,

I don't think this is true. The vast majority of closed source programs are created in house for in house use. No one is ever offered it publicly so no one buys it either.

It's hard for people to see the scale though because there is usually no central repository.

> most open source code is poor or unusable

Is there any special reason to believe that closed source is any better? If we can count the sea of abandonware on Github, then we get to count the mountain of "interesting" code that remains internal to large companies, or worse, the stuff that actually got released on unsuspecting customers.

Generally, a business pays developers to write software that it expects to be put to immediate use, which means that most of the time it functions, at least minimally. I'm sure exceptions exist, of course.

FLOSS abandonware has no such constraint.

Generally, expectations do not define reality.

I'm not just being contrary, I've worked on a project for a year or two that got scrapped, and I think it happens a lot.

Another way to look at it, is that if you have programmers that are regular employees and not temps or something, you might as well have them work on something that's not very important, or that's risky, when you don't have something better for them to do, because you're paying them anyway.

I’m convinced if they stopped paying us all to rewrite things that work the market would fall out from under us.

Every job I’ve ever been involved with has always had some code base that was in framework X and now needs to be in framework Y, or on in house hardware and needs to move to the cloud, etc etc.

And having just interviewed every place has a project like that. “We were using PHP for the mvp but are moving to RoR”, “We were using class based react but now we’re using hooks”, etc etc.

I feel half of it is valuable migrations and half of it is to always have something on our plates.

And it always has to get done yesterday. What a world.

Most software engineers I know love to build software for the sake of itself. They are not all that interested in the business. Most businesses for which software is being produced is not personally relatable or interesting to most software engineers.

To keep engineers motivated, the work has to have sufficient amount technical accomplishments and not just churning out business features.

Luckily, due to continuous growth in market and scale of users, and improvements in hardware, there is opportunity for serious amount of technical work in any project.

Also, faster time to market is of real business value. Compromises are done in ideal perfect designs to meet timelines (or resource constraints) – hence, tech debt occurs and it is real and just as useful as financial debt to help scale a business in a competing marketplace.

And speculative/iterative market feedback based development is also real. All these things add up to one thing – things that were right for yesterday isn't right for tomorrow.

So we have to rewrite or refactor software continuously. And that's perfectly normal. It is actually one of the biggest strengths of software based solutions – that we can change things live - while it is already in use - change the engines of the rocket while it is flying - so to speak. We have to embrace this and leverage it disruptively.

Those who pine for perfect software are denying realities of the larger real-world.

My first six months of employment at my current job consisted of going from one project to another that the C-levels listed as "a top priority for the company"--like, called all-hands meetings and told everyone about this great thing and how important the people working on it were--and then decided to cancel.

Two years later, that still happens from time to time.

So, from my point of view, it looks much more like businesses pay developers to write software to be thrown away. Which I guess is fine with me, as long as my paycheck keeps coming....

Honestly, isn't it bothering you that they are wasting your time ?

Yes. But they give me a regular paycheck and extremely generous vacation and sick days, and nobody else I've interviewed with or been contacted by recruiters for can match those benefits... so I stay. At least until my RSUs vest.

Sounds like you work at a startup that was very young when you first joined.

If this is the case, did any of those projects contribute to you guys still being around now, two years later?

Nope. I work for a major corporation that owns its own entire office building for headquarters and operates in over 30 states.

None of those projects contributed to still being around 2 years later, but one of them (if simulations based on prior years' data are assumed to be reasonably accurate) would've improved revenue on one of our sales channels by approximately 30% if it had actually been put into production....

Closed-source crap is better because it's not available to the world, so millions of developers don't waste time exploring it and finding out that it's unusable.

> Is there any special reason to believe that closed source is any better?

As many (most?) of us on HN are in some sense or another professional programmers I would think that would be an easy question to answer.

In my opinion the vast bulk of closed source code suffers from exactly the same constraints.

And most definitely there are vast quantities of quite execrable code in closed source programs. I've seen plenty and written some.

Those things I agree with it. The good open-source programs mostly are better than other programs in these ways.

Yes there will be many badly written programs (I think someone said that 90% of anything will be not very good), but there is good stuff too. I think it may be the case even with the closed sources, but in that case they are unavailable so we will not have any statistics about it.

> 90% of anything will be not very good),

Theodore Sturgeon is usually credited with it but he was a little less polite and rather more definite, see: https://en.wikipedia.org/wiki/Sturgeon%27s_law

The author is wrong on innovation as well usability. One of his arguments is summarized as follows:

> [Open source] did not reward innovation. The most successful products of open source are knockoffs of old ideas; Linux included.

So what about Spark? What about Python numpy, pandas, and scikit-learn? Most people don't just use them because they are open source. They are innovative solutions to difficult problems.

> The fact that I wrote a toy renderer in Rust five years ago and haven't touched it since then doesn't change this.

I ask you in good faith...if you pushed this to a public github repo that was your reasoning? If it was something I found to be of value and you'd LICENSE'd it under a promiscuous license and I took that code and made a few million dollars from your idea (and kept all the money), would you be happy with that?

Was this just a thing to show potential employers you can code and use git, how does an employer even know actually you wrote that code?

> they're referring to serious projects like the Linux kernel, Apache, PostreSQL, Firefox, etc.

I feel you're missing the point here, not all the contributors to these projects are getting paid for their efforts. See his point about Linux Torvalds comment about:

This model of using people was acknowledged right from the beginning with a sly wink to Linus Torvalds from Eric S. Raymond in The Cathedral and the Bazaar.

Eric S. Raymond: "In fact, I think Linus’s cleverest and most consequential hack was not the construction of the Linux kernel itself, but rather his invention of the Linux development model. When I expressed this opinion in his presence once, he smiled and quietly repeated something he has often said: “I’m basically a very lazy person who likes to get credit for things other people actually do."

>...an article puts the abandonment rate of open source projects on Github at about 98% - meaning that there is no activity on 98% of projects after a year [0]. This has coined a phrase - abandonware.

I'm not a programmer, I'm a researcher. Let me tell you a fact: about 100% of research articles are not updated.

Somehow the link [0] cited in the article makes an indirect similar point, but what's wrong with programs which are not updated? After all there is a (small) group of people who keep telling that programs are the next level in mathematics, (ideally they are) better than proofs.

[0] https://www.techrepublic.com/article/open-source-failure-is-...

Moreover he's using the large amount of wreckage to argue that the Darwinian process of selecting "category killers" isn't working, which is completely backwards.

If "category killer" is a winner-take-all competition, then there are going to be a vast number of losers, and we do see a vast number of unmaintained projects. It's just that no one is bothering to remove them from view when they're defeated.

I'm unaware that the goal of evolution is to have one lion win the lion competition and eat all other lions.

Maybe we should stop using bad metaphors? Writing software is not a Darwinian process, neither is competition on the market place. Unless you got your OS by randomly twiddling bits in the binary file, in which case why are we even talking about open source?

> have one lion win the lion competition and eat all other lions

If that improved the chances of that lion's genes surviving, it would absolutely be selected for. Obviously it generally doesn't because there's safety in numbers, etc.

> the goal of evolution

Minor nit: evolution has no goal. It is undirected, random, capricious, etc.

The analogy might be clearer if instead of GitHub we used Fossil.

He does talk about how huge corporations use open source and exploit open source developers for their own profit...

Which is an important advantage to open-source. Abandoned, failed close-source software is almost never available. Talk about wasted productivity... If an abandoned github repo has any redeemable value, it will get found and used.

I thought proofs and programs were the same thing

Constructive proofs are equivalent to programs. But until somebody writes the program, there are always details, context, style, etc which make a proof understandable by a human but not a program. A program is more rigorous than a proof, in this sense.

The concept does need some qualification. “Small software” can probably become “done.” Grep, as an example, is done. Frameworks, libraries, and applications all have a steering wheel requirement of support.

I’d suggest that a fair amount of github open source is resume padding, or at least an attempt at it rather than “scratching a developer’s itch” which the original ESR paper mentioned as a reason.

I suspect most software is overdone, I was looking at Node Express the other day and it struck me they should have stopped when promises and async/await came along and moved on to another project.

Are there any numbers on closed source projects to compare this with? Say, if closed source projects have a 99% abandonment rate, what do these numbers tell us?

I agree. There are plenty of good points to consider in the article, but this is not one of them. Lots and lots of things on Github are little experiments that someone tried out and made available so that others can benefit from what they learned or tried. That can be called 'abandonware' in order to reinforce a straw man, or it could also be called a source of community learning.

> This has coined a phrase - abandonware.

Huh. "Abandonware" is a very old word as far as internet culture goes, referring to something entirely different.

I'm not a researcher, I'm a programmer.

The same abandonment happens within healthy corporations with proprietary source code.

There are even times when the source code is abandoned but the object code is in active use :-)

emulators make the object code portable :)

>Somehow the link [0] cited in the article makes an indirect similar point, but what's wrong with programs which are not updated?

Bit rot, for one (not keeping up with changing OSes, APIs they talk to, environments, libs they use, security concerns, etc), but also the fact that when they stopped updating they were still incomplete and more often than not half arsed.

Not to mention that abandoned github repos wind up in google searches and I have learned something from looking at them in such a context, even if untouched for years.

Don't delete your repos because they are "old". You could be doing a favor for some random person with a google search, looking at something you looked at too.

I’m saying nothing for or against the rest of the article, but this one sentence does not describe software as I know it: “But above all this is the sheer waste of human effort in terms of the production of rotting software in repositories.”

The act of writing code changes the programmer. I have written many things that didn’t turn into successful open source, nor did they act as resume-ware. But the act of writing them taught me something.

There is no waste of human effort involved when you practice your craft. This is like telling me that if I climb a cliff and then lower myself back to the ground, I haven’t gone anywhere.

Right now I’m working on implementing a subset of regular expressions that compile to finite-state recognizers. There is a zero-percent chance that my working code will be used in production.

So what? I am learning something about pattern-matching, and perhaps it will grow into a little language like SNOBOL.

http://raganwald.com/2019/09/21/regular-expressions.html

If that is a waste, then most of my most enjoyable moments programming were also a waste.

> But above all this is the sheer waste of human effort in terms of the production of rotting software in repositories.

Yup, what about the amount of human effort that has been wasted in the course of producing off-the-shelf proprietary software? How much of that proprietary stuff is now entirely useless, for some reason or another? It's just so weird to make an engineering-focused argument for proprietary stuff, when the only case that's even remotely plausible is about production incentives, and everything else hugely favors FLOSS.

As an aside, I'm pretty sure that, in the medium-to-long run, we'll manage to fund FLOSS development to an entirely satisfactory extent via crowdfunding, and that while saving a lot of dough compared to what it would take to fund proprietary development in the same way.

But in fact, even his point about support revenues is misguided. Well-designed and easy-to-use software does not really draw fewer paid-support arrangements than software that's hackish, unintuitive and the like. The nees for paid support is almost entirely an organizational one, so it matters zilch how easy the software is to support in reality; it's far more important that the organization can point to that support contract if any such need should arise.

I have like 30 projects on github, I don't expect any of them to really go anywhere, it's just where I put stuff when I'm playing with a new concept.

There are ~3 projects that I would have actually have liked to go somewhere, real projects not toys, that I have abandoned.

So that "98% of projects" statistic is fair, but I think those numbers would be looking a lot better if you accounted for the fact that github is just a place to throw stuff up. Just because something is on github doesn't mean it's an open source "project".

It seems like a silly metric. Almost all code gets abandoned, and at an alarming rate. Unless you happened to build a popular database or an operating system component, it’s likely that your code is gone in 10 years. And if not, your code is likely the future bane of some developer who despises maintaining it since it is such a bear to update to their unforeseeable requirements.

> I have like 30 projects on github, I don't expect any of them to really go anywhere, it's just where I put stuff when I'm playing with a new concept.

Then why put it there?

git works fine without github. And by putting your projects on github all you did was add to the baseline white noise that people have to filter through.

For backups, ease of access, and so potential clients and employers can see. Although most of my newer work resides on my privately hosted gitea.

Where should I put it? Or are you suggesting not sharing it at all?

My view is different from the person you're answering to, but - on an actual open-source based website that hasn't been bought by a closed-source transnational corporation ?

Lots of proprietary software are abandoned & unmaintained on old storage devices, the statistic about abandoned software on Github is meaningless.

On quality: quality is very suggestive. Quality changes if you factor in questions such as: Does it run well on low-end devices, does it track you, will it still open your files in 10 years, will the company still exist in 10 years, does it work well on many platforms, can it be automated, etc.

Writing software within old-school enterprises is a daunting task involving lots of proprietary systems that create a lock-in of sorts. Complex build and test systems, complex deployment and production systems are the norm. Weird tools and libraries with their own proprietary names and conventions increase the learning curve for any new-joinee. Some even have very proprietary non-standard language, runtime etc. making the skills acquired in such environments to be largely useless outside of such environments.

Open source culture changed all this drastically. This culture started with individuals loosely connected via Internet who collaborated on writing software and without much complex central infrastructure. Common conventions and standards emerged as these individuals openly discussed them and took with them to other projects – not just conventions and standards, but also the code. Without the enterprise lock-in, this has proven to be extremely successful and cost-effective.

Most commercial software companies today follow this new model for their internal software development practices. Using openly available technology components lets everyone to acquire portable skills that they can continue to improve throughout their careers across many different companies. This drastically reduces the cost of software production as cost of acquiring skills is amortized over many companies and cost of development effort of these open source standardized components is also amortized over many corporations/projects.

Every company I worked in has supported contributing to open-source with the selfish motive to attract talented engineers. And the engineers like contributing to open-source projects as a way of showing off their skills and building their resume.

Unfortunately that only leads to another kind of lock-in. The culture assumes everyone is using the same tools, so you are not allowed to use any kind of non mainstream software. And any smaller project using non mainstream libraries gets killed.

For example I use Mercurial rather than Git. Mercurial had better Windows support, a better command line syntax and with TortoiseHg a better Linux GUI. However, it happens all the time that linux distributions include wrong versions of Mercurial and TortoiseHg. You can use an old version of Mercurial and an old version of TortoiseHg, or a new version of Mercurial and an new version of TortoiseHg, but the distributions package a new version of Mercurial and an old version of TortoiseHg, and then TortoiseHg does not start. So they ship a completely broken package and no one cares because it is not a standard tool. (and last time I tried to install TortoiseHg from source separately, it also did not run, because was not compatible to the Python version installed by the distribution)

Recently I wanted to use a tool written in OCaml. It is really well written, has a GUI, using gnomecanvas. And it does not start anymore. Debian/Ubuntu has decided, gnomecanvas from gnome2 is deprecated and will no longer be installed. But the new gnome does not seem to have OCaml bindings. I am not aware of anyone working on a update to fix that. OCaml is too small to be allowed to make GUIs.

I write my software in FreePascal. After a system update, my projects do not start anymore on Android. Turns out binutils has enabled relro by default and FreePascal does not work with relro enabled. Imagine GCC could not create Android apps in the default settings. That would be fixed quickly, but with FreePascal no one cares

there's a lot of nits to pick about this essay, but the real punchline is that corporations have reaped the profits from open source software, far more than any developers do directly, and that claim seems hard to dispute.

I can't argue with this assertion. Can we start discussing how to correct this a bit. I have no issue with corporations reaping profits from open source software but would love to see more developers benefiting directly.

I think a lot of people have reaped the benefits of open source software. And corporations are formed of people.

It is a check and balance on proprietary software, in quality, cost, privacy and independence.

It has benefitted developers directly, in education, in transferrence of skills to other employers, and in direct customization of software for their personal needs.

> And corporations are formed of people.

So was the Nazi party.

The problem is with the distribution of benefits. Much like the vaulted Lebensraum the question is who gets the land and who gets to be fertilizer for the land.

So far the answer to the first is Bezos, and to the second is everyone who isn't Bezos. A distribution that leaves something to be desired of.

Utter nonsense. There are many people getting great benefit from Amazon other than bezos. Including many of people employed there and the many investors that have reaped benefit from the stock increase.

> There are many people getting great benefit from Amazon other than bezos. Including many of people employed there

https://www.google.com/search?q=amazon+warehouse+workers

The article is wrong in attributing "open source" to Eric Raymond alone. As it's now used, it was apparently coined by Christine Patterson and mutually agreed to at a meeting where Eric Raymond was present in February 5, 1998. [1]

According to the release history, "The Cathedral and the Bazaar" was written in 1997, but originally used the term "free software". The term was changed to "open source" on February 9th. [2]

[1] https://opensource.com/article/18/2/coining-term-open-source... [2] http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral...

"No mathematical theorem, no enjoyable novel, no work of art of any importance, have ever been produced by a herd. I fail to see why innovative software ought to play by a different set of rules."

Nobody tell this guy about Bourbaki... He's also probably never enjoyed a movie.

A novel also is produced by a “herd”. There are a lot of people involved. People that make suggestions, corrections, the editor... It’s not a single person effort.

Naked Came the Stranger was written, not just produced, by a herd, to massive success. (It was intended to fail, and the herd writing was an element of that intention, but none of that stopped it from experiencing massive success.)

See also: Atlanta Nights, another sort of "successful failure" group-written project meant to demonstrate that an alleged "publisher" was really a vanity press with no actual standards or editorial process. You can now buy the book via various links from the official website of the "author" [1], but in case you're in need of a few giggles right this minute, the submitted manuscript was also released by one of the conspirators [2].

[1] http://www.travistea.com/

[2] https://web.archive.org/web/20050305143929/http://www.cs.du....

Is that…all? It says 71,503 words at the top but I only see a hundred…

It's readable (y'know, technically) for me in Firefox 71. Maybe there's something weird about the PDF?

The single person is the one who does the conception and creation.

That proof-readers, editors, etc work on it too is inconsequential.

TFA didn't say more than one person can't contribute something or work an artistic project. It said that one or a very few people have the direction and the overall command and vision -- e.g. the opposite of "community process" or "design by committee".

However, you can still read a novel the same after someone else makes corrections or add things, even if the new things are not good.

Software can become complete broken by a bad patch.

That is seen the best at security issues. Like someone thinks it is a good idea to add a heartbeat extension to ssl. A basically pointless extension, if you have a novel with a pointless appendix from an editor, basically no one would read the appendix and it does not matter that it is there. But with software it ruins the entire server security completely

It is better to pass on 100 good patches than risking one bad patch

Well, to be fair, a textbook is not a theorem or an enjoyable novel... Also, a movie is not made by a “herd” (unless one thinks of a company as a herd.) Wikipedia would be a better example.

The Elements of Mathematics volumes weren't exactly textbooks and contained novel proofs. Also, the Bourbaki group published some papers containing novel theorems. For example:

https://en.wikipedia.org/wiki/Bourbaki%E2%80%93Witt_theorem

Bourbaki were certainly made up of a collection of leading mathematicians, but the works of Bourbaki are not innovations themselves. As much as I love the books, they do not contain major new results, rather they are an excellent organisation of pre-existing knowledge, perhaps with some novel proofs.

Meh. Bourbaki is merely indicative... Someone who believes that only individuals can produce theorems hasn't really worked in mathematics.

It's a no true Scotsman argument.

I guess Terry Tao would be a good example of collaborative mathematics? He was mentioned in another thread and I noticed the following while reading his wiki page:

> Within the field of mathematics, Tao is known for his collaboration with Ben J. Green of Oxford University; together they proved the Green–Tao theorem. Known for his collaborative mindset, by 2006, Tao had worked with over 30 others in his discoveries,[15] reaching 68 co-authors by October 2015.

Indeed! Erdos is another great example. 1200+ collaborative papers.

The Bible has something like 40 authors. It would probably be classified as a "work of art of importance", even if you don't classify it as anything more than that.

I would say of course, there is art to be found in it, but as a whole, it's obviously not a work of art simply because it's a bunch of unrelated and overlapping pieces, with arbitrary parts removed via historical accident, no matter how much people try to claim otherwise.

The Bible is a collection of many different texts; that is why it has many authors. While it can be evaluated as the whole collection, I think that each book of the Bible can be evaluated by its own, too. Actually, the book of Genesis does seem to come from multiple sources; also, it is believed that parts of the Gospels come from the Q Gospel, so even then there are different authors adding stuff within a single book.

First they said there was just one God, and the Bible contains "His" words. Then it turned out there were actually three, and one of them is edible and tastes just like crackers and wine. Now you're telling me there are 40? JFC, get your story straight, Christians!

Bourbaki summed, did not innovate.

Movies are also team work only in the sense of many people working under the vision of a single person: the director. But that's also true for any system designed by 1-2-3 people where 100s also work on.

Not all open source software is produced by a "herd"; many are written by one person.

My own projects I write by myself; often there are no other contributors at all (although I do accept bug reports and sometimes accept contributed code too, I am the only maintainer).

SQLite is a good open source software project (there are some proprietary extensions, but the core system is open source, and there are also open source extensions which are as good as the proprietary ones if you do not need the warranties and professional support they provide), and is written by only three people. Still, the commit history is public and I often look to see what changes they will have in the next version (and in one case, even found a bug and reported it, and they quickly corrected it).

I have heard Bourbaki before but didn't know much about that, but now I look it up and I can see what it is. I did not know that Bourbaki is actually a group of mathematicians, but I looked and now I know. I also did not know that the "dangerous bend" sign used by Knuth was invented by Bourbaki.

A main point of this seems to be that most projects on github are abandoned.

What a stupid comparison. That would be like listing all the commercial projects that go out of bussiness, or dont even progress far enough to get funding in the first place, and saying closed source model is a failure because most closed source projects fail.

This article seems to leave out why open-source changed it's meaning from "source available" to "source shared". That results from the FSF, GNU, copyleft and the rise of Linux.

⌘-F show only two references to GNU or Stallman, both derogatory references to Emacs. All references to Linux are about how early versions sucked and how it was a "copy" of Unix.

This highlights a lack of understanding from the author which may explain a lot of the viewpoints.

The harsh reality is that most software, no matter whether it's open source or closed, never makes it big. Take a stroll through an app store on iPhone-- or better yet, on Android! just to see how many apps there. You'll find hundreds of flashlight apps, calendar apps, and so on. Buggy, abandoned software is hardly unique to open source.

Then there is the argument that "open source isn't innovative." This seems like a really tired argument. Most software of any kind can be painted as just a mere modification or variation on something that someone once wrote in the past. We've had almost a century of software, after all. I also like the links to late-90s and 2000-era websites saying that Linux will never be innovative. The only way this could be better is if we could create a deep fake of Steve Ballmer saying it onstage.

But wait! Tensorflow is open source, so we can't use that, clearly. Financial deficiency disease and all. Luckily, I'm willing to make a special enterprise edition for this guy, at a modest charge.

Tensorflow is backed by Google, an advertising transnational corporation.

> In 1990 when you said a program was 'open source', you meant that you could read the source code; the actual code the person had written to create the program.

Citation needed. I don't think people were saying "open source" in 1990. FSF/GNU had the term "free software" for longer but they had to explain it on every first use. (Probably still do to an extent.)

> In fact an article puts the abandonment rate of open source projects on Github at about 98% - meaning that there is no activity on 98% of projects after a year. This has coined a phrase - abandonware.

That's factually incorrect. The term "abandonware" dates back to the 90s (long before github) and was originally used to describe software that was predominantly proprietary. Virtually all proprietary software ever written has become abandoned or will eventually become abandoned, but I guess that fact doesn't really square with what the author wants to say.

I am not really responding to the article, but to the general concept of the Cathedral and the Bazarr.

The main problem is this: in sufficiently complex programs, roughly associated with LOC, there becomes a point where there are no longer enough eyes on enough pieces of code.

Therefore, I propose that the future of FOSS in the Bazarr style is to focus on reducing code complexity and increase readability as much as possible.

“It’s not the daily increase but daily decrease. Hack away at the unessential.” - Bruce Lee

Cute title. But wow, this guy apparently does not live in the same universe as, I think, the rest of us.

I hadn't read ESRs paper in a few years, but just reread it again now. This author is definitely oversimplifying and mischaracterizing it. It's not right to conflate "Cathedral" with "Closed-Source" and "Commercial", nor to incorrectly imply that's what ESRs paper point was. Plenty of earlier Open Source software, notably FSF software itself, is (or was) developed "Cathedral" style and ESR definitely talks about that. I always read his paper as kind of "GNU FSF" process vs. "Linux", both being Open Source (or Free to please RMS). And I think history has proven his points rather nicely, IMHO.

But "Cathedral" and "Bazaar" are just two points on a spectrum anyway, and it's mostly orthogonal to a "Free/Open" vs. "Closed/Commercial" spectrum.

Now to the rest of the paper, while couched in formal and civil language, the author would have been much more succinct if he just came right out and said "Open Source Software Sux!". Which is an odd point to make for an academic having little experience writing either or any kind of real software.

I can see he thinks that, but there's not much in the way of actual arguments, other than opinions and cherry-picked sentences from ESR's paper, to make it worth picking apart or disagreeing with.

So yup, I fully agree: Dr Mark Tarver thinks that "Open Source Sux because Reasons".

Now then, I will just be getting back to the universe in which I live, where Open Source software is the foundation of the internet and just about every other piece of modern technology from doorbells to cars, including nearly all proprietary commercial products too...

> There's a larger group of not-so-harmless people than the givers who are driven by greed for free stuff and a sense of entitlement. We can call these the takers. Takers are generally abusive if their entitlement is challenged; because to criticise the open source model is to take away their intellectual candy and the result is a tantrum. Amongst this larger group are a smaller group of DRM crackers and pirates. In nearly all open source projects, they outnumber the givers. Michael 'Monty' Widenius, an open source advocate, acknowledges the problem.

> Help create a generation of takers that believed that all code should be open source and embraced open source as a religion. Online bullying of dissent began in forums. A large number of believers then traded 'code' for 'digitisable media' and the most active of those went in for DRM cracking and piracy.

Judging by the comments so far, I think he struck a nerve.

Not really, that would imply they read the article instead of the title.

You can definitely filter out the worst customers by raising prices, the bottom of any market is usually a cesspit. I suspect the very top is as well although I haven't any experience of that.

Unfinished experiments that aspire(d) to be something relevant are more like vaporware, not abandonware. I'd argue the vast majority of what's on github is this or dirty hacks that never warranted any further development or maintenance.

There's actually substantial value in access to such things. Rather than repeating the experiments yourself, you can go see where they led others without investing the time and energy to repeat the effort. If you find something that mostly resembles what you would have made, can get along with the license, and still wish to move forward - huzzah, a jump starting fork is born.

Abandonware IMHO is more appropriately applied to examples like Google's ever growing list of abandoned projects that actually had users, where the world noticed the abandonment of something useful.

That is a good point. Sometimes the code is useful even if it is incomplete. (I have sometimes done such things, taking parts of other programs; more often, I write most of the code myself, but some of it will come from other sources like that. As an example, I found the source code for a JPEG encoder, and modified it to improve the compression and other things. There are many other such cases too.)

The facts as presented by this article do not match the facts I remember. I'm an old fart and was actually alive and active writing software at that time.

The cathedral and the bazaar was not a book, it was an essay. They apparently later turned it into a book but I know nobody who read the book. The essay, in contrast, was read by everybody in the scene.

Here's how I remember it.

Open source has no specific meaning before esr claimed it. And after he claimed it, crucially, open source did not mean you could not charge for the software.

I practice almost no source code was released under non-liberal licenses, but there are some notable exceptions. For example, Microsoft released the source code of the Windows kernel for research use to universities. AFAIK that technically means Windows is open source.

esr was not against releasing source code under non-liberal licenses. He was just convinced those would not stand a chance against code released under liberal licenses. Time has proven him right, I would say.

Also this article misses the point, I think. Open source was not meant to define or enshrine a methodology. It was coined because at the time people felt that the free software movement was holding itself back in the fight against Microsoft (google "halloween documents" to get more insight into esr's struggle against Microsoft) because it sounded like you could not make money with it. Also because the GNU General Public License said if you make changes to a piece of GPL code, the result must also be under the GPL. So in a sense GPL actually did impede commercial exploitation of free software.

At the time many libertarians (like esr) felt that the market would solve this if you just let it, and software is not the place to fight ideological wars about the freedom of software.

I'm just paraphrasing the arguments here as I understood them. Personally I come down on the pro-GPL side of the aisle.

Before personal computers, wasn't free/open software the default because there wasn't a mass market for software? I think I've been told something like that by someone of an older generation.

Cathedral & Bizarre has never made sense to me.

What do people think of Social Architecture, Pieter Hintjens' ideas for building successful open source software?

Collective Code Construction Contract https://rfc.zeromq.org/spec:42/C4/

https://legacy.gitbook.com/book/hintjens/social-architecture...

http://hintjens.com/books

https://en.wikipedia.org/wiki/Pieter_Hintjens

Hintjens' worldview appeals to me.

I've done a bit of activism, with (very) modest success. A Design Patterns book club I started 20 years ago continues today; we somehow stumbled on a sustainable format.

Not having any successful OSS projects of my own (yet), I can't say if Hintjens' ideas work. I hope to find out.

I wonder how much of the software between me and that blog post is open source?

Some true points mixed in with a load of bullshit.

Yep, he's just torpedoing his own credibility with shit-nuggets like :

> What the world does not need is another buggy open source copy of Windows Media Player.

Is this supposed to refer to VLC ?!? (Admittedly that one is not in in the current version of his "manifesto".)

- Somehow using GitHub as an example that Darwinian competition doesn't work.

- Retconning the term "abandonware"

- The tired (and wrong) argument equating counterfeiting to theft. (No wonder that he thinks that mindshare is unrelated to making money !)

> Instead [F(L)OSS] produced a graveyard of dead software; the largest sink of wasted human labour in history - bigger than even the Mao's Great Leap Forward or the Stalin's building of the White Sea Canal.

And after all of this, he has the gall to call F(L)OSS proponent "Zealots" !

"Why do you look at the speck of sawdust in your brother's eye and pay no attention to the plank in your own eye?"

> Most open source is barely usable and empirical inspection of Github will show that to be true.

There is no barrier to entry on GitHub or similar repositories.

It's quite another thing to contribute to the Linux kernel, or gnu, or similar, which are overwhelmingly excellent, and an essential part of running the world as it's currently configured.

I've put some code I wrote up on GitHub. Why did I do it?

1. As a learning experience, how does GitHub work?

2. As advertisement for myself as software developer

3. As a check on myself, is the software good enough for others to use. When you do something publicly "on stage" it better be good. Not only good in terms of "code quality" but in terms of "does it do something useful?".

So I think I've got some benefits from publishing my code as open source. Not financially, like the corporations.

I think the world is still waiting for a better monetizing model for opens source, as well as for component-software in general. As things go Cloud, I think such a model will eventually if not soon emerge.

On the article, I found this to the point:

"The truth revealed that OpenSSL was seriously underfinanced with only one full time operative working on a code base of hundreds of thousands of lines of C."

I've got another one:

4. Breaking down that barrier of perfectionism that prevents you from publishing or finding worth in your creations.

HackerNews is open source, right? We share our ideas free of charge here. I'm not even aware of the copyright if any that applies to postings here. And we do it for fun, not for profit. But does it a great book make? Does it even make a magazine?

So I think the point of the article is that while Open Source has its uses, it is not and as far as we can see never will be a wholesale replacement for commercial software.

"However the problem is that the open source user may not stumble on this magical fraction and the invisible iceberg of buggy, ill-conceived open source lies submerged ... open source uses massive amounts of user time trawling through defunct and buggy applications and posting to forums ..." Regardless of what you are looking for (open source computer programs, closed source computer programs, television shows, books, etc) some won't be so good, or might be difference what you need for your use, and so they have reviews about such thing.

"An awful lot of popular open source is inferior reverse-engineered copies of existing commercial software (Gimp, OpenOffice etc)." Is Gimp a reverse-engineered copy of existing software? Just because it is similar to some proprietary software does not necessarily mean that it is a "reverse-engineered copy" of it. OpenOffice presumably involves the reverse-engineered formats of Microsoft Office, although now there is LibreOffice. Both Microsoft Office and LibreOffice each have some features the other lacks. What is "inferior" may also depend on your opinion, because in some cases it isn't so clear, and in some cases it may be inferior in some ways but superior in others.

"If open source programmers do innovate and their innovation is good then its just as likely to be swept away from them by the corporations who have the capital to exploit it." In my experience that doesn't always happen. Many features first implemented in GNU/Linux systems are things that they did not implement similar features in Windows until much later.

"... the Linux desktop still lags behind Windows and the interface looks stuck in the 90s." Actually, I think both in Linux and Windows many programs try to be too fancy, but in Linux I can change the window manager and all of that stuff easily and still works. I uninstalled the desktop environment, and I think it works much better than Windows ever did, actually.

"Good software is properly documented, does not break and does not require hand-holding to use it." True, but that isn't the issue; the issue is customizing it.

Love of money is the root of all evil.

I’m going to play devil’s advocate and suggest this essay is worth an open-minded read to anyone working in the software industry. The vast majority of comments that take issue with it in this thread thus far are nitpicks about minor points, and almost none address the essential thrusts of Tarver’s argument, all of which are actually quite cogent.

As he notes, it’s still the case today that the vast majority of successful open source projects are only those coming out of mega corporations that actually pay tons of smart engineers to spend time on them (fb, google, uber, et al). It happens every now and then, but it’s quite rare that some solitary programmer or duo whip up an os library or product that snags as much of the market as an os project managed by one of the big dogs.

Burnout on the part of solitary maintainers of non-corporate backed os projects is also quite a real and problematic issue.

The adoption of the MIT license, I’d wager, borders on a cultic practice these days and is rarely the outcome of long deliberation on the appropriate license selection on the part of a project maintainer (excluding of course the big companies, whom are incredibly savy and smart about this because they have lawyers).

Os projects also have, generally speaking, horrible support. If an os library isn’t small enough or written in such a way that I can read the source and understand it in about a day and a half, maximum, I won’t use it. The adoption of certain libraries also trends toward tribal practice and ritualistic behavior. People select librires based on their popularity or a personal incentive (these two motivations merge, since a thing’s being popular stirs a personal incentive to learn it in order to be more valuable on the market, see react et al.) more oft than they do based on a thorough technical assessment.

All in all, os has benefited massive corporations much more than it has benefited solo programmers or users. Tbh, I would not be surprised if a world w/o open source actually would have given us more diversity in the tech market—spawning several boutiques of software houses churning out small-scale bespoke, but closed source, tech instead of massive corps who not only use os as a “good grace” but also as a tool to create yet another relation of dependency between themselves and users or other companies—i.e. gain more power.

OS’s one invoiable benefit is that it has significantly liberated the auto-didact—its a huge boon for those that can’t learn through other channels...but even this benefit comes with wrinkles. One can’t know which of the practices one sees in os code are instances of good or bad technique without having external knowledge about programming techniques, which cannot come from code reading alone (unless you think a pattern’s frequency equates to it being a good practice, arguably a mistake in most cases).

This guy is so wrong. Truth is that innumerable closed source programs are abandoned (and that's actually what "abandonware" stands for). However, contrary to open-source, they're nowhere to be seen.

When you're depending upon some piece of closed source abandonware, you're without recourse. If by any chance, you'd be depending upon an abandoned open-source software, most probably it has been abandoned because there are better alternatives; or in case there aren't, you're still free to salvage it from oblivion.

The whole article reeks of bad faith.

What an absurd dichotomy. Even if you work full time on open source, you probably can't be a major contributor to more than a handful of large projects, at most; many people specialize in just one. But you probably use dozens to hundreds of open source programs. So for any project with a modicum of popularity, of course the number of people who use the software without contributing dwarfs the number of contributors. That would be true even if everyone worked full time on open source.

Admittedly, the OP's definition of "takers" is not just "people who use without contributing", but tacks on some ad hominem insults to supposedly qualify it. The result has very little resemblance to reality. Sure, there will always be some people who become truly abusive if a project makes a change they don't like. But they're definitely a minority. Most people, facing such a situation, don't speak up at all; they might be annoyed, but they keep it to themselves. Others do speak up, but know how to be polite about it. Perhaps the OP assumes that people should not voice negative opinions at all, if they're not contributing to a given project. I disagree. A good default assumption is that open source maintainers are open to polite and well-reasoned negative feedback. A maintainer can always forbid such feedback in official project forums if they so choose; if they do, users should respect it, but it's is not a very gracious move.

And what the heck do DRM cracking and piracy have to do with anything? In my experience, those people – whom I respect – form communities which are oriented around piracy, and pretty separate from the open source world. In part this is because the nature of the work they do is different; reverse engineering is different from software development, and many pirates do neither of those, but just use existing tools to distribute pirated copies of whatever type of media they're fans of (e.g. anime). I'd say they're also more likely (compared to programmers as a whole) to use cracked proprietary software instead of open source. Of course, this is a massive overgeneralization, but the point is that bringing them up makes no sense.

He should’ve counted the impacts of the open source software instead of the number of it. Does he think Apache web server with million or billions usage the same as one random open source software he found on GitHub and less than hundreds usage? Further, If he counts all of OSS on github and or sourceforge, he should’ve counted all crappy and abandoned closed software buried in any corporate office.

Old, large tapestries were often gigantic, complex, expensive, time-consuming, exquisitely intricate carpets that hung on walls. They could take from years to decades to complete, and consume fortunes. One of their primary functions was to show off how wealthy or important you were.

Now that we have machines that can do weaving and sew patterns automatically, it's possible to create a tapestry without as much time and expense. But until recent times it was time-consuming, laborious, difficult, and expensive.

One day we'll have machines to write code for us in minutes that would have taken weeks or months for humans, and we won't need to pay people to slave over keyboards to churn out pithy little programs and litter source code repositories all over the internet. We won't have to whine about the unkept documentation, or use complex systems that feed random data into test harnesses just to find the bugs that we accidentally put into the code. One day programmers will be obsolete.

Until the new machines come around to replace us, our software tapestries are going to remain really frigging difficult to make. So, yes, without proper financing, most of them will probably be abandoned.

What I'm wondering is: how many of the new machines already exist, but we can't use them because they're proprietary or patented?

well, there are tools for auto-generating code from schema files but no machines yet for creating whole applications. Based on my experience, I think the last frontier will be mapping the user-possibility-workflow space to action-consequence-application-state space.

He wrote this essay as if people haven’t made billions and billions of dollars from open source software.

I think he covers that a bit discussing Red Hat and other OS successes, and goes on to say "There's a larger group of not-so-harmless people than the givers who are driven by greed for free stuff and a sense of entitlement"

It's interesting to ponder the points made in this essay with the recent discussion about "FUSE for macOS is no longer open source" https://news.ycombinator.com/item?id=21759967

The FUSE situation would seem to be a key piece of evidence supporting Dr Mark Tarver's main thesis that writing open source generally doesn't pay or mostly pays corporations who are in a better position to monetize the value in the software. His linked essay at the bottom about Stallman states the Stallman offers no economic model that would pay a programmer for their labor and states that may be because he has no business experience.

ESR is the truth while this shill’s critique of the open source model is both incorrect and a waste of time.

I skimmed these responses, I'd say that most of them are the result of not reading the material in the essay or nit-picking or ad hominem remarks. I really tried to find something of real worth.

'His argument rests on the claim that "most open source code is poor or unusable." When most people refer to Open Source Software, they're referring to serious projects like the Linux kernel, Apache, PostreSQL, Firefox, etc. They're not referring to random crap on Github.

I'm afraid this in an attempt to redefine open source and is an example of the 'isolated points fallacy' covered in the essay. Open source includes everything with an open source license. To say OS is successful by defining all projects that do not succeed as not part of OS is simply to coin a tautology.

'Is there any special reason to believe that closed source is any better? If we can count the sea of abandonware on Github, then we get to count the mountain of "interesting" code that remains internal to large companies, or worse, the stuff that actually got released on unsuspecting customers.'

Again covered in the essay

'Proprietary software vendors typically make money by producing software that people want to use. This is a strong incentive to make it more usable. (It doesn’t always work: for example, Microsoft, Apple, and Adobe software sometimes becomes worse but remains dominant through network effects. But it works most of the time.)

With volunteer projects, though, any incentive is much weaker. The number of users rarely makes any financial difference to developers, and with freely redistributable software, it’s near-impossible to count users anyway. There are other incentives — impressing future employers, or getting your software included in a popular OS — but they’re rather oblique.'

What an absurd dichotomy. Even if you work full time on open source, you probably can't be a major contributor to more than a handful of large projects, at most; many people specialize in just one. But you probably use dozens to hundreds of open source programs. So for any project with a modicum of popularity, of course the number of people who use the software without contributing dwarfs the number of contributors. That would be true even if everyone worked full time on open source.

A giver is a person who gives as much or more than he takes. A taker is one who takes more than he gives. Define it in relation to open source as a whole and the dichotomy is valid. There are users and corporations who take far more from OS collectively than they give back. Is this hard to grasp?

'The vast majority of comments that take issue with it in this thread thus far are nitpicks about minor points, and almost none address the essential thrusts of Tarver’s argument, all of which are actually quite cogent.'

That's basically right and that's why after 20 years the OS movement is still stuck where ESR left it.

Reply on news.ycombinator.com