Hacker News Reader

You cannot reasonably expect people to check into a plethora of information streams, digest it, and be able to come to a meaningful consensus of action.

The meetings, the management, the prioritization, they all act as information filters of a sort. Otherwise, you experience death by information overload.

The most "open" of open source and remote companies still have meetings, product roadmaps, and prioritization. Resources and time are finite.

How does Linux do it? Does anyone know of any documentation for the process they follow?

Linux doesn't solve nearly the same set of problems that Windows does. One is a kernel. The other is an operating system.

Typically this is just semantics, but it's meaningful here because you still need all of the other components (GNOME and X and PulseAudio and SystemD and so on, as concrete motivating examples) to make up the whole OS.

And unsurprisingly, you end up losing out on features that would require vertical integration. Accessibility on Linux is more or less a non-starter, the design of every application and toolkit is blindingly variable, nothing about the system can be depended on (even the standard C library!) unless your app has been intentionally built and tested with that particular version of that grab bag of components.

Windows is one thing. People know what they're getting when they use it. I don't like it much, but it certainly achieves the goal it set out to accomplish. I certainly don't lie awake in bed wondering why everyone uses it over my favorite grab bag of widgets that more or less looks like an OS.

I think someday we might need to have systemd-accessibilityd at this rate

Or we could pick one UI toolkit and stick with it!

Kidding, I know that'll never happen. (Unless Electron counts as a UI toolkit, because that's apparently where we're headed.)

See the Linux documentation's "How the development process works" [0].

The way a changeset goes from a developer's machine to the root is reasonably similar (it usually gets pulled through a number a intermediate trees, from a development tree to a sub-subsystem tree to a major subsystem tree to Linus's tree), riding the train from the bottom to the top, with an additional time constraint on it (the merge window being open).

The kernel also has a "next" tree that is a snapshot of what the kernel would look like with incoming changes merged right now, surfacing early the exact long-distance coordination issues described in the article. Plus, of course, everything is in the open, so maintainers of different subsystem can then coordinate directly on things that impact both sides, even if the required patches will if possible make their way up separately.

[0]: https://www.kernel.org/doc/html/latest/process/2.Process.htm...

What you say is true, but the open source model does make it easier for people to subscribe to a subset of threads, issues, projects, etc that they find relevant to their goals, and to follow and contribute to those selectively.

In other words: core project team(s) may follow traditional organizational practices, but it's also possible for them to accept contributions from pro-bono specialist interests.

(that's theoretically possible for companies using inner-source approaches too, although the audience size in those situations is likely to be smaller)

Are you aware of any open source project of the size and magnitude of Windows (including and especially the number of engineers involved?)

And I don't think Linux is a possible answer here, because remember Vista/Longhorn pretty much started as a bottoms-up rewrite (I'm sure there is a legion of blog posts out there about why THAT is a bad idea too), and also involved in the scope of "windows" is all the applications that will ship with it.

I agree with you that in the context of an open source development framework, you would never need 24 people to design a menu, but can you even build something the size of Windows in an open source framework on any sensible timeline?

My impression of the most impressive and successful open source projects - even large ones - is that they are a labour of LOVE of one specific genius who eventually figures out how to scale themselves for ongoing maintainace. But it is almost certainly at that time a) their personal passion, b) not their paid profession, c) not the key contributor to the stock price of a major billion-dollar multi-national company.

This is specifically addressed in the appendix of later editions of The Cathedral And The Bazaar. I think what people are missing is that traditional project management does not deliver projects any faster or better. "Oh, it's really big and complicated, so all this inefficient process is necessary!" But then you bring in a Six Sigma or Lean or Agile consultant, and they look around and go, "What the fuck are you people doing?" They write up a list of recommendations and leave, and then the corporation does exactly jack shit with those recommendations. But many of the recommendations are to stop doing so much bullshit process, because it's often making coordination on that big complicated thing even harder.

Also addressed by the paper is the fact that, yes, Open Source does benefit from people with very big brains motivated to do what they love. The traditional model of "throw 100 random engineers at the problem" is simply not going to result in the efficiency of 10 highly motivated, highly talented individuals. But that doesn't mean the Open Source way wouldn't result in just as successful a software product. The difference is in how you motivate people, and how to get the hell out of their way so they can be effective.

The closest thing I can think of is the Debian Project, which does have an organizational structure based on committees: https://www.debian.org/intro/organization. Debian is the base on which most Linux-based operating systems build themselves. A lot appears to be done by resolutions, which has its own set of problems. It's at least a counterpoint to the idea of the most successful projects being led by the personality of one person, as leadership has changed regularly.

And this is why Open Source has managed to produce paradigmatically beautiful user interfaces that everyone loves and chooses over chaotic proprietary ones.

No, wait, somehow that hasn't happened?

I was kind of hoping this was some form of parody/sarcasm, but it appears not. The open source development model does not address this in any meaningful fashion.

You are suggesting that the solution to communication problems is to force everybody to have complete knowledge of everything. That definitely doesn't scale.

Remember the description given of the bazaar is "a great babbling bazaar of differing agendas and approaches out of which a coherent and stable system could seemingly emerge only by a succession of miracles"

which turns out to be true a lot of the time. Large open source projects don't produce stable systems without some form of additional social order imposed. They become less stable as they grow larger, and either someone notices and attempts to impose various forms of non-bazaar-like order (as happened to the linux kernel, for example), or the project fails/becomes a mess (i will avoid slagging on projects here :P)

Beyond that, the way open source addresses the problems listed in the article is by ignoring them. That is by design.

"Differences between the two styles of development, according to Bar and Fogel, are in general the handling (and creation) of bug reports and feature requests, and the constraints under which the programmers are working.[2] In closed-source software development, the programmers are often spending a lot of time dealing with and creating bug reports, as well as handling feature requests. This time is spent on creating and prioritizing further development plans. This leads to part of the development team spending a lot of time on these issues, and not on the actual development. Also, in closed-source projects, the development teams must often work under management-related constraints (such as deadlines, budgets, etc.) that interfere with technical issues of the software."

IE In closed source systems, time is spent thinking, planning, and prioritizing. Also, fixing bugs and handling feature requests. The bazaar says "don't do that, doing that takes away from focusing on technical goodness" This is true. The strict bazaar model prioritizes the technical goodness over everything. Unfortunately, technical goodness does not create good products. Thinking, planning, and prioritization do :). See, e.g., cascade of attention deficit teenagers (https://www.jwz.org/doc/cadt.html).

This is also why you rarely, if ever see straight bazaar models behind successful consumer-facing products. Good products require careful coordinated curation and choice. You do see straight bazaar models occasionally in infrastructure, but even there, not for large projects.

If you want to see the bazaar method in practice to products, you don't have to look any further than the huge mess that was (and is) linux desktops - it is only by moving away from strict bazaar models that they have made any progress at all over the past two decades.

That's fine if your proposed changes are small enough to require minimal intervention from other folks, but typically in these kinds of orgs there are all sorts of dependencies that you simply can't account for in code and in a pull request alone.

As a hypothetical example, let's say you want to send an email nudging your users about teams under their purview who could do better in adopting parts of your product suite. Suddenly other product teams are worried the recipient will choose to simply churn from a product that their teams haven't adopted, legal starts talking to you about the GDPR, etc etc, and things start to look a lot less hopeful.

Your best bet is to try and account for these sorts of issues before you even start writing the code. If they're reasonable issues and your code happened to land anyway, you would have done damage. Even if they're not actually issues, you have made other people's lives harder. And if your code never ends up landing, then you've wasted your time!

So, you get out ahead of it and set up a team that can interact with all of the stakeholders that might actually be affected. It might take time, but at least the risks are lower and your code isn't being needlessly thrown onto the cutting room floor.

And that's even without mentioning any sort of technical concern, of which I'm sure there will be many for anything larger than the smallest bugfix. Other engineering teams don't like the way you've duct taped your thing to their thing, others still are not excited to have to maintain your new addition to the infrastructure. Hopefully your open source bazaar thing has a decent way of resolving these issues, but most of them just have a BDFL that you can only hope will be polite as they smite your unwanted PR.

In a "cathedral" team with decent practices, you just have other engineering teams as stakeholders. It's just an extension of the set of connections you've already built.

How does this differ from a bazaar, exactly? Isn't all this possible under that model too?

Maybe, but you can't know for certain that you have exhaustively captured all the stakeholders or passed around your request to every relevant person. Additionally, decentralizing heavily enough will make roadmapping basically impossible, and your customers probably want to know when their desired feature is actually going to land.

Finally, the mapping holds the other way. The problem described in the article is that too many people hold veto power, and so nothing actually got done besides the lowest common denominator. It would be entirely possible for your OSS project to suffer the same fate if enough people in the mailing list complained that your new change breaks their workflow. You can take a look at all of these open protocols and projects that end up building the least common denominator product just because they do not dare break arcane promises made in a different time about some bits being in such and such place.

Centralized decision-making is its own sort of effective, at least in making sure stuff like accessibility and design are kept consistent. Microsoft gets a lot of crap from us for a lot of things, but some of these things are necessities that could just as well be forgotten otherwise.

The solution is kind of the same in both situations:

* Minimize the list of people who have veto power

* Work with the people who must retain it so that they trust you when you want to do something a little unexpected or different (and manage your own expectations when they disagree with you)

* Ensure someone above the conflicting parties exists that doesn't always resolve ties in a way that is risk-averse (or guide them to a solution that isn't always risk-averse if necessary)

and you should be well on your way.