Hacker News Reader

The problem was stated in Felker's response: "...corporate development teams are uniquely qualified to utterly botch a libc, yet still push it into widespread use..."

This can come innocently as well. For a simple non-code example: google has a code style guide. For C++ it says "don't use exceptions" and even (or used to?) explains that the writers consider exceptions to be a good thing but they have enough legacy / third party code base that doesn't use them that they can't switch. Lots of people use the google style guide because it exists, rather than write one themselves, or rather than spend time hashing one out. So they end up shackling themselves to a constraint entirely resulting from an environment they are not in.

I see this indirectly in your own comment, where you casually referred to gRPC or protobufs as "fundamental piece[s] of architecture". Well they aren't standards, they are technologies that google likes. Protobufs is a perfect example of a technology in, well not widespread use, but is used by a bunch of projects, and as discussed elsewhere contains a number of decisions that reflect the google environment, probably not resulting in a great fit for a number of places where it is used.

There's nothing malign in that; the whole point of writing code is to meet your needs. But when you're an 800 pound gorilla, don't presume your every step is what everyone else needs. They are what you need and without proactive effort may not help many others....but can crowd out what could.

>I see this indirectly in your own comment, where you casually referred to gRPC or protobufs as "fundamental piece[s] of architecture". Well they aren't standards, they are technologies that google likes. Protobufs is a perfect example of a technology in, well not widespread use, but is used by a bunch of projects, and as discussed elsewhere contains a number of decisions that reflect the google environment, probably not resulting in a great fit for a number of places where it is used.

I am not suggesting they're fundamental to the world, rather just fundamental pieces of architecture where they're being used at. That is to say, fundamental in a similar (though not identical) way to a libc would be. This is more of a miscommunication than a disagreement I think.

>There's nothing malign in that; the whole point of writing code is to meet your needs. But when you're an 800 pound gorilla, don't presume your every step is what everyone else needs. They are what you need and without proactive effort may not help many others....but can crowd out what could.

So, I'm not in the know, but... To quote the original post:

> So, what do you think about incorporating this new libc under the LLVM project?

That would, to me, imply under LLVM governance, not Google, and I think that's the primary thing people should focus on if they are worried. Kubernetes is a good example of where I think this ended up going extremely well; Kubernetes is under control of CNCF and not Google.

Moving governance elsewhere is surely beneficial for the project's health, and since it's a low level piece of architecture rather than an end user product, I see no reason why there wouldn't be motivation to do that. It appears it is the intent to do that in the first place.

Hi, full disclosure, it's my team working on this at Google FWIW, and the technical lead for much of Google's contributions to LLVM.

When we asked the community if they were interested in us developing this in the open as part of LLVM, we meant as part of LLVM. We're very committed to the LLVM open source project, and think it would be great for this to be developed within that framework (technical, project, community, the whole works). I'm hopeful that the community is in fact interested.

But we also were asking -- we have some specific technical goals that we'd like to make sure we can accomplish. It really is the community's decision whether this makes sense. =]

Hi Chandler!

Is it just me, or the original Googler's posting on llvm-dev does not explain their motivation beyond the vague "doesn't quite address"? I guess some solid list of problems hard to resolve could help.

Also, as you say, it's a community decision now, what will be a plan if community NACKs the proposal? A BoringLibc from scratch? Commitment to contribute hard fixes to glibc and/or mail?

I think it's hard to predict what a community will be interested in deep-diving to understand. And I'm not sure that HN is super representative of its interests either. The email was aimed at LLVM folks, not HN.

I generally trust the LLVM community to ask for any details they need to reasonably evaluate a proposal like this, and I also trust the folks on my team to work to address those requests to the extent we can.

I don't think it makes sense to try and speculate about what option will make the most sense if LLVM says "nope". Generally, I plan to encourage the team to see if there is a good way for us to address concerns the LLVM community has while still getting the technical things we need. IMO, it would be somewhat surprising if there were no reasonable path where this could both be reasonable for the LLVM community and Google. Doesn't mean it is impossible, but having detailed and precise plans don't seem like a priority. IMO, the priority is finding a good way to work with the LLVM community here.

On a more meta level, I also think it would be good for lots of folks (HN, twitter, etc.) to be a bit less harsh in their criticism of initial posts proposing new efforts/projects. I've seen this several times recently (ranging from this to the V language stuff). I'd suggest folks maybe ask questions and give people a chance to flesh out their thoughts and provide missing context rather than hammering in feedback. In many cases, I think the feedback is actually good, but the method of delivery makes it much harder for people to learn from and respond to constructively.

Anyways, enough meta...

> I've seen [harsh criticism] several times recently (ranging from this to the V language stuff).

It's probably not a good idea to put yourself on one level with a scammer.

Seriously though, any effort made by Google will of course be scrutinized much more vigorously than the same effort made by a smaller org or a single person. If I decide to build my own libc, no one will care because they probably don't need to care. But big companies like Google have the power and inertia to just shove decisions in people's faces. I think for most people here, a large part of the dayjob is spent dealin with and working around the stupid decisions of third-party hardware/software vendors.

There are also many example of Google doing just that to various degrees when it comes to open source projects, security research, their own products, dealing with vendors, customers or employment among other things. This of course isn't something unique to Google. What is unique however is their position.

They are probably the most powerful company when it comes computers, software and the Internet. They have the culture, means and goodwill to do a lot more damage than almost anyone else. I am sure that is at part of why people want to work for Google. Not to do damage of course, but because you can make a difference. Unfortunately making a difference for Google frequently isn't the best for rest of us.

>And I'm not sure that HN is super representative of its interests either. The email was aimed at LLVM folks, not HN.

>On a more meta level, I also think it would be good for lots of folks (HN, twitter, etc.) to be a bit less harsh in their criticism of initial posts proposing new efforts/projects.

I'm sure it's super annoying when folks in the larger community with vested interests in the continuing success of projects like LLVM start noticing harsh criticism coming from credible experts like Rich Felker. =]

And then post it to HN or tweet it out to others so the concern spreads. =]

But all feedback is good feedback, right? And anyways, I'm sure the folks at LLVM will let Google have their way in the end because there's no other foreseeable outcome of this discussion, is there? At least, none worth discussing. =]

Rich Felker was being civil. You are not being civil. In the spirit of civility and honestly just adhering to Hacker News community guidelines, please reconsider your tone. I believe this kind of behavior is damaging to open source, and mocking the person you are replying to reflects pretty poorly on yourself.

I agree that I was out of line, which is why I posted a more civil reply (I hope) afterwards, since at that point I wasn't able to delete it.

Snark aside, Google's desire to own the de facto non-GPL libc is worthy of a discussion in the broader community. Your expressed desire to limit the discussion and unwillingness to directly address the issues raised by Rich Felker speaks volumes.

- Google doesn’t claim to want to own it, at least not anywhere that I’ve seen written. They just want to design it, for their needs.

- Nowhere do they say it should be the “defacto” libc. If anything, Clang and libc++ remain not defacto. Glibc would likely continue to be the preferred libc in my personal opinion. Nothing wrong with that.

My concern is that if Google's libc becomes part of the LLVM package, it will push out Clang and musl and any others. It should be a separate package like the others.

I assume you mean glibc instead of Clang here.

Well, I think personally making it part of LLVM works to the community’s advantage. If Google is the sole owner and maintainer, external forces will have less impact, much to the detriment of the library quality (imo.) As a part of LLVM, there are many huge players with interest in improving the library and many maintainers externally.

Google and other big companies already contribute to LLVM, so I don’t think the status quo is changing at all.

"...corporate development teams are uniquely qualified to utterly botch a libc, yet still push it into widespread use..."

That doesn't seem to be uniquely corporate, there are lots of botched, poorly implemented, or designed, community projects not backed or funded by corporations that have come into widespread use.

There are also lots of attempts by corporations to force adoption of some piece of open source software only to fail, look no further than Polymer for example, or NaCL/PNaCL for that matter.

Sounds to me like what they're proposing is a simplified subset of libc, since the exact justifications for what's being supported and what's being left out haven't been discussed, it really seems premature to dismiss it out of hand. There could be plenty of valid reasons for doing it.

Look at OpenGL, a hideously complex spec that is onerous to implement for full compatibility and compliance. John Carmack forked it to create mini-GL drivers that were useful for games. Eventually this became MCD, and eventually the largest most cash flush vendors funded full ICD drivers. Carmack was correct to fork OpenGL for consumer hardware to support games as the fully OpenGL was really incompatible with market needs and backwards compatibility with the full spec was needed by a tiny minority, but held back drivers for millions of consumers.

And now, decades after that decision, we have Vulkan, because honestly, the original OpenGL design needed to be replaced.

Perhaps libc is reaching that point where it is suffering under technical debt?

> ... there are lots of botched ...

Yes, but have they been "push(ed) into widespread use ..."? That was the point Rich was trying to make, I think.

Yes. Does OpenSSL ring any bells? How about PHP and its ecosystem? How about Sendmail? Or GNOME.

There have been OSS projects 'push'ed into widespread use by the popularity of a few strong personalities and herd mentality, but have questionable design. As someone with a long history of Perl, I'd lump the entire Perl language and ecosystem into that category, along with CGI and FastCGI which existed far longer than they should have before someone put their foot down and made a better open-source application interface.

Developer adoption is by no means a process by which 'he who has the most money wins', there's plenty of examples of big failures to thrust commercial designs into the public space, and there are plenty examples of shitty OSS designs getting uptake, I'm sure if you ask Linus, he could go on a long rant.

As I mentioned with Carmack's mini-GL/MCD model, new forks often get traction if they solve very real customer problems. In Carmack's case, he was developing on NeXT Hardware, for PCs, and PC consumer cards didn't run OpenGL, but often ran either DirectX or a proprietary driver API. If you are in the video game market, multiplatform portability is a given, so forking OpenGL to contain only the high performance subset for games that could run on consumer hardware was so obvious in retrospect, and solved a real program for game developers, who quickly latched onto it, and for 2nd and 3rd tier 3D accelerator vendors who couldn't get devs to use their proprietary API but could benefit from the easy to implement mini-GL.

I have no clue why they want to fork or subset libc, but before dismissing it out of hand, maybe there's a real compelling reason for it, and if there isn't, it'll have as much shelf life as proprietary Unix forks.

But using ad hominem critique in absence of a concrete proposal seems out of place to me.

Its not really ad hominem, Android itself us a massive pile of hacks (zygote process, project treble, binder, etc) that while it worked for google in fixing their issues it is far from ideal. Also what API has replaced FCGI and CGI in general? Hopefully not in process modules.

What's replaced it is running services or application servers that contain an embedded HTTP handler, and putting a load balancer in front of it.

Also, rather than guilt by association (Android et al, which was done under different circumstances compared with the hundreds of other libraries Google has released), why not actually hear out the concrete proposal and then criticize it?

Saying no one in category X can ever build anything good is just illogical.

NDK still feels like it.

'getting uptake' is not the same as being 'pushed'. In fact, the former sounds a lot like 'pull'.

Have you used OpenSSL? It sucks. The API returns strange and inconsistent error codes. Error reporting was terrible. Strange non orthogonalities abound. It is a lesson on how not to design an API. The only reason it got traction was it was first and no one bothered to do better.

systemd? pulseaudio?

Bad examples. Lennart benefited from Redhat backing his work.

Great examples. What happens if Google botches up LLVM? They're far more powerful than Redhat.

They were bad examples of non-enterprise backed software being pushed into widespread use.

They are great examples of poorly-designed replacements for existing open source software being pushed into widespread adoption solely because the largest Enterprise distribution of Linux willed it.

And also great examples of software developers lying about their intentions early on to silence criticism of their design and the subsequent ramifications of adopting their solutions.

Multiply that by 100 when comparing Google's influence with Redhat's.

If Google wants to develop their own libc, they can. I'm f they want to make it open source, they can. There's nothing about doing those things that necessitates doing so as part of the LLVM project.

Seems to me that trying to upstream is common courtesy before forking. If there is some reason it needs to be integrated with changes to LLVM itself then you might end up with an LLVM fork similar to WebKit/Blink, is that a better outcome?

The point is that no existing libc is part of the LLVM project; i.e., under that umbrella. What's different about Google wanting their own libc and following what Clang and musl do currently?

The question I have is, is there a legitimate reason for tight integration between a compiler and libc? Perhaps there are very good reasons for them to be designed together. A lot of compilers to this for other languages because it allows them to have special logic around intrinsics for example.

Good examples. They were pushed into widespread use by Red Hat's corporate interests.

Which makes them bad examples for the argument discussed here.

> Also, Google has a pretty good corpus of code to test their libc on

I left Google more than 5 years ago but at the time the subset of the language (C++) one was allowed to use was quite limited. Testing your implementation of libc on that codebase could still lead to poor coverage.

> Google has a pretty good corpus of code to test their libc on...

That is a good point, but I'm in no way disputing that Google could do a great job of creating their own libc. I would never be foolish enough to challenge the merit of Google's engineers, the proof of this is clear in the tasting of the pudding that is Google's software. My concerns lie in the open-source community becoming further beholden to Google, or even worse with Google dictating the direction of development on what could become a cornerstone of the architecture of many critical pieces of software.

(as stated elsewhere, this effort is in my team at Google)

FWIW, while I think libc is super important, and I'm really excited about this project and the opportunity it represents, I think the LLVM project and community is much larger than libc. =D I don't think you need to worry about this effort changing anything about how LLVM operates. And we're currently just asking if they're interested. =]

What people are afraid of isn't Google having a negative effect by lack of motivation or technical competence, but the opposite. That Google will be very successful in doing what Google think is important while not taking anyone else's merit, interest or concerns seriously.

Being excited and not seeing a problem isn't so much reassuring as what people are afraid of. If you do end up being wrong and Google does gain too much influence from the perspective of someone outside of Google not only would you probably be getting a promotion but also be in a better position with more influence yourself since you are a part of Google.

So unless you are specifically convinced that what people are concerned about won't happen, which would be pretty hard to argue unless this activity differs substantially from others at Google, your comment doesn't really say much. As it doesn't speak to anyone else's interests.

Google has been a contributor to LLVM for about 10 years, and a major contributor for most of that time.

At no point have they tried to force something into LLVM that goes against the wishes of the community. They have always worked with the LLVM community to improve LLVM, and LLVM has benefited greatly from Google's contributions. I expect this project to be no different.

In the beginning of Google's letter they let us understand they are going to create simplified version for their own needs. It does mean they don't care about compatibility and bugs, if it doesn't affect their software. That's not how this kind of libraries should be implemented.

> Chrome, an end user product. libc is a fundamental piece of architecture

The web browsers is not the end user products anymore. The web sites(and these days, smartphone apps which is just a wrapper of the browser) are the end user products. The web browser is, just like libc, fundamental piece of architecture.

Sure, the users directly interact with web browsers GUI, but average users can't recognize the difference between the Browser, Internet, Google and Android. They are all same things to most of them.

In that sense, browser and libc are the same, all users use it without noticing it.

The browser is a architectural in the electron case where it is shipped as part of a product, a whole browser is a product by itself.

Same way as LCD screens are achitectural pieces but a television is a product.

I think trying to work with protobufs by analogy here fails because the scopes of power are so vastly different.