Comment by gpm | Hacker News Reader

Comment by gpm | Hacker News Reader

Parent Full thread

This is a common complaint about companies starting new open source projects, and it really annoys me.

Anyone who works with software knows there are tons of very good reasons why you might want to start a new project from scratch instead of contributing to an existing one. You might find the existing architecture limiting. The trade offs taken to be non optimal (for your use case or in general). Consider the existing code to be low quality. The code contribution process to lax (introducing bugs) or strict (slowing work). Find the community toxic. Want to utilize existing internal code bases. Etc.

There are tons of valid reasons to make a new project, and for most of them you don't want to say it out loud. Saying "your code sucks" or "we don't like you (i.e. toxic community)" is rude. Saying "we don't like the processes" is asking for a long political fight, and likely to be taken as rude by the people who invented them. Saying that they want to utilize internal code bases could be problematic internally. Etc.

In this case I get the feeling that quite a bit of it is disagreeing with the tradeoff's musl took (see: Wanting to make libc more like a library), which has to be about the only reason you are allowed to publicly say. Maybe there are other reasons as well, maybe not.

Sorry to jump down your throat, it's just a common type of comment that I wanted to give a general reply to. (Edit: To be clear I'm not involved with this project, or a google employee for that matter)

This is reasonable.

A simple answer like "we are not starting with musl because ..." would be quite welcome, though.

I assume, that Google's answer would sound like this:

"We are not starting with musl, because we want to own all our software the same way people own hamsters or dogs: care for them, feed them, train them, single handedly decide whether to castrate or kill them. We don't need the risk of owning something we don't fully control."

As far, as I am aware, musl isn't owned by public company or foundation. Google can't exert desirable degree of control over musl project, because there is no board of directors to buy their way into, — it is all maintained by one guy (and chattel slavery is outlawed in US...)

Bribing or hiring the dev can still work.

When a new project is proposing to do something that's been done before, potentially multiple times, I think it is reasonable to want to understand how the new project will compare and contrast to work that's already been done.

Asking "How will this project compare to <existing implementation>?" seems quite reasonable to me. In fact, I would expect the person making the proposal to address this question as part of the proposal. It is not desirable to reinvent the wheel without good reason.

It's totally fine if the reason is something like: we want to use a different license, or a different development process, or a different coding style, or whatever. Whatever the actual motivations are, clarifying them up front will be good for everyone.

I agree this with as some one working at a company that avoids most open source like the plague. But in this particular case, the code quality in musl is pretty good. I haven't had an issue with it so far, and after trying to upstream some things in glibc I have to say I can't see why they wouldn't use musl.

Well, there are still huge holes in musl. towlower e.g uses a linear search. my binary search made it 2x faster, but we still have to wait for that to appear. Implemented Unicode standards are years old. You don't know which standard is implemented. Still many problems with dynamic linking.

And: musl is POSIX. POSIX blocking IO should not be used in a modern OS or language. You can never guarantee concurrency safety with POSIX. This is the Fuchsia problem. They went for unsafety and POSIX compat. L4 ditto. Microsoft's Singularity not.

A modern libc would need to get rid of blocking IO and implement strings as utf8 only. There's no need for anything else. The C standard made too many mistakes.

I don't see how you're planning on writing a libc that chooses to interpret the C standard as full of "mistakes" without writing an entirely different language.

I'm already maintaining the libc security extensions, which are in the standard but all libc's refuse to implement. I also added the missing string API's for the wrong wide API, but thinking about adding the proper u8 api instead. Nobody really should use the wide (UTF-32/UTF-16) api.

But the C standard committee is somewhere completely else. There are not even aware of the problems, even if I'm getting interviewed from time to time, so that they will change their attitudes towards insecurity and functionality. searching for strings should be a pretty common usage pattern, but it's still not implemented. coreutils or grep or sed would like to use it, I'm sure. But not 10x slower as the current multibyte patches. Which I'm also maintaining btw. for coreutils.

Non-blocking IO into the standard? I only know about libuv which was fine before the drama. Despised it since then, but I'm still using it. The standard mostly needs to provide a namespace to block blocking IO if someone needs a concurrency-safe subset. Then the unhealthy need for POSIX compat can be circumvented somewhat. And gets() can be finally.

What problems with dynamic linking? Considering docker and most container Linux systems use musl im sure they wouldnt mind knowing. Also musl has no problem having utf8 strings, in fact its one of the highlighted features.

pthread and dynamic linking: Look at their bug reports and mailing list discussions.

musl has no problem with utf8, because musl doesn't implement an utf8 string api. musl is strict C11. This is one of the problems. Most libc's do have this problem. There is casefolding, which is small but too slow (as explained). And 6 years behind. (Unicode 6.0 vs 12.1) There's no normalization at all, so you cannot search for strings, just for memory buffers. Which leads to insecure identifier handling, such as all public names, paths, ... There is a lot of identifier API in C11 and various syscalls/kernels, which are all insecure. Not musl's or any libc fault, but they should lead the pack, not follow wrong standards. Having checked unicode collate handling, guess there's none neither. It's planned for later this year I read. Unicode (the "wide" api) is UTF-32 which is too big and too slow.

Yes, there are reasons to rewrite a library, but people expect you to tell them why you are rewriting it.

So... why are they rewriting it?

Reply on news.ycombinator.com