CyberDildonics

No, it's like saying you're addicted to nicotine or routine or a break with people, because that's what cigarettes give you and there are other sources of that.

It is possible to get information from other sources just as it is possible to get nicotine or a break with people from other sources. Because of this your metaphor breaks down.

I don't think it's fair to write a misleading title then tell people "they're reading too much into it".

Implying one thing then walking it back in the article or having the article be about something totally different, then saying people should read the article and not pay attention to the title is just manipulative to your audience.

You can try to call my single replies "spam", but I think if you could actually respond to the things I've said you would have done it already.

If you do use MUSL then you aren't using the fastest library,

You are contradicting yourself with logic that doesn't add up.

The default libc isn't the fastest way to do things anyway. So by your own logic the title should be "don't use libc if you care about performance". This of course doesn't make sense either because if you aren't using libc then musl wouldn't matter. This is why the title doesn't make sense and why your defense of it doesn't make sense either.

You having software that is not only slow but slow because it calls into the default libc has nothing to do with this. Why aren't you mentioning taking it further and linking in a better allocator?

Maybe because it's the same idea but contradicts what you keep trying to say to avoid the actual point, which is that the title is wrong.

If you're going on a long trip and you get better shoes and can walk to your car 25% faster, how much is that going to speed up your trip overall? It will be basically nothing, because you are optimizing something almost irrelevant.

I can use musl and create software that outperforms whatever someone else makes using the default standard library. If this is possible, and I explained why it is in detail, then why would the title be true? Explain that instead of getting upset and avoiding the heart of the discussion with irrelevant tangents about your own legacy programs.

You can try to be patronizing if you want, but this doesn't make sense with you not understanding memory access patterns. If someone uses C++ they will use C++ strings. Have you used C++? It works well and it has been around for 40 years, you should give it a try.

Keep deflecting with bullshit that is irrelevant to the choice of library, and that can't be changed but for massive refactoring.

You painting yourself into a corner has nothing to do with a C library being a few percentage points slower. This is not a general scenario, it's something you are dealing with because you can't optimize C strings.

If you think memory access patterns are 'bullshit' you haven't done much optimization. That's everything in modern optimization.

If there is an expert somewhere around you, please talk to them so you can get up to speed on modern optimization. Minimizing allocations is table stakes, memory access is the meat.

If you have any examples on github of things you think "can't" be optimized, go ahead and link it and tell me what a profiler shows, and I will explain what to do.

I've made my point already, you just haven't accepted it.

You made your claim, you haven't had any evidence or explanation, that's how it works. You say the same thing over and over and get more upset, but that isn't evidence. Show me where musl prevents someone from writing a fast program.

Insults and hallucinated quotes aren't a good foundation for proving your point. If you have to pretend that someone said something different, then maybe you aren't really making sense.

how about applying those elite optimization skills you claim to have to the MUSL code to make it faster.

Why would I do that when I don't use the C library for performance sensitive programs? You never seem to be able to confront this. The stuff made 50 years ago isn't the fastest possible stuff. It's still pretty fast and if you want something faster, do something else. It's not that complicated but it seems to really upset you.

I don't have any other conflicts of interest such as being a glibc developer, so don't start up with that shit either.

No one said anything about that, I think you're hallucinating or predicting something that never happened.

What you keep forgetting is that's 25% of the part that you're actually using.

You keep avoiding and denying that for a program where any sort of speed matters you just aren't spending any of your time hitting the C standard library.

If a program is slow because of the C standard library and it needs to be faster, someone messed up a long time ago. That isn't the C libraries fault. You can squeeze a little more out of it if you have find a more optimized library, but it's all a drop in the ocean compared to how much faster it would go by doing basic real optimizations.

Performance-sensitive programs and libraries are often written in C. C-string representation is widely used by all programming languages, which are usually written in C or C++ (which uses C).

You can say that, but really it's almost all C++ and people avoid C strings for exactly why I outlined in detail. You don't have length up front, it's all ascii, you're dealing with pointers to arbitrary runs of bytes, etc.

This is not good for memory access patterns, dealing with lots of characters at one time, dealing with unicode, minimizing memory allocations etc.

It's true that I'm making claims and you're not accepting them.

That is true that you are making lots of claims and that I'm not accepting them, because you don't have any evidence or explanations and they don't make sense.

All I've said is that MUSL doesn't prevent someone from making fast programs and you barely have even confronted that, let alone explained how it isn't true.

Linking MUSL to any program that heavily uses the slow functions will make it slower.

Any program that is hammering the the C standard library can be sped up by orders of magnitude and 25% is nothing. Again, lifting memory allocations will speed something up by 10x, so that 25% isn't going to matter anymore because it's 25% of something that isn't even going to show up on a profiler, let alone be a bottleneck.

Your position seems to be that the title is wrong because it is theoretically possible to make MUSL-dependent programs fast according to some unstated performance metric

I think you mean it's trivially possible according to the detailed explanation I gave from a lot of experience optimizing.

Sometimes the objective of caring about performance is to have literally the fastest thing possible, not just "fast enough".

25% better might be fast enough for you, I like speeding up programs by 100x by changing to C++ instead of C and focusing on the optimizations that matter.

The title says "Don't use MUSL if you care about performance."

I use musl and I care a lot about performance. It doesn't matter because has no bearing on how fast my programs are.

I'm done with this bullshit conversation.

I'm sure it seems that way when someone doesn't accept the same claim over and over without any actual explanation or evidence. Saying the same thing and getting more upset is not an effective way to make your point. You need real information, not insults and fake quotes.

There are a few problems here. The first saying that anything is dramatically worse. The second is thinking that nothing can be changed. The third is thinking that there are lots of programs out there that spend all their time in C string functions yet nothing can be altered except for linking in a different standard library.

I can't tell you where, or show you code, or anything like that. Get used to it.

I didn't expect at any point that you to be able to back up what you are saying with examples.

Everything you've said is "It's NOT a problem because I'VE never seen it be a problem!"

That couldn't be further from the truth. I'm saying it isn't a problem because the problems are easily fixable and musl doesn't prevent them from being fixed.

You hallucinated a quote and made up something completely different in your head.

If you don't think string processing is a bottleneck, you should consider how many applications are document-based and string-based. Basically, it's a MAJORITY of applications in the world, and I'd put money on that.

You think these programs are bottlenecked by the C-string functions in their standard library? That's a bold claim. Why would a program completely dependent on strings even use C string functions in the first place? You have to scan to a newline to find the length, they work with ascii and they are known to be incredibly insecure. What you're saying doesn't make sense.

You clearly deny having experience with the stuff I'm talking about,

I deny that there are programs that can only be sped up by switching to a different C library and nothing else, since that's nonsense.

just not use MUSL if that's your problem.

People can do whatever they want, all I've ever said is that musl doesn't prevent anyone from making a fast program. That's it.

Saying you're inexperienced is not insulting,

I'll take your word for it because you seem extremely inexperienced at optimizing, especially if you think making fast software should include leaning on C strings and having memory allocation show up on your profiler.

This is youthful arrogance (I can only assume you're young; if not, you at least haven't matured in your career).

You can go for the insults and try to be patronizing again, I expect that as the last resort of someone frustrated that repeating their claim isn't taken as evidence. I explained a lot in detail about why musl isn't going to prevent anyone from writing fast software because I've done it over and over.

You seem to be saying that you can speed up legacy programs somewhat that weren't made well in the first place with a faster libc and I'm sure that's true, but it has nothing at all to do with the premise the musl prevents a program from being fast or even is much of a bump in the road.

The speed ups from a more optimized libc are percentage points shaved off the the times where you actually use it. Using better string functions, minimizing allocations and paying attention to memory access patterns are going to be order of magnitude changes. Minimizing allocations is going to be at least 7x on a single core, better memory access patterns are going to be 20x-25x.

You don't have any evidence or explanation that musl prevents someone from writing fast software, which is the title and the title is wrong.

No, this is something that nuanced. The title is wrong because musl isn't going to prevent you from writing fast software.

Whatever benefit there is to a different libc, is absolutely miniscule compared to do actual optimizations like avoiding allocations.

I'll give you real numbers: if you put allocations of short vectors of a dozen floats in a hot loop, when you lift the allocations out your program is going to instantly get about 10x faster. The allocation is no longer going to be the bottleneck, it will be marginal and then a faster allocator isn't going to matter at all.

If someone gets an easy speedup from using a different libc that's great, but the vast majority of time it isn't going to matter and isn't going to be where any real speedups come from. The difference is a small percentage speedup vs orders of magnitude.

The amounts to the title being wrong, using musl or a small standard library just doesn't prevent a program from running fast. It is a tiny difference and even that tiny difference can be changed from things like better allocators which you would do anyway with a standard libc.

To be clear, you're saying that in video games and robotics people are using strings instead of numbers and when that becomes a performance problem you think simple C string functions are to blame? How about not using strings as values?

By the way most software does copious amounts of string processing... I think that should be common knowledge, but I guess it isn't.

It isn't because it's not true if "copious" is about CPU time. String processing is rarely the bottleneck.

I believe the title is accurate

Well, it isn't. It's unlikely that musl prevents anyone from making a fast program. It doesn't even make sense. In the off chance anything was a real bottleneck you could bring in something faster and you would want to do that anyway.

People in performance-sensitive areas gripe about libraries, even standard libraries, quite often.

I have done a lot of optimization and I've never seen the standard library be a problem for exactly what I just outlined.

A lot of what you're saying is just "it's a problem because it is, trust me". That isn't evidence or an explanation.

As soon as allocation is slow you can avoid allocations (which you should do anyway) or use a different one (which you would do even with a standard libc anyway).

If you really have string problems (and not some fake problem like just parsing strings over and over instead of caching values) then you would use an optimized library. The benefit from a regular libc over musl is minuscule compared to the real solutions to optimizing.

you're making bold assertions despite clearly lacking the experience to know how things are done in industry generally.

Trying for insults doesn't add any sort of technical explanation. To be very clear any program written by someone who says their memory allocator is their bottleneck is something I could speed up by orders of magnitude and the standard library isn't going to matter.

I would dispute this in anything that isn't mostly about string processing and in that case you can always easily grab different string functions, which you should probably do anyway if strings are that important.

It also tends to be very difficult to improve because the strings are everywhere in that kind of program,

I don't know what 'that kind of program' is supposed to mean.

and refactoring to eliminate them is either impossible or very risky.

This doesn't sound like a general purpose statement that applies to anything broadly.

All I'm saying is the the title is wrong and musl doesn't do much to prevent speed in a program. If someone was really trying to optimize, blaming the standard library is not going to get them very far and it's easy to work around, but needing to do that is very rare.

+ every application doing manual memory pools

Usually it's simple data structures in flat memory.

If allocation is taking all the time, that's a poorly optimized program with lots of low hanging fruit and a different allocator is not the right fix. It's like having a boat with a hole in the bottom and someone says the solution is a smaller hole.

But TCmalloc / jemalloc are 21-22 years old, respectively, and jemalloc has been the FreeBSD (released) default malloc implementation for the last 18.

jemalloc is also possibly bigger than all of musl. If it was a problem after optimization I would use it and I have in the past, it's just nowhere near as important as minimizing allocations in the first place. OpenBSD uses straight mmap.

The point is that memory allocation shouldn't be a bottleneck either way. If it is the program needs to be optimized or redesigned. Better allocators give you more slack, they don't solve the problem. If the problem is already solved, then a basic allocator isn't going to make a big performance difference because it isn't the bottleneck.

those bad data structures would also cause "damage to speed and interactivity" or whatever. This isn't very hard to understand.

It depends on how much they are used and how much contention there is. Sometimes putting a mutex around things is fine.

But these are extremely common specified functions, they are called everywhere all the time

Not necessarily, especially for C string functions, but they do get linked in so it's a good thing musl makes them small.

I said "backbone", not "bottleneck".

Then the point is lost, because 'backbone' doesn't mean anything if it works. If it isn't a bottleneck in throughput or latency anywhere then the speed doesn't matter.

The other important thing is that better stuff can be included in pieces as it's needed. The reverse isn't true. If you use a big fat C library, you have a dependency that isn't going to get better.

Not really. I have to spell it out apparently: actual programs written by normal human programmers do those things, all the time,

You spelled it out last time, it's just not true in the sense that programs have to have these functions as bottlenecks. Strings, allocators and memory copying can all be dealt with independently, but again it's rare that strings and allocations really need to be the bottleneck and in those circumstances you probably want more than a different standard library anyway.

in the imaginary fantasy land people have in their heads where they make up arguments to themselves about how, if every program was written how they liked it,

I'm not sure what this is supposed to mean, there is nothing I've said that doesn't make perfect sense. If you want something to go faster you can make it go faster. A better allocator pales in comparison to lifting allocations out of hot loops.

My point is the musl is useful and the disadvantages are easy to work around. I'm not really sure what your point is, do you think people are going to force you to use it?

Every default malloc implementation worked this way about 12 years ago. Making lots of small allocations, even from multiple threads then blaming the allocator is a losing strategy. An allocator is only going to be able to mitigate the damage to speed and interactivity.

The solution is and always has been to make larger allocations and use those efficiently.

They are just naive loops with nearly no optimization.

The compiler should be able to take something with good access patterns and make something fast, especially out of the basic C functions.

they are the backbone of vast amounts of code

Performance wise it's unlikely C string functions are actually the bottleneck in a program. Maybe for specific programs a naive memory copy function could benefit from AVX instructions.

Real programs have to often do things like allocate memory

"Have to" and "often" are debatable. Any allocations in a hot loop are the very first things that should be optimized away after profiling.

Desktops can have plenty of cooling so you actually get the real speed of whatever you are using.

I don't know what you are upset by this, cppreference is great and it explains the problem you had.

I blamed it on STL (and its complexity).

If you make a simple dequeue with a vector any resize is going to invalidate pointers. It isn't the STL's fault, it's meant to queue simple data for ordering so that you can copy it in and out, not hold on to a pointer to something internal. It isn't meant for actual storage just basic structure.

std::deque is a super bad offender, in many ways. I advise against using it.

You brought it up as something you were using.

No I have not misunderstood anything. Again you're coming back to your arrogant pattern.

It's not arrogant to point out how things work. In this case the pointer to a reference count is pointing to internal data in the data structure so that other threads can see it. This is not the same as shared_ptr which is tracking a reference count of itself.

But I rarely don't do that. And that's not implied by "C style" at all.

So you frequently do that? It's your C style, that's what you showed me and it takes two heap allocations so that a pointer can be returned. If you create a pointer to a struct in a function it can't point to the stack inside the function.

So please stop repeating made-up contradictions.

You did say you got burned by an assumption of pointer invalidation and that was your explanation for how the STL gave you problems with concurrency.

Claims without evidence unfortunately...

There are benchmarks and lots of people use these queues. I've used them and you can use them yourself. It isn't like saying something is bad then not being able to explain it. You and anyone else can and do use these. It is an opinion, but it is backed up by a lot including a great interface.

Good, because I don't do that at all.

Then you probably have memory leaks because you need to call the free functions that you make when you create a data structure.

And I criticize that RAII is a system that sneaks in resource management _implicitly_ everywhere

No, you said that it created implicit program flow, now you're walking that back I guess.

Also it isn't everywhere, it's only where it needs to be, so I don't know why you wouldn't want it there.

I at least showed you cppreference so you can look up the data structures and their guarantees.

True, but my problem was neither proper locking / thread safety, nor reference counting.

But you did blame the STL for concurrency bugs so there must have been something.

prefer to invite tons of unnecessary boilerplate

I'm not sure a heavily tested and fast lock free queue library is boilerplate.

You used C++'s dequeue, wouldn't that be boilerplate by this bizarre definition? Wouldn't everything?

I DO NOT THINK THAT. Why do you keep implying that my thinking is wrong? That is so arrogant of you.

That's good, I must have misunderstood since you were blaming concurrency bugs on the standard library data structures.

In my case, the queue was used as a "global" kind of object, so no reference counting needed.

I think you might have misunderstood that the reference counting is for anything returned from a data structure so that it can see that something is being used and not modify it. The reference counts of the returned object are actually pointers to the internal reference counts in the data structure, like checking out a library book.

This is not how I would do a queue though and not how the queues I linked work. They copy data in and out and are best used for small data. Large amounts of data can be handled in a different way by a different structure.

where did I imply making "double allocations"?

The C style allocation of structs to pointers then allocation of the underlying data is two allocations and double indirection. This isn't good for multi-threading because allocations have their price, just a heads up.

You're arguing all the time for just buying into stuff as a cargo cult

I don't think so, I've made a lot of stuff that works.

Don't explain basic C++ stuff to me. I understand it.

Well.. we all get bit by standard library assumptions from time to time and need to read the docs, but it just isn't a concurrency problem with the STL.

There's a lot of "abstraction" slop that brings more downsides than upsides.

Claims without evidence unfortunately. The fast concurrent queues I linked are great and using destructors to keep track of reference counts is great. Both are minimal. I would say inserting resource management manually into every function is boilerplate.