Hacker News Reader

You think that

  *((char *) mempcpy (dst, src, n)) = '\0';

is a good solution then?

Also, I don't know where this 'srlcpy silently fails' meme comes from. It's your responsibility to check the return value. It only 'silently fails' if you don't check. You can argue about whether it is a good idea or not to make client code responsible for that kind of thing but it is the C philosophy. If you think string copy functions should call abort on failure then, like Theo says in the OP, implement that and submit patches to all those projects using strcpy.

> mempcpy

For chain copying? Why not.

The reality is that both strcpy and strlcpy require the person using the API to use it correctly in order to not end up with a buggy program; but, in the real world, people hype strlcpy as being fundamentally safer than strcpy, and so if you actually look into how people use strlcpy no one ever checks its return value (even projects like OpenSSH).

For more on this (and a specific ludicrous example from OpenSSH where someone spent the time to write a comment above the strlcpy about truncation and yet still didn't actually bother to check the return value of the function), please see this response I made to someone else:

https://news.ycombinator.com/item?id=6940724

> These APIs do not "fail safe" > This is still a bug in your program

Define "fail safe". They don't write more than what you want it to write, which could be exactly what you're looking for. Does it hide bugs? Maybe (depends on how you use it), but it's easier to figure out a bug regarding a short string, than trying to figure out why the program is crashing because you have a buffer overflow somewhere in your program.

> Instead the program must reallocate or signal an error

Who is he, to tell us what is and what isn't acceptable behaviour in our programs? There are numerous cases where discarding some parts of a string is perfectly acceptable.

> I can construct you cases where the use of these stupid functions is creating new security problem.

Sure, I can construct such cases for almost every single function in the C std library. This is C, that's the tradeof we make for speed and efficiency.

strlcpy is better because strncpy doesn't indicate an error (and strcpy, well, we know what that does), not because it prevents a crash. I'd rather debug a crash because in that case, I get a nice stack trace and core dump to gaze at.

Also, I disagree C is a tradeoff for speed an efficiency. Runtime speed and efficiency? Development time perhaps? I can easily write parallel code in scala that defeats idiomatic C on both. I use C when I want something portable. I can't use the scala code on a tiny mips device, but I can port the C code if it's reasonably well written. Same goes for any other target, there's nothing so pervasive as C.

> strlcpy is better because strncpy...

I was mostly thinking about about strlcpy and strcpy. The latter indicates no error and can cause a buffer overflow, which IMHO is harder to debug than erronous logic due to a stringer that is shorter than it is supposed to be.

> I get a nice stack trace and core dump to gaze at

I keep forgetting that C actually has stack traces on normal platforms. We lose all information when the program crashes, stupid custom proprietary platform.

> Also, I disagree C is a tradeoff for speed an efficiency

In C, you sacrifice safety (undefined behaviour etc.) for speed. Rust gives you safety, but (acording to themselves) you can't expect Rust to beat C/C++ in performance for certain operations.

> that defeats idiomatic C on both

But has a minimum of three times the memory overhead. I could write a paralell Clojure application that is way easier to reason about in less time than I could write said application in Scala, but I sacrifice memory and speed efficiency. Every language has tradeofs. C sacrifices safety for speed and memory efficiency.

> I can't use the scala code on a tiny mips device

Why not? There is a JVM for mips isn't there? Or don't you have enough memory? JVM languages uses alot more memory than a native program, that doesn't necessarily make JVM-based languages any less portable.

The same .jar file can be executed on Windows, Linux, Mac and more operating systems without a recompile. That's portability that's hard to beat. Considering you can also compile Java applications for iOS now, and of course Android as well, JVM-based languages are remarkably portable, and I'd argue that they are more portable than C.

The reasons for using C IMHO, is that it's fast, memory efficient, doesn't do any magic (like GC and JIT compiling and whatnot), and that it doesn't require warmup to be fast. It is however slower to develop in, and the language is inherently unsafe.

Last but not least, it's important to use the right tool for the right job. I would never write a VM in a JVM-based language (I'm writing one in C now, fun fun fun!). On the flip-side I would never write a GUI application in C.

> [...] We lose all information when the program crashes [...]

Ah, what you wrote earlier makes sense in light of this :)

> In C, you sacrifice safety (undefined behaviour etc.) for speed.[...]

You can write slow code in any language, a program isn't necessarily fast because it's been written in C. It's fast because it's been written with runtime efficiency in mind. Undefined behaviour comes from not making unnecessary assumptions on the underlying platform, which aids portability.

> But has a minimum of three times the memory overhead.[...]C sacrifices safety for speed and memory efficiency

Memory efficiency, yes. I, for one, prefer not to rely on a garbage collector and verify correctness with valgrind. You do have to diligently manage memory, or any language that assumes GC may win in the long run. Speed, however, was never a goal of C. You could argue the same thing with assembly, if you do all of the hard work yourself the result can be faster and more efficient than the same application in C. Neither was meant to be fast or efficient on its own, but may be considered as such when compared to higher level languages.

> There is a JVM for mips isn't there?

It exists, but it's proprietary.

> Or don't you have enough memory?

Not by a long shot, usually.

> JVM languages uses alot more memory than a native program, that doesn't necessarily make JVM-based languages any less portable.

Not on principle, but in practise it does.

> The same .jar file can be executed on Windows, Linux, Mac and more operating systems without a recompile. [...] I'd argue that they are more portable than C.

You simply moved the problem from the program itself to the underlying platform. By that logic, I can prove a PPC binary is portable by running it in qemu on x86. As long as there's a PPC or Java bytecode implementation, the hypothetical programs will work. Fact remains that given Java or C source code, the C code will end up being more portable.

> Last but not least, it's important to use the right tool for the right job. I would never write a VM in a JVM-based language [...] never write a GUI application in C.

I absolutely agree with using the right tool for the job, so never say never. Both examples can be perfectly reasonable ways to get a certain job done, we just don't see them very often.

Excellent points :) I always heard that the reason for undefined behaviour in C was to allow for certain operations to be implemented in a way that was faster for a given platform. Guess I was wrong.

I do disagree regarding comparing the JVM to Quemu though. A PPC binary can't be said to be portable when you're emulating the CPU that it was meant to run on. I do understand the point you're making though, we just happen to disagree.

Let's see:

Scenario 1: a bug in your program leads to incorrect program behavior because strlcpy just did what it was told and the client code didn't detect the error.

Scenario 2: a bug in your program leads to a buffer overflow vulnerability which can potentially lead to remote code execution.

I think I know which set of problems I'd rather have.

You are making the assumption that scenario one does not lead to an exploitable vulnerability... the whole point of this complaint is that randomly truncating strings because they don't fit in buffers because you believe "strlcpy is magically better than strcpy" actually causes a different class of potentially-exploitable vulnerability... it doesn't, as the person I responded to claimed, "fail safe".

Yes, it might lead to an exploitable bug, but it seems to me that it's a much lower probability than a buffer overflow leads to an exploitable bug. Surely an API that lowers the percentage of exploitable bugs isn't all bad (even if it's not perfect)?

Until about 2 years ago there wasn't really a better solution around...

It's usually pretty difficult to create an exploitable remote code execution bug in code without very obviously heinous (read: amateur level) code. Whereas with buffer overflows it's fairly trivial. Moreover, in order to exploit a more complex vulnerability an attacker would need to know something about the nature of the vulnerability, making it more difficult to come up with exactly the right steps to exploit it to maximum advantage. In contrast, with a typical buffer overflow you only need to tune how much junk to throw at the buffer before you start getting code to execute, and then you can just inject whatever existing payload you have already. This is so trivial that there are now tools designed to automate the process.

> You are making the assumption that scenario one does not lead to an exploitable vulnerability

Personally, I'm making the assumption that scenario two will lead to a probably exploitable buffer overflow whereas the scenario one might lead to another class of exploitable vulnerability.

By significantly reducing how much time I have to spend focusing specifically on the buffer overflows of scenario two, I have more time to spend on the other logical errors that may lead to vulnerabilities in scenario one. It's not fail safe -- nothing is in the most extreme interpretations of the phrase -- but it is fail safe against one specific failure case, which is valuable.

If you are going to switch to some different API in the hope of no longer having an exploitable vulnerability, how about just using a real string library, rather than trading in one class of bug for another one that you are hoping is less dangerous? Preferably, one where you can also store embedded NULL characters in the string, and thereby aren't also subject to strcmp failing to protect you from yet another class of exploit (due to C programs often truncating strings after null characters, which is a pretty common attack surface) due to using null-terminated strings in the first place?

FWIW, if people want to say "this API still sucks and users still have to be really careful when using it, but it slightly decreases a specific class of error" that's one thing, but that doesn't mean it "fails safe", nor is the panacea that people make it out to be: it is seriously dangerous for people to keep proselytizing that strlcpy is so fundamentally more safe that it can be used with reckless abandon (which, as I point out in the other comments I've made in this thread, is precisely what has happened due to the dogma that "strlcpy is secure and strcpy is not").

> If you are going to switch to some different API in the hope of no longer having an exploitable vulnerability, how about just using a real string library, rather than trading in one class of bug for another one that you are hoping is less dangerous?

Sure. Hell, I apply this to entire languages and prefer C# where you generally have to opt into rather than opt out of corrupting buffer overflows, and where finding the length of a native string literal is an O(1) operation rather than an O(N) operation.

But 3rd party APIs mean you will at some point have to deal with fixed length character buffers both for reading and for writing. Even in C#, if you ever find yourself needing to do native interop, you'll almost certainly eventually run into them.

> nor is the panacea that people make it out to be: it is seriously dangerous for people to keep proselytizing that strlcpy is so fundamentally more safe that it can be used with reckless abandon

I think the linked article shows we have a much bigger problem: people simply do not care about security and use strcpy with reckless abandon. I won't say "don't add strlcpy to your banned.h". I won't say, "don't annotate the documentation to point programmers to better alternatives". I won't say, "make use of the function generate warnings or errors in the default settings of your project, standard library, or compiler configuration".

But I'm unconvinced that strlcpy is ineffective at harm reduction, and thus that Christoph Hellwig's stance is the correct one.

Can you give examples of how incorrectly using strlcpy leads to vulnerabilities as severe and as common as buffer overflow?

Thanks.

This question is difficult to answer, because in its curtness it fails to ask the real question, which is "does claiming this function solves the security problems of strcpy lead to more of fewer exploitable vulnerabilities". People really need to be paying much more attention to their strings, and preferably using real string libraries rather than these low-level memory primitives as if they were a high-level string library. People also need to be paying close attention to these APIs in all cases where they use them: you don't get a "free pass" to write a bunch of string manipulation because you are using these l-function variants.

We then also have to take into consideration that actual stack overflows are nowadays rare with strcpy due to compilers being able to automatically swap in __strcpy_chk whenever sizeof the stack buffer is known at compile-time. Humorously, swapping strcpy out for strlcpy in these cases will take a situation where the program aborts when the buffer overflows (which isn't perfect, of course, as you might be able to use that to DoS part of a system that you need to temporarily go away while you attack something else) and replaces it with strings that randomly truncate their values at different points in your program's logic.

Heap-allocated buffers are still problems, but are also much more difficult to deterministically exploit. However, randomly truncated filenames, usernames, or keynames offer tons of opportunities to mess with program logic: get a system to verify properties of one path (which can be made arbitrarily long using a cyclic symlink) and then access another one, for example (which is incredibly common due to everyone's misconception that you can use a buffer of size MAXPATHLEN to store a complete path...).

I've read over several of your answers, and what your arguments boil down to (as far as I can tell) is that people use these functions wrong. While true, incorrect use can lead to an incorrect program, and that people's opinions of these functions thus need to change, does this mean that we shouldn't use this functions? If the goal is to reduce the possibilities of buffer overflows, wouldn't it be better to use these functions? Isn't it actually easier to find bugs involving strings, than bugs involving overflows?

These functions do solve a real issue. Of course, people not using the functions correctly should be made aware of that fact, but still, avoiding buffer overflows > bugs due to short strings.

I definitely agree, and the point about __strcpy_chk is an excellent one. However I still get the feeling that switching from strcpy to strlcpy is a pretty substantial and easy win even while the battle to get people to use bstrings or what-have-you continues apace. Perfect should not be the enemy of good, or at least better. Let's get the unwashed masses washed first before we get them pedicured and into tuxedos.

strl* fail just like any other C function in libc. They return, like snprintf(), the total length of the string and let the programmer deal with it.

strlcpy(buf, s, sizeof(buf)) >= sizeof(buf)

Rocket science.

Ulrichs reputation is well deserved, he worked very hard on it.

If you are going to do that check, there is almost no value to using this function as you can do a similar check before copying the string. (The only benefit you get is that if you don't know the length of the source string you save having to first calculate it; but that's a performance optimization, not a fundamentally different paradigm.)

The thing is, though: no one actually checks that return value, as this idea that using this API is what makes things secure has become zeitgeist. This post from OpenBSD is even making this mistake: they care only that people are using the API, not that they are using it correctly (despite pointing out the truncation issue at the end, they are just analyzing the usage, not the correctness). Just using this API is not a magic bullet: if you are using the API incorrectly, your program is still incorrect, and if you are using it correctly you can just use strcpy correctly.

If you don't believe me, this is a really depressing search to perform:

site:opensource.apple.com "strlcpy" -"strlcpy.c" -"strlcpy.s" -"strlcpy.3" -"strlcpy.h"

OpenSSH doesn't even check strlcpy correctly...

    317         /* No MIN_SIZEOF here - we absolutely *must not* truncate the
    318          * username (XXX - so check for trunc!) */
    319         strlcpy(li->username, pw->pw_name, sizeof(li->username));

Yeah, that's great: we "absolutely must not" do something, but it's marked as a XXX?!?!?

If the world wrote perfect code, nobody would have ever needed to replace strcpy. Water is wet.

Yet somehow, even you admit that they have some use because "almost no value" is infinitely more value than "no value". The benefit you describe is exactly what's beneficial. strlcpy does the length calculation, and prevents overflowing the buffer, which is a reasonable improvement over calling strcpy and trusting that previous checks (if they exist at all) are correct. Truncation (unless intended) can never occur when the program is correct, it merely exchanges one vulnerability for another one, which rarely can be exploited.

That noone checks for the return value is trivially false. That Theo only cares about usage and not about correct usage is also false. Just because something isn't in a mail doesn't mean you can come to the reverse conclusion and claim the person doesn't care about a topic not mentioned. Most people tackle bigger problems first, before getting into discussions about writing "perfect" code.

Given you are a perfect programmer (I assume), why don't you check the openssh code you quoted and let us know how you would've handled it? I looked at it, it's part of (portable) utmp logging and will truncate the username from passwd(5) to fit into the utmp structure. Nothing much can be done about it, because the two structures do not agree on common size limits.

Demanding perfection while being unwilling to improve the status quo is hypocritical.

Yeah, that's wrong and I'll fix it. Impact: if your username is > 511 characters long and truncates to another valid username that is exactly 511 characters long then you will see their "last logged in" date instead of yours.

Meanwhile, how many overflows have even the naive conversions of strc=>strl saved?

> how many overflows have even the naive conversions of strc=>strl saved?

We should ask you.

Isn't the point that strlcpy/cat are a better alternative to strncpy/cat, since the latter don't let you know at all that you didn't copy the entire string? And of course, past simple use cases, one would write one's own string library based on strlcpy/cat (for whatever cases in which it's not ok to include string length in the string struct).