Stop Using Std::endl [video] | Hacker News Reader

Stop Using Std::endl [video]

articles.emptycrate.com

39 pointslefticus45 comments

Don’t optimize prematurely, especially since debugging time is a factor here.

If a long-running program crashes, a missing flush can be the difference between seeing or not seeing an important detail that preceded the crash. Instead of going straight to the cause, you might be wasting time exploring something that happened a bit earlier, or you may be forced to rerun the long program again to see the same issue with better logging.

Then there’s the inconsistency aspect; if you become used to "\n", and see it “work” all the time for streams that happen to go to the terminal, you will be confused when “the same” thing does not have the same behavior for a file stream. Use the abstraction ("endl") until you know you need it to be different.

Working around a broken API is not premature optimization. Stroustrup et al. point out the same problem with std::endl[1] and advise the same remedy as the presentation linked here. Conflating EOL with flush() was a mistake the moment it was first written.

Systems programming languages aren't about easy troubleshooting. C/C++ APIs should be performant by default because when they are not we end up working around them, often badly. If I want the stream flushed I'll configure it to behave as such or I'll call flush explicitly. Any other mindset belongs in Python, Java or some other higher level language.

[1] https://github.com/isocpp/CppCoreGuidelines/issues/357#issue...

I’m not defending the implementation (I hate most of std::iostream), just that "\n" is not exactly equivalent so switching to that may cause surprising behavior and create problems. Auto-flush has its uses.

If the main purpose of a program is to generate output then it ought to know what it’s doing anyway and know how to avoid poor performance. For most programs, I’d say that Unix rules of thumb apply: if you are already trying to be “silent on success”, your program will either print nothing or maybe an error or two and not have to worry about print performance. If your program might only print one error and that doesn’t even show up because a flush didn’t happen before a crash, there is a big problem.

> If I want the stream flushed I'll configure it to behave as such or I'll call flush explicitly. Any other mindset belongs in Python, Java or some other higher level language.

This is a bit unfair to HLLs, IMO. Flushing to disk can add unacceptable overhead even for, say, Python. (Particularly on spinning rust, but...).

The short and long of it is that "endl" should probably be deprecated in the standard and a good replacement created.

("std::newline"? Maybe we could just specify it as "\n" and get rid of the whole text/binary distinction that another poster on this topic was rightly complaining about.)

TLDR: std::endl causes a flush of the buffer to the file for every new line, vs "\n" or that doesn't. Flushing the buffer for every new line is costly, and will slow down your file write.

My addition (which might have been in the video, didn't watch it all): Don't worry about \r\n vs \n, if you're on windows outputting \n will convert to \r\n, as long as you opened the file in text mode.

Personally I use \n for line endings regardless of Windows or not. notepad.exe can't understand \n and displays everything on one line, but everything else can.

I've completely given up on Notepad compatibility. I can only assume that the reason Notepad in 2016 can't handle '\n' is due to pure spite by someone high up inside Microsoft, Hanlon's Razor be damned.

The sad part is it's not even just Notepad. It's the underlying "EDIT" HWND class itself. So anyone coding in raw Win32 APIs has to basically capture paste events from the clipboard to transform text into the "\r\n" state, and upon reading text out of it, potentially convert it back to "\n".

Probably a five-line patch to that class would eliminate so much extra work for developers. And we could drop the whole ridiculous "text mode" for working with files while we were at it.

Plus I'm sure there must have been thousands upon thousands of external requests to Microsoft to be able to use Notepad with all their non-Notepad text/configuration files over the years.

It's hard to believe this is anything but very deliberate at this point. Add to this, the mess Microsoft perpetuates with not supporting UTF-8 (std::wstring, wchar_t, _wfopen, non-standard iostream extensions for Unicode filenames, etc), and one really begins to have a deep seething hatred for Microsoft.

"The sad part is it's not even just Notepad. It's the underlying "TEXT" HWND class itself. "

That's it. Notepad is only a wrapper over a text edit. Nothing more.

That's like asking why does Windows use \ instead of /. It's not hatred, it's just... the way things work. Changing it would be very costly and would create compatibility problems for decades presumably, so why do it?

\ vs / is an interesting example. fopen() and family from MinGW can use / in paths on Windows. I use that feature all the time. cmd.exe can't, and I understand that would break things due to the use of / instead of - for flags in the Windows world.

Whenever it comes to something programming-related that might improve code portability between different OSes (C99, LF, UTF-8, etc), there's a mountain of after-the-fact justifications for why it would break backward compatibility and other such tripe.

But honestly, what would making "EDIT" accept "\n" alone do to break backward-compatibility? "\r\n" would still work, of course. Is there a group of people inserting stray "\n"s alone into text files, and expecting Notepad to silently ignore them and dump all of that text onto the same line? If so, those people are absolutely horrible :P

But okay, fine. Require a program .Manifest flag to opt-in, then another special ES_ACCEPT_LF flag with a caveat that it's Windows 10+.

Even all the people that have jumped through hoops with things like hooking paste operations to transform "\n" into "\r\n" for "EDIT" ... that would still work just fine, it would just be unnecessary.

The only cost would be that there's a short time where people are trying to use these "\n" only text files on older versions of Windows. But we've been dealing with the fallout of that anyway for the past 21+ years now.

You can't possibly tell me that supporting "\n" alone is an immense burden for a company with as much money and developers as Microsoft. No, I'm sorry, but this, C99, UTF-8, things like _wfopen, WSAPoll over poll, etc are all very much intentional: they're designed to maximize vendor lock in. Microsoft doesn't want your code easily moving between Windows and Linux. Even though they're a convicted monopoly with a mountain of evidence of their abusive actions, they still have so many white knights ready to defend their actions as benign =(

IIRC, '/' as a path component separator works just fine in every Windows API call that matters that takes a path and (unless I'm remembering wrong) has for a really long time. It works just fine in Windows 10's version of Windows Explorer. I would be surprised if it didn't work in W7's Explorer, too.

CMD.EXE is -sadly- a bucket of suck.

> ...WSAPoll over poll...

Don't fucking get me started about the WinSock API. >:(

The parent poster stated the advantages, but just in case: The CRLF vs. LF is essentially technical debt that millions of developers pay -- as long as Windows remains an interesting target for developers (or Microsoft fixes the CRLF thing). It's basically a continual downside vs., say, 10-20 years of compatibility problems.

(I don't buy the "decades" bit, but I'll accept it just for argument's sake.)

I'd say it's more of a TLDW here.

You can use C-library functions in C++, so just use fprintf, and you can flush however you see fit. I've been writing C++ since basically forever, and find the streaming operators in C++ to be way too much of a pain. It's doubly painful if you ever consider internationalizing your software.

fprintf(fd, "Number: %.3f\n", number)

Is a whole lot easier than

std::streamsize ss = std::cout.precision(); std::cout << "Number: " << std::setprecision(3) << number << std::setprecision(ss) << std::endl;

Since setting precision is modal, you have to set it back if you don't want to screw up future output.

Streams suck!

One thing that streams have that the c routines don't is type safety and overloading for user-defined output.

Also take a look at cppformat for some of the best of both worlds.

Yup, cppformat is nice. I could also have a nice type-safe wrapper around libc functions. It doesn't change the fact that the API in the language is tedious.

When you're doing internationalization, a common thing you might do is something along these lines:

fprintf(fd, translate_string(context, "Result %d of %d"), num1, num2);

The translation may be "%d of %d results" in another language. It's really difficult to map that into streams.

Yes, I'm over-simplifying, since you can't have arbitrary placeholders due to order differences between languages, but it's _much_ harder to write simple code with streams than libc, and I stand by that!

As for user-defined data, I've added .toString() functions on objects which I can call whenever needed, either from the ostream operator or pass the result to libc.

Modern compilers are smart enough to catch type safety bugs for most printf use-cases.

Also interesting to note is that as opposed to std::cout; std::cerr (associated with stderr) is automatically flushed, and that std::clog (also associated with stderr) is not automatically flushed.

http://en.cppreference.com/w/cpp/io/clog

I guess it's fitting that something called "clog" does not automatically flush. :-)

I believe the logic is that error handling (std::cerr) is optimized for being immediate because you need to see that last error message before your program went off the rails.

Logging (std::clog) on the other hand is optimized for throughput since you expect to use some of that even in the happy path and don't want to bog everything down. I believe that since they're both associated with stderr, the error handling routine will also flush the last of the log.

One case where you would want to use endl and not \n is if you're outputting to something like a log file that's being monitored with tail -f.

This is part of the coding standard for LLVM for years: http://llvm.org/docs/CodingStandards.html#avoid-std-endl

Of course if you're debugging you want to flush ASAP.

no. you put that thing in the language. fix the language. This kind of nonsense is why people don't like using c++, I remember Qt had a similar thing with QThread. They published a popular article called QThread you're doing it wrong, and it turns out their own documentation was doing it wrong.

endl is not broken, and does not need fixing. it's one of several ways to do things and this video is pointing out that its documented features may not be exactly what you want, and alternatives are available.

However, sometimes endl is indeed what you want. There is nothing to "fix" here, except maybe your attitude.

I think there's a valid argument to be made that it is at least poorly named, and that naming has resulted in people using it wrong, including among people who teach the language and so compound the problem.

Agreed, names are the first line of documentation. Having endl flush is like using single letter variable names, but maybe a bit worse since single letter variables make you go find where they were declared but this lulls you into a false sense of security.

This violated the principal of least surprise enough that someone made a 10 minute long video on it :) It's really hard to blame the user this time.

People not using descriptive names for the important stuff is probably my number one pet peeve. We're not on a VAX, bytes aren't precious, and we aren't naming terminal commands, keystrokes aren't either. And nowadays every editor has autocomplete, so there's really no excuse.

The great vowel shortage of the late 80s is finally behind us :)

I don't see any reason why, in the few instances you actually intend to flush, you won't just say << std::flush. This will make it very clear that you intend to flush the output. And we should definitely be prioritizing the documentation of our intentions over shaving a few bytes off of our source code file sizes.

It was weird to ever merge two unrelated functions into something, and then name it something as cryptic as "endl". Especially when the most common case novices would encounter this would be std::cout, where terminals will automatically flush text to the screen upon receiving "\n" anyway.

Except in C++, std::endl has been documented from the start to end the line and flush the buffers.

But why couldn't it have been named std::eol_flush? or something else that communicates what it actually does?

Because that name is too verbose for regular use. People would hate it and make their own ad hoc nicknames for it not to have to type eol_flush:

   // ef -> std::eol_flush
   template class <typename X, typename Y> basic_ostream<X, Y> &ef(basic_ostream <X, Y> &os)
   {
     return std::eol_flush(&os);
   }

A four-letter name could have been found with better mnemonic value covering the full semantics. We can look toward assembly languages for inspiration: they pack a lot into one, two, three or four letters. For instance std::nlfl (add NewLine and FLush).

It's been years since I used C++ iostreams. I totally forgot that endl also flushes. When I saw this submission, my reaction was: what is this fuss about. Then "oh, right! That endl manipulator flushes, even on a fully buffered stream!". The name didn't remind me of this at all; but "nlfl" would almost certainly have served that purpose.

Who said it couldn't have been named std::eol_flush?

And why do you think its name (std::endl) doesn't match its effect (ends the line)?

It does match one effect, but it doesn't match the most important effect (flushes the output stream). And so a lot of people get the wrong idea. (I had multiple experienced professors teach me that std::endl was the "C++ way" as opposed to "\n" being the "C way").

You always want to follow the Principle of Least Astonishment. Suppose a hypothetical programming language has two DSLs for interfacing with SQL databases. Which of these is immediately understandable without having to consult the docs? Which one are you more likely to "learn" in a way that litters your codebase with subtle bugs?

  // Returns the SQL query as a string
  DB.table("users").where("id").in($args).toSQL()
  
  // Returns the results of the query
  DB.table("users").where("id").in($args)
  
  // Locks the DB connection and returns the results of the query
  DB.conn.lock() { DB.table("users").where("id").in($args) }

vs.

  // Returns the SQL query as a string
  DB.query.find("users", ["id", sqlSafe($args)])
  
  // Returns the results of the query
  DB.query.find("users", ["id", sqlSafe($args)]).run()
  
  // Locks the DB connection and returns the results of the query
  DB.query.find("users", ["id", sqlSafe($args)]).exec()

You assume "ending the line" means only "move to the next line". But "ending the line" could just as easily mean "ensure the line reaches its destination".

If it was called std::newline I would agree with you about it being misnamed. But it isn't called that.

Could the name be better? Maybe. Bike shedding.

>You assume "ending the line" means only "move to the next line". But "ending the line" could just as easily mean "ensure the line reaches its destination".

Besides the other suggestions, I'd say something like std::eof_nl would make that interpretation a lot more obvious. Although like others I'd question the need for it at all as opposed to just an explicit "\n" << std::flush .

>Bike shedding.

I don't think anything is "bike shedding" when it comes to defining the standard library of a programming language (especially one for which all updates to the standard must maintain 100% backwards compatibility).

(Not) choosing descriptive identifiers might not have the glamour of defining semantics or optimizing performance, but it can have huge real-world implications.

The (std::endl == "\n") misconception is enormously widespread even amongst experienced C++ people, and is one of the most common sources of performance issues with the language.

You would propose std::eof_nl and have it not mean "end of file something"?

> [std::endl] is one of the most common sources of performance issues with the language.

Citation needed.

> I remember Qt had a similar thing with QThread. They published a popular article called QThread you're doing it wrong, and it turns out their own documentation was doing it wrong.

The main thrust of that article [1] was suggesting a new way to use the existing class and pointing out common mistakes he'd seen. Yeah, the docs were slightly outdated and needed a little TLC. They didn't mention the alternatives that had recently been made possible, but that's hardly an earth-shattering mistake.

[1] http://blog.qt.io/blog/2010/06/17/youre-doing-it-wrong/

Or, stop using C++

That is like saying that because pigs fart we should abstain from bacon. Would make sense, but not feasible.

Yeah, I'll just go convert the LibreOffice codebase to Go right away.

Finished already? I have a JVM that waits to be converted, so I can be sure that no C++ pollutes my Java code. Kidding aside, I'm still curious if Rust/Go (or whatever) will be serious contenders to develop current c++ codebases further, e.g. the old code stays c++ but new code will be written in something else.

Rust is probably more appropriate for that, since (like C++) it uses system APIs directly instead of importing a runtime.

If you have the man power to do it, then please feel free. But on an active project, the advise to just not use C++ is not really ever something that can be practically achieved.

I sure will, just as soon as there's full Qt bindings for Rust.

Reply on news.ycombinator.com