Hacker News Reader

Comment by macgyverismo | Hacker News Reader

I was suprised at the C++ outcome, so I started testing myself. I was very amazed to find a 4~5x speedup when reading directly from file (std::ifstream) instead of reading stdin. This was on a 10.5.7 MBP 2014

[edit]

more like 9~10x actually...

  time { ./build/Release/countwords_ifstream; } | tail
  ...
  real 0m1.337s
  user 0m1.332s
  sys 0m0.022s

  time { ./build/Release/countwords_original < kjvbible_x10.txt; } | tail
  ...
  real 0m12.184s
  user 0m12.138s
  sys 0m0.044s

changes between these two:

  6a7
  > #include <fstream>
  8a10
  >  std::ifstream inp( "/Users/macgyverismo/Desktop/test/kjvbible_x10.txt" );
  13c15
  <     while (std::cin >> word) {
  ---
  >     while (inp >> word) {

View on HN

I dug into this once because reading from stdin on my Mac was so slow, but only when using std::cin. Using C APIs to read stdin was fast. Turns out libc++ on mac uses `getc` to read from stdin one character at a time. Madness.

Huh, well that makes a lot of sense. Three years ago when I was in University practicing for competitive coding we came across a limit in the performance (ms) we could get on even simple problems. The professor leading the practice poked away at his own solution to some problem for days because he couldn't get a better time than the top score online. Eventually he implemented his own versions of cin and cout. Needless to say he was pretty frustrated but there's just some problems that are hard to let go of without a solution.

Similar here, reading the file from stdin on a 2020 MBP on macOS 11.2.2 is incredibly slow.

    int main() {
        std::ios::sync_with_stdio(false);
        std::string str((std::istreambuf_iterator<char>(std::cin)),
                         std::istreambuf_iterator<char>());
    }

takes around 8 seconds to run, no counting at all.

    int main() {
        std::ifstream f("kjvbible_x10.txt");
        std::string str((std::istreambuf_iterator<char>(f)),
                         std::istreambuf_iterator<char>());
    }

runs in ~270 miliseconds. Factor 30x, yikes.

Could be a lock on stdin that is acquired in a loop? Rust has such a lock and typically one should lock stdin at the beginning and read from within the critical section.

Try this maybe? https://codeforces.com/blog/entry/6251

Asuming you mean 'std::ios::sync_with_stdio(false);', that was already present in the original code.

I tried that on Linux and got at best a 5% improvement.

Yeah, I'm on Linux too and tried that with no noticeable difference in results. There's something off about macOS's approach.

Is that due to something at the OS or filesystem driver level?

Reply on news.ycombinator.com