Hacker News Reader

How in the world is this "faster" as stated?

If you read the whole file, then process that file, the program first blocks on disk I/O for the time it takes to read the file, then it blocks the CPU for the time it takes to process it.

By streaming the data, you can do both at the same time. (The OS will already fetch the next line from the disk while you are still processing the previous one.)

If you're processing something one line at a time, and outputting something based on each line, then you don't need to read-and-process the entire file before printing everything out.

Other than that, it's more memory efficient.

That's not "faster" (i.e. it doesn't take less time to run), it just has less delay from when it send back the first piece of output.

As part of a larger system, removing the delay can cause other parts of the system to do their processing in parallel. While this doesn't reduce the amount of time that it takes that piece of the pipeline to run, it will reduce the total runtime of the system.

It can also be faster if you're searching for something in the file, because you can short-circuit reading the rest of the file when you find it.

Reply on news.ycombinator.com