Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
benhoyt.com
benhoyt.com
I'm no expert in any of the languages used except maybe C, at least I'm quick to form opinions about C code. In C, I'm not so sure that there is a well-established "idiomatic" concept, since C is not so tightly bound to a community as more modern languages. At least that's my feeling.
That said, I was shocked to see that the example code did heap-allocations to store integers. Single integers, one per allocation, per "counting bucket". I would store the count in the data pointer without a second thought, I just "can't" make 4/8-byte heap allocations and still sleep at night. Especially after all the concern about processing the input in large blocks for performance! Casting an integer to/from a void pointer should be pretty safe according to my gut feel. If that turns out to be false, I would dump the libc's hash API altogether, since it makes me do heap allocations to store single integers.
I tried making the change locally, and the time taken (as measured with the OP's benchmarking program) dropped from 0.11 to 0.10 on my system (a Dell laptop running an Intel i5-7300). That's at least something.
As I said, and others have commented, it requires some complicated castery to avoid UB, but it's totally doable (and at least GCC does the right thing, while warning, with no advanced casts).
Thanks for an interesting article!
If you want to round-trip an integer through a pointer and back to the same integer, you can use any integer type provided it is not bigger than intptr_t or uintptr_t.
I thought POSIX added a bunch more safe pointer casts, but I don't know off the top of my head...
Coming up with a benchmark is hard. Benchmarks like these are models. All models are wrong. But. Some are useful.
Follow you thinking to its logical conclusion. If you open up the parallel programming flood gates, now you need to do that for all of the programs. Now your effort to contribute a sample for each language goes up. Maybe enough where actually doing the benchmark isn't practical.
So let's play the tape. If someone told me, "sure, have fun, throw threads at this!" I'd probably say, "can I use memory maps?" And let's say, "yes, sure, go for it!" I say okay, here's my first crack at it:
1. Hope stdin is a file and open it as a memory map.
2. Split the memory map slice into N chunks where N is the number of logical CPU cores on the system.
3. Run approximately the same algorithm devised in existing code submissions in N different threads, with one thread per chunk.
4. At the end, join the results and print.
If that ends up being the best strategy overall, then your model has become over-complicated because performance now likely depends on step (3). Which is exactly what the existing benchmark is. Now, maybe some languages have a harder time passing data over thread boundaries than others and maybe that impacts things. But that doesn't apply to, say, C, Rust, C++ or Go.
If that's not the best strategy, then maybe there is a cleverer approach that involves synchronizing on one shared data structure. I doubt it, but maybe. If so, your benchmark turns into one that is very different. It's no longer just about data transformation, but now you have to worry about synchronization trickiness. And now you're way outside of a standard goroutine pool that you might have used in Go.
Building a good benchmark is an art. Sometimes the constraints look dumb and arbitrary. But you have to sit down and really look at it from a bunch of different angles:
* Do the constraints approximate some real world conditions?
* Do the constraints inhibit writing code that looks like real world code?
* Do the constraints inhibit optimizing the crap out of the code?
* Do the constraints make the benchmark difficult to create in the first place?
* Do the constraints permit easy and clear measurement?
There are probably more things that I didn't think about.
This is the source for it:
https://github.com/raitechnology/raikv/blob/master/test/ctes...
The speedup of the multi-threaded version vs the single-threaded version is about linear. The single threaded version uses 2 threads, one to read stdin and one to hash the keys, the 16 threaded version uses one thread to read, 16 to hash.
$ time ctest -t 1 < ~/data/enwiki-p10p30303 18.88user 0.25system 0:09.58elapsed 199%CPU
$ time ctest -t 16 < ~/data/enwiki-p10p30303 8.08user 0.25system 0:00.49elapsed 1680%CPU
Consider re-reading my comment. It anticipated your criticism and responded directly to it.
If a "dumb" thread pool is indeed what ends up being the best concurrent solution, then Rust, Go, C and C++ (and probably most of the other languages) are going to have almost no problems implementing it. It's just Not That Interesting.
Now, maybe I'm wrong. Maybe there is something clever here to exploit in a concurrent program, and that some languages let you do that easier than others. I doubt it. It would be interesting if it turned out to be the case, but it would be a different benchmark. It wouldn't just be about how well a language can deal with simple data transformation tasks, but also about how it deals with tricky concurrency problems. But this doesn't seem like the kind of problem that needs such things. It complicates the model.
Like honestly, just saying that any benchmark that ignores parallelism "isn't very useful" is just totally ridiculous.
Incidentally, this problem set the scene for a wizard duel between two computer scientists several decades ago. In 1986, Jon Bentley asked Donald Knuth to show off “literate programming” with a solution to this problem, and he came up with an exquisite, ten-page Knuthian masterpiece. Then Doug McIlroy (the inventor of Unix pipelines) replied with a one-liner Unix shell version using tr, sort, and uniq.
http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...
> Knuth has shown us here how to program intelligibly, but not wisely. I buy the discipline. I do not buy the result. He has fashioned a sort of industrial-strength Fabergé egg—intricate, wonderfully worked, refined beyond all ordinary desires, a museum piece from the start.
Well, yes – absolutely.
If you read the initial column[1] pre-McIlroy's response, you'll see that Knuth is not presenting a word-counting solution, but instead the concept of literate programming and how intertwining code and prose can be better for the programmer. Knuth isn't trying to count words, he's trying to teach you a different way to develop, and he's doing it by showing an example that has been simplified so the average person can chew on it.
I'm not sure if the popular interpretation of "ivory tower Knuth versus tactical genius McIlroy" is a modern take or if that was how it was received when the column was published, but it feels very unfair to present it as such today.
In modern terms, it would be something like complaining that $LANGUAGE's web server implementation is a grotesque bloated mess (and therefore $LANGUAGE isn't very good) because in shell all I have to do is run "nginx". There is a true and useful sense in which that is true, but there is also a true and useful sense in which that is utterly missing the point.
I have written more about it here: https://henrikwarne.com/2018/03/13/exercises-in-programming-...
I don't think it was linked from the article
My one line has less features, but I'll take it.
That said, if you are new to that, Programmer Pearls is a great series. And literate programming is neat to at least know of.
[1] https://news.ycombinator.com/item?id=24817594
[2] https://github.com/c-blake/adix/blob/master/tests/wf.nim
Honestly, the optimized version you created is kind of opaque; I don't want people to think that that's what "typical" nim looks like.
I'll do a pull request on the project for mine as simple (I'd have added your version as an optimized version, but I couldn't get it to compile).
On my machine, the simple-c version runs in 0.70 seconds, my simple nim runs in 0.73 seconds (and they produce the same output, which is nice :-), the nim executable is 95k.
[0]: https://github.com/csterritt/word_frequency_nim/blob/master/...
Feedback-wise, yours isn't so bad, though I haven't tested it. There is definitely a simpler one that looks roughly like the Python and probably runs a bit slower than yours. That would probably be a better/more fair candidate for the "simple" category, but it probably still has ok-ish performance.
import tables, strutils
var counts = initCountTable[string](16384)
for line in stdin.lines:
for word in line.toLowerASCII.split:
counts.inc word
counts.sort # SortOrder.Ascending to reverse
for word, count in counts:
echo word, " ", count
If we apply "BS-scaling" to the article table that would be 2.27*.2 = 0.454 sec on the author's machine which would make it twice as fast as the "simple C". Yet, if I actually run the simple C, I get 0.651 seconds, so only 651/454=1.43x "simple C" speed. This mismatch is, again, bigger than Go vs. RustB (0.38/0.28 = 1.35x). My only point is that you cannot just apply BS scaling and these results may very well fail to generalize across environments.
For what it's worth, literally every time I re-do anything in Rust in Nim, the Nim is faster. But benchmarks are like opinions...everyone has them and you should very much form your own, not delegate to "reputation". The best benchmark is actual application code. I make no positive general claims, but say this only because Rust people so often do. { EDIT: I think it is generally a big mistake to make many assumptions about prog.lang performance, especially if the assuming is on the "must be fast" side. }
And, cool, "BS-scaling" is my new term of art :-)
0.00 0.00 32187/32187 std::vector<std::pair<std::string,int> >::vector<std::__detail::_Node_iterator<std::pair<std::string const, int>, false, true>,void>
(
std::__detail::_Node_iterator<std::pair<std::string const, int>, false, true>,
std::__detail::_Node_iterator<std::pair<std::string const, int>, false, true> > const&
) [11]
0.0 0.00 0.00 32187 void std::string::_M_construct<char*>(char*, char\*, std::forward_iterator_tag) [8]
[edit]
more like 9~10x actually...
time { ./build/Release/countwords_ifstream; } | tail
...
real 0m1.337s
user 0m1.332s
sys 0m0.022s
time { ./build/Release/countwords_original < kjvbible_x10.txt; } | tail
...
real 0m12.184s
user 0m12.138s
sys 0m0.044s
6a7
> #include <fstream>
8a10
> std::ifstream inp( "/Users/macgyverismo/Desktop/test/kjvbible_x10.txt" );
13c15
< while (std::cin >> word) {
---
> while (inp >> word) { int main() {
std::ios::sync_with_stdio(false);
std::string str((std::istreambuf_iterator<char>(std::cin)),
std::istreambuf_iterator<char>());
}
int main() {
std::ifstream f("kjvbible_x10.txt");
std::string str((std::istreambuf_iterator<char>(f)),
std::istreambuf_iterator<char>());
}
I suppose SBCL would be in a similar ballpark for this task, though, and would love to see an unoptimized SBCL version.
But like all GCs, there are cases where it has issues. Anecdotally, I've seen multiple cases where "ballast"[1] reduced RPC response times by around 25%. A bit of GC tuning on some tests sped them up by over 10x (due to pathological thrashing - each test would go from just-below the GC threshold to just-over). Etc. Granted, it's usually on not-great code, but the point still stands - if you care about performance, GC cannot be ignored, regardless of how good its marketing is.
[1] https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...
Some domains might state a worst case for a single transaction or percentage of transactions that definitely exceeds what can be achieved here
$ wc -l words.txt SCOWL-wl.txt
99171 words.txt
662349 SCOWL-wl.txt
761520 total
# finding common lines between two files
# shorter file passed as the first argument here
$ time gawk 'NR==FNR{a[$0]; next} $0 in a' words.txt SCOWL-wl.txt > t1
real 0m0.376s
$ time perl -ne 'if(!$#ARGV){$h{$_}=1; next}
print if exists $h{$_}' words.txt SCOWL-wl.txt > t2
real 0m0.284s perl -lane '$c{lc $_}++ for @F;END{print "$_ $c{$_}" for sort {$c{$b}<=>$c{$a}||$a cmp $b} keys %c}' < input.txt
perl -nle '$w{lc s/[[:punct:]]//r}++ foreach split(/\s+/,$_); print map{"$w{$_} $_ \n"} (sort{$w{$a} <=> $w{$b}} keys(%w)) if eof()' < kjvbible.txt
If anyone's curious this takes .47 seconds on my local machine whereas the optimized shell solution takes about 3.5 seconds.
Edit: It looks like mhd's usage of autosplit -a and @F are faster than split($_) but calling print foreach key is slower than my print map strategy.
So our solutions are combined to get:
perl -anle '$w{lc s/[[:punct:]]//r}++ foreach @F; END{print map{"$w{$_} $_ \n"} (sort{$w{$a} <=> $w{$b}} keys(%w))}' < kjvbible.txt
Which runs in .39 seconds. mhd's solution runs in .42 seconds. For reference the optimized C version takes .059 seconds (possibly cause of clang?), wc -w takes .023s and the optimized python version takes .288
To be fair, I once came across a sentiment that was pretty close to "readability is bad". Taken from both the (bourne) shell and awk, perl has lots of short variables that determine parsing input and other matters, like "$/". You can then `use English` to have alternative names for that, but those might not be as well known ("$/" would be "$INPUT_RECORD_SEPARATOR", or "$RS" as a tribute to awk). So the line noise might be more common and thus more understandable than the "readable" version.
I don't quite buy that for e.g. "$INPUT_RECORD_SEPARATOR", but there's an argument to be made for "$RS" or "$ARG" being worse than the Asterix swear words.
And all great perl hackers deliver readable code. But we still enjoy a round of perl golf once in a while. And honor demands, that, if you respond in perl golf style to an interview question, you make it count ;)
perl -anle '$w{lc s/[[:punct:]]//r}++ foreach @F; END{print map{"$w{$_} $_ \n"} (sort{$w{$a} <=> $w{$b}} keys(%w))}'Throw in some needless array-referencing-and-dereferencing so people can rightfully consider perl a write-only language...
Perl has plenty of "good parts", too. But it is not an easy language to learn and even good idiomatic Perl code can be difficult to read. I don't blame people for favoring Python and Ruby.
That also makes it hard to win for total outsiders, though. This touches on core functionality, which often has quite a legacy behind it and thus its own vernacular. I can write `while (my $line = <STDIN>)` instead of `while(<>)`, but I can't get as easily rid of "chomp" being confusing to most people. A bit like car/cdr in Lisp.
But that happens even if you have plain english functions and no special cased syntax. Looking at the simple python version, it took some effort for me to remember what `Count.update(lst)` does, or that `most_common` without arguments returns all elements.
Since learning perl a couple years ago I personally only use it for one liners to replace sed/awk. If I do use perl in a script it’s usually in one liner form in a bash script to post process output from something like ripgrep.
Regex handling is still far nicer in perl than anywhere else, so for any script which is primarily about string parsing with regular expressions, perl is the right tool for the job.
Is there, in 2021, a "use strict" equivalent for Python, or can one still misspell variable names and not be warned about it?
Although maybe it's hard to judge a programming language separately from the culture around that programming language.
I haven't looked into it yet (whether the benchmarks are comparable) but it would be interesting to check the programs there against the ones in this post, and either update the benchmarks here or add new submissions to that question.
For a while the fastest program there was one in Rust, then out of curiosity I translated Knuth's program from 1986 into C++ and a simplified version based on it was even faster; currently a Rust translation of the idea (a custom trie using not too much memory) is the fastest one there. (In fact, the current fastest two Rust submissions there came after I linked to the question from HN in Feb 2020, in a couple of comments at https://news.ycombinator.com/item?id=22221592 and in the discussion on a "Donald Knuth was framed" article: https://news.ycombinator.com/item?id=22406070.)
Edit: I tried to do the comparison myself but one issue I ran into (apart from minor things like taking input from filename vs stdin, and printing top N words versus all, which are easily handled) is that the benchmark on the StackExchange question considers "words" to be made of alphabetic characters (a–z), and the trie solutions make use of that fact (a trie node has 26 children etc), while the benchmark from this post includes punctuation (e.g. it has "words" like "him," and "him." i.e. "him" followed by a comma or period are counted as separate words). I gave up at this point, but maybe someone with more energy could modify the benchmark here and try comparing them.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Here, they have code samples and multiple implementations of each task in each language.
It's a great resource for, at a glance, how performant a language is at basic cpu and memory intensive tasks and how simple the related code is.
It still has the issue that some code is unrealistically optimized for the language. And also that many solutions use standard libraries written in other languages. It's worth looking at the source, therefore.
For the benchmarks game, C is not free to implement an optimized hash table for the problem.
The hash table used is from "Klib: a Generic Library in C".
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
// Define a custom hash function to use instead of khash's default hash
// function. This custom hash function uses a simpler bit shift and XOR which
// results in several percent faster performance compared to when khash's
// default hash function is used.
Any of the programs are free to implement a "custom hash function" and many of them do.
hash table != hash function
Wikipedia is your friend -- "A hash table uses a hash function to compute an index, also called a hash code …"
Which languages shown on the benchmarks game website 'don't allow to specify custom hash functions …" ? https://en.wikipedia.org/wiki/Hash_function
2 tasks — pidigits & regex-redux — explicitly allow use of standard libraries (GMP, PCRE2, RE2) written in other languages.
8 tasks do not.
(Those standard libraries were allowed because some language implementations provided arbitrary precision arithmetic or regex by wrapping those standard libraries.)
Of course, realistic use of some languages relies on calling standard libraries written in other languages.
The performance of Java there is super impressive. It should port relatively quickly to this file too...
var map =
new BufferedReader(new InputStreamReader(in, UTF_8))
.lines()
.flatMap(compile(" ")::splitAsStream)
.collect(groupingBy(w -> w, () -> new TreeMap<>(reverseOrder()), counting()));
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.TreeMap;
import static java.lang.System.in;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.Collections.reverseOrder;
import static java.util.regex.Pattern.compile;
import static java.util.stream.Collectors.counting;
import static java.util.stream.Collectors.groupingBy;That's definitely a flaw in the language, but like with C you would just use a different implementation for this purpose.
Note that tolower is called for every /character/. I don't know everything it does, but I know it's doing some semi-esoteric locale stuff, and in general it's doing a lot more than a naive ascii implementation would. I seem to recall it's even doing a dynamic_cast in there somewhere. For each character of input.
I was curious, so I ran callgrind on it. Here are the top 10 offenders (by Ir, 'instructions retired'). For clarity I have elided many details. Total Ir is 5,647,099,795, so the stuff below accounts for greater than 50% of all Ir.
1,079,594,033 std::istream& operator>>(std::istream&, std::string&)
553,657,533 std::istream::sentry::sentry(std::istream&, bool)
550,159,646 __cxxabiv1::__vmi_class_type_info::__do_dyncast(...)
426,992,280 __dynamic_cast
413,240,330 std::_Hash_bytes(void const*, unsigned long, unsigned long)
383,294,340 tolower
230,374,111 std::string::_M_append(char const*, unsigned long)
223,653,549 /usr/include/c++/9/bits/stl_algo.h:main
172,438,098 __strcmp_avx2
164,226,680 std::ctype const& std::use_facet(std::locale const&)You pay for this with extra memory allocations and non-contiguous memory access.
In most low level benchmarks, other collision mechanisms (probing, etc) perform much better.
Also, nice to see Forth included. I strongly suspect that if the Python solution were unable to leverage its rich and well optimised standard-library, it would be easily outpaced by Forth.
Also, the Forth solution used Gforth, which is far from the fastest Forth engine around. [0] It would likely have performed close to C if they had used a native-code-compiling Forth engine (SwiftForth, VFX Forth, or iForth, all of which unfortunately are proprietary payware).
[0] https://github.com/ForthHub/discussion/issues/88#issuecommen...
collections.Counter is a very straightforward collection backed by dict. The code isn't specially optimized, you can write the same code yourself with dict and even avoid (probably a negligible amount of) overhead.
Now, dict is optimized, but that's a builtin type. You don't need PSL.
https://github.com/python/cpython/blob/2fe408497e6838b6fb761...
Of course, they're right to optimize this way, and it's to Python's credit that it's possible to leverage this approach so successfully using only the standard-library, but I think my earlier point stands.
Again, very straightforward, based on dict. Python itself does not function at all without the dict implementation since it's the underpinning of the object model.
str is a builtin type. Calling str.split() a standard library method is... okay.
try: # Load C helper function if available
from _collections import _count_elements
except ImportError:
pass
> Python itself does not function at all without the dict implementation since it's the underpinning of the object model.
Sure, but I don't see the point here.
> Calling str.split() a standard library method is... okay.
For our purposes there's no reason to draw a distinction between standard-library functionality closely integrated into the language, and standard-library functionality that isn't. The relevant point is that computational work is being handed off to optimised C code.
Again, if you're optimising Python, the smart move is indeed to make maximal use of the optimised machinery in the standard-library (or for that matter some other dependable library). My point still stands: it would be interesting to look at a problem that isn't amenable to this approach, where the performance of the Python interpreter itself would be brought to the fore by necessity.
How easy it is to find such a problem will be a function of how good a job Python does of providing applicable optimised functionality in its standard-library. Perhaps something like computing matrix determinants? I don't know enough Python to say.
It would still be interesting to see for benchmark purposes.
It wouldn't matter, you'd be dealing with PyObject pointers in C anyway.
The same isn't true for, say, string operations, where there might be plenty of opportunity to move tight loops from Python to C and speed things up considerably.
Reading the bug report that introduced Counter[1], the author states it is the simplest implementation they came up with, and Guido and other maintainers prioritized simplicity.
No article like that is complete without a little C++ critique :-)
> I think it’s the simple, idiomatic versions that are the most telling. This is the code programmers are likely to write in real life.
I very much agree with him on this statement. If people started writing "optimized" versions of algorithms (on that level); who would ever be able to read others peoples code...
The one thing I would say is: as you dig more into C++ perf, you can develop an intuition for the kinds of things the compiler can and can't optimize. For instance, the C++ example from the article uses `std::unordered_map`, which is just an absolute mess from a cache locality perspective—the best compilers today (or of the foreseeable future) can't do a thing to fix that. Improving the programmer's choice of data structure is just not on the radar. :(
It seems that the two hot spots are the std::string constructor (unsurprisingly) and the vector constructor.
Making string faster is not easy. I would check that gcc is configured to use C++11 strings with the small string optimization and not the older C++03 compatible refcounted string. Writing your own string or a custom allocator are options, but probably overkill for the problem.
The vector constructor is more surprising. It is possible that -O3 could help a bit there. Wrapping the map iterators with std::move_iterator would also avoid a lot of string copies.
Maybe use a vector-of-pointer-to-pair, since we're leaving the map around and we don't really need the vector to own anything.
I used Rust and thought it's the textbook case for a trie, but what I found is that crates.io had quite a few tries but not all of them working or ergonomic enough. Only one of the libraries I tried (huh) gave me a slight advantage over a HashMap, although statistically insignificant, which is rather disappointing, considering the program outputted sorted frequency groups.
I'm pretty pleased with the results. I considered adding stemming[1], silly me. Too hard of a problem to bother.
Language | Simple | Optimized | Notes
------------- | ------ | --------- | -----
`wc -w` | 0.17 | 0.16 | `wc` reference; optimized sets `LC_ALL=C`
`grep` | 0.53 | 0.54 | `grep` reference; optimized sets `LC_ALL=C`
Go | 0.82 | 0.29 |
C | 0.83 | 0.20 |
Rust B | 1.10 | 0.28 | also by Andrew: bonus and custom hash
JavaScript | 1.29 | | no readline
JavaScript | 1.81 | | with readline
Rust A | 1.55 | 0.30 | by Andrew Gallant
Python | 2.14 | 1.17 |
Ruby | 3.59 | |
AWK | 4.28 | 1.22 | optimized uses `mawk`
C++ | 6.68 | 0.83 | "optimized" isn't very optimized
Shell | 42.60 | 9.50 | optimized does `LC_ALL=C sort -S 2G`
//with readline
const rl = require('readline').createInterface(process.stdin);
const wordCounter = {};
rl.on('line', (line) => {
let words = line.toLowerCase().split(/\s+/).forEach(word => {
wordCounter[word] = (wordCounter[word] || 0) + 1;
});
}).on('close',() => {
let output = Object.entries(wordCounter)
.sort(([,countA],[,countB]) => countB - countA)
.map(wc => wc.join(" "))
.join("\n");
console.log(output);
});
//without readline
const wordCounter = {};
let lastChunk = '';
process.stdin.on('data', chunk => {
let words = (lastChunk + chunk.toString().toLowerCase()).split(/\s+/);
if (!words.length) return;
lastChunk = words.pop();
for (let word of words) {
if (word) wordCounter[word] = (wordCounter[word] || 0)+1;
}
}).on('end',() => {
if (lastChunk) wordCounter[lastChunk] = (wordCounter[lastChunk] || 0)+1;
let output = Object.entries(wordCounter)
.sort(([,countA],[,countB]) => countB - countA)
.map(wc => wc.join(" "))
.join("\n");
console.log(output);
});Finding the "lower case of a character" is immensely harder in face of unicode, because the table is way larger and there's no nice speed hacks by manipulating an index into the ASCII table. JFGI: "tolower performance unicode".
If you notice setting LC_ALL makes a difference in performance for you, you ought to be aware that now you're no longer comparing same capabilities.
I'm not sure how to constructively state that except for: One shouldn't compare ASCII and unicode "tolower" in performance comparisons, as you end up comparing apples and oranges (at best. More like apples and snails) - which I did.
If the author uses code that uses "tolower" in job interviews, i.e., evaluates candidates based on their input WRT case normalization - and he even writes ("This is Unicode-aware ...") - he should know that Unicode awareness is not ubiquitous, and comes at quite a cost. Knowing about unicode awareness, one would assume he'd be aware of not making a fair comparison.
The only one I see is the comparison between 'simple' and 'optimized'. But that's more about "what does a simple idiomatic solution look like" and what does an "optimized and possibly less simple" solution look like. That comparison isn't designed to be apples-to-apples in the way you're saying. The simple variant will use Unicode-aware casing in environments where that's the simple and natural thing to do.
IIRC, most of the 'optimized' programs are using ASCII casing. Python doesn't, but casing isn't even close to the bottleneck in that program.
Also, as somebody else mentioned, iostreams are slow. Even simple reading of file can be several times slower than plain C (http://0x80.pl/notesen/2019-01-07-cpp-read-file.html).
https://en.cppreference.com/w/cpp/io/ios_base/sync_with_stdi...
"This line makes it run almost twice as fast:
ios::sync_with_stdio(false);"I've seen ripgrep (in Rust, also by article contributor Andrew Gallant aka burntsushi) outperform grep in almost every way possible, both in benchmarks and in my experience.
In the case of grep and wc, the "optimized" variant is running with LC_ALL=C, which sets the locale to C. Presumably, the non-optimized variant is using the OP's default locale, which is maybe something like en_US.UTF-8. (Apologies for the US assumption, but what matters is that it's not the C locale.)
This is a common trick for speeding up GNU utilities when you're only working with ASCII data. It also works for things like `sort`.
In the case of grep, sometimes running with and without the locale will be about as fast, particularly if you're only searching for a simple ASCII literal pattern.
I suppose they are included as a baseline for reading a file and for tokenising strings into words. See https://github.com/benhoyt/countwords/blob/eb2a8adf21c895907...
The loop ending condition might miss non-handled `remaining` (when there is no new-line at the end of the file). (The fix should be simple. Just move the check one line below.)
When there is no newline in some chunk, it would handle the word incorrectly at the chunk boundaries. (But ok, with the constraint that lines cannot be longer than the chunk size, this should not happen. But this could have been fixed easily anyway. Just `remaining = chunk; continue` + the other fix.)
Does he have a solution for the problem using grep, or is he just including "grep x <input" as a reference point for how fast you can read a file?
[1] https://github.com/benhoyt/countwords/blob/eb2a8adf21c895907...
And even then, you can't memory map all kinds of files. This is what happens when you assume that you can:
$ ag MHz /proc/cpuinfo
$ grep MHz /proc/cpuinfo
cpu MHz : 2300.000
cpu MHz : 988.934
cpu MHz : 2300.000
cpu MHz : 800.044
cpu MHz : 2300.000
cpu MHz : 2300.000
cpu MHz : 2300.000
cpu MHz : 1100.949
As for "bad spatial and temporal locality," can you be more specific? I guess the only thing I can see is perhaps inlining words smaller than some size into the same allocation as the hash table. But the hash table is otherwise one contiguous allocation.
One could use a lexicographic Btree-like structure with multiple characters per node, sharing consecutive cache-lines. For looking up the word "bar", you would traverse the "b" key from the root node, then its child "a" node, then the "r" child of the "a" node. Most of the nodes near the top would remain hot in the cache, and deeper less-visited nodes could be joined or compacted based on some criteria. Of course there are many subtleties, special cases and complications and the solution would be far more complex in terms of source code length.
With a hash table, you effectively compute a hash for every word, and then jump at arbitrary locations throughout the whole table.
Like, maybe your idea works. Maybe. I don't know. You'd have to try it. But it's not clear to me that it will. And it doesn't support "is far from what an experienced C programmer would write if performance was paramount" IMO.
I cannot find any stream constraint in either the article or the countwords repo. He just says "standard input" and his "test.sh" uses "<kjvbible_x10.txt".
You can mmap stdin/fd 0 after an `fstat(0,..)` no problemo. Can it fail and should you check errors? Sure (as can accessing stdin in any way, actually).
He even mentions memory-mapped IO as a further way to go in the article in the C part.
> Memory: don’t read whole file into memory. Buffering it line-by-line is okay, or in chunks with a maximum buffer size of 64KB. That said, it’s okay to keep the whole word-count map in memory (we’re assuming the input is text in a real language, not full of randomized unique words).
A shorter but less precise way of saying that is, "make the program work on streams."
> You can mmap stdin/fd 0 after an `fstat(0,..)` no problemo. Can it fail and should you check errors? Sure (as can accessing stdin in any way, actually).
Of course. And when it fails, what do you do? You defer to something that can handle a stream! Which is exactly the problem in the OP.
Honestly, I think the author himself saying memory-mapped IO is a forward direction but "enough for now!" is pretty conclusive that he didn't think it was a forbidden optimization direction. I think you are over interpreting his early step-by-step style language explaining finite memory as a strict spec, or perhaps he shared an earlier draft of the article with you before he wrote that.
It's fine to fail over to a stream, but often mmap is faster, as I know you know.
> It's fine to fail over to a stream, but often mmap is faster, as I know you know.
But that's exactly the point. You literally cannot use mmap in all cases, either because you just can't or because it's actually slower. And in those cases, you need to fail over to a streaming implementation. And that streaming implementation needs to be fast too. And that's what this benchmark is.
If you submitted a program that only used mmaps, and I sent a stream into that program, it would fail. So then you would need to modify the program to handle streams. And your strategy for dealing with mmaps couldn't be used. So then you'd need to optimize your handling of streams. Which is exactly what the programs in the OP are doing. So in the end, mmaps are a distraction for a benchmark like this, and I suspect that's why the OP didn't bother with them.
>I’m sure there’s further you could go with the C version: investigate memory-mapped I/O, avoid processing byte-at-a-time, use a fancier data structure for counting, etc. But this is quite enough for now!
This subthread started with bluetomcat talking about the optimized C and what experienced C devs might do. That strikes me as fair game to open up discussion of alternative IO anyway even if the author hadn't already (which he clearly did, as quoted). So, that's two reasons it's worth bringing up, and I was never criticizing your program!
If what you are on about is "Who wins what scorecard against what arbitrary constraints" or "My hands were tied, really!" or whatever, then, sorry, but this benchmark is too uncontrolled for great answers even on its own terms. Various stdio-using things would be "out of constraints" if a system was configured with bigger than 64K default buffers anyway which is certainly possible, if unlikely, today. None of the sample programs or the test harness "check" for that. Some of the languages may not be able to ensure it. Using that to leapfrog to "only streams" is a stretch. Is CPU freq scaling controlled? Min of N trials to filter background noise/get repeatability? Even simple mean+-sdev? No, no, and no. And probably four more things.
Beyond all of that, I also don't think that's what this subthread was ever about. bluetomcat's opening was literally the opposite - equivalent to "real devs would do xyz implicitly independent of the arbitrary constraints posed if performance is paramount"..seemingly a follow up on my quote from the author. He can of course chime in if I misread that. Presumably, the author would have had to relax that already problematic 64K constraint when moving on to mmap (which moving on he might have done if the article were fewer languages or if he had just done it that way first in his open coded C).
The positions you seem dug into here seems to me "don't mention mmap to people questioning the posing of the contest even though the author did" or "you cannot default to mmap and fail over like fstat||mmap||do_streams||aiie_noStdInEven". Yes, which is faster varies by OS/situation. So? Maybe "experienced" C/whatever devs know their situation. (You use mmap in ripgrep...). Maybe these are all honest communication errors, but I don't think your positions are very tenable.
As I mentioned in a few places, performance conclusions here are harder than they might look. As for "distractions", one might say that about literally all the optimized variants, including their numbers in the table since the numbers are likely to change more. The article might be stronger to focus on only the simple variants of all the rest, leaving the optimized ones for the github repo and weird "contest rule" debates on the github issues.
Anyway, to add a little more actual information for passersby less dug in to defending some weird position like "", Nim's stdlib has a trie in critbits module. So, that test is "in bounds" and an easy experiment someone might enjoy.
Have a nice day.
It isn't. It was only added in POSIX 2001. Still there without major changes in 2018. [0] (There's actually a lot of similar libraries in POSIX, that don't include re-entrant forms.)
Thankfully, re-entrant versions are supplied as extensions by GNU, which is significantly less horrifying.
[0] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/se...
Also: this comparison honestly just makes me love AWK more and more. So little code needed! So concise, and pretty ok performance as well! AWK really is an underrated little language.
Of course you could write significantly faster implementations in C (and probably in the other languages) by writing data structures from scratch especially for the job (and memory allocation strategies and maybe other stuff), and if you're the author of wc or grep then that is well worth doing because it will be used so many times relative to the amount of time for you to write that code. The comparison of the idiomatic C program vs grep is basically proof of that. But the article seemed to be targetting more typical developers than just want to plug together some common existing components.
the reason C can be so fast is because a SKILLED developer can write very optimized code. The C and C++ examples are not written well by any means.
All this article does is demonstrate that a less complex language is easier to write "good enough" code in.
Is Rust less complex than C?
Count the frequency of words in a book, then get the most frequent words you don't know in Anki, so you can learn the words that will help you understand more of the text.
This is of course taken from the Fluent Forever book.
Example from Малкият принц:
какво 62
са 61
ако 58
когато 56
ме 55
беше 53
а 51
планета 51
ли 49
човек 43
едно 42
бе 42
ден 42
който 40[0] https://github.com/Machiaweliczny/KindleClippingsTranslator/...
- foreach has hidden allocations
- there are better data structures than Dictionary
- GetValueOrDefault has higher cost than a plain if
C# supports non-allocating splitting via the Span / ReadOnlySpan see this article about this very question:
https://www.meziantou.net/split-a-string-into-lines-without-...
You can also use the Win32 APIs to more efficiently hook the console input:
https://stackoverflow.com/questions/33342340/fast-reading-of...
You can change foreach to for or whatever, but after you deal with the major program's bottlenecks.
What built in you'd use?
I would try it with a dictonary, that links to an array list with the counts.
For kjvbible.txt at (unoptimized) 0.365 is 3rd fastest beating C, and 4th faststest beating optimized Go
(would be interesting to see how fast the optimized Lisp would be)
Common Lisp (sbcl.org):
(defmethod performance-count ((path-file string))
(let ((map (make-hash-table :test 'equal)))
(with-open-file (stream path-file :direction :input :if-does-not-exist nil)
(when stream
(loop for line = (read-line stream nil 'end)
until (eq line 'end)
do
(let ((split (split-string #\space (string-downcase line))))
(dolist (word split)
(let ((index (gethash word map)))
(if index
(setf (gethash word map) (incf index))
(setf (gethash word map) 1))))))
(let ((keys (sort (alexandria:hash-table-keys map) (lambda(x y)(> (gethash x map)(gethash y map))))))
(dolist (key keys)
(format t "~A ~A~%" key (gethash key map))))))))
the 4
foo 2
defenestration 1
I suppose you could also recreate all their examples to create your own baseline of runtime performance but that's a lot of work for what seems to be a not-very empirical benchmark (at least to me).
Disclaimer: I did not check runtime complexity of any of the implementations because I didn't really care and skipped straight to the performance results table.
https://github.com/benhoyt/countwords/blob/master/kjvbible.t...
firmament: 10
firmament, 10
genesis 10
version 10
Evaluation took:
2.798 seconds of real time
2.813069 seconds of total run time (2.677126 user, 0.135943 system)
100.54% CPU
8,125,595,437 processor cycles
934,110,048 bytes consedI'm also pretty certain the article is benchmarking against the 10x copy file for the actual benchmarks.
See this example command in the article:
time $PROGRAM <kjvbible_x10.txt >/dev/null
the 64015
and 51313
of 34634
26879
to 13567
that 12784
in 12503
he 10261
shall 9838
unto 8987
for 8810
i 8708
... Evaluation took:
0.365 seconds of real time
0.370322 seconds of total run time (0.338364 user, 0.031958 system)
101.37% CPU
1,060,005,621 processor cycles
106,297,040 bytes consed
((keys (sort (alexandria:hash-table-keys map) (lambda(x y)(> (gethash x map)(gethash y map))))))
Any specifics on why it's not?
I wrote the simple Rust program. I've also been writing Go for a long time. It's tough to say precisely why, but here are some guesses:
* The simple variants, by virtue of being simple, do a lot of extra allocation. Go, because of its GC, might be able to do a bit better here. (I believe the Rust program also does more allocations than the Go program.)
* In Rust, all strings are UTF-8 validated. In Go, they are not. In Go, strings are only conventionally UTF-8.
* A good portion of these programs is spent interacting with a hashmap. Rust's default hashing algorithm is chosen to prevent HashDoS attacks[1] at the expense of slower hashing. I actually don't know whether Go's hashmap does the same. So I'm highlighting it here as a potential difference that perhaps someone else can elaborate on.
[1] - https://doc.rust-lang.org/std/collections/struct.HashMap.htm...
ordered.sort_by(|&(_, cnt1), &(_, cnt2)| cnt2.cmp(&cnt1));
ordered.sort_by(|&(_, cnt1), &(_, cnt2)| cnt1.cmp(&cnt2).reverse());
1) Yes, almost certainly zero-cost. 2) This isn't a hot part of the program.
See also: https://old.reddit.com/r/rust/comments/m5ix0s/performance_co...
From looking at the code[1], it seems like it does. From[2] it looks like it is only for platforms that have AES.
1. https://github.com/golang/go/blob/7bfe32f39c59056c49f5776b10...
Thanks for taking the time to clarify specifics. I appreciate your attention.
I guess in Go strings are just a slice of bytes?
Isn't it possible to do the same with Rust for this benchmark?
But this is not the only additional cost in the Rust program. I outlined a few others.
> Isn't it possible to do the same with Rust for this benchmark?
This benchmark has two programs for each language. The "simple" and "optimized" variant. The "simple" version is supposed to be written in an idiomatic style for that language. Taking extra steps to make tweaks and optimize the code is, I think, against the spirit of the challenge. Obviously, this is a very fuzzy concept, and everyone can make up their own mind on the extent to which this framing is useful. (I think it is, personally, especially when you also allow for a second submission that tries to make the program fast.)
In Go I have used this [1] in the past when I had to validate UTF8 encoding but on hot paths where I'm sure UTF8 is valid (coming from sanitized database data for example) I skipped that part.
That's what every single Rust program in this benchmark does, except for the simple variant.
Except for the printing, this runs at about C speed because it has very little interpreter overhead:
c = Counter(sys.stdin.read().casefold().split())
for word, count in c.most_common():
print(word, count)
counts = Hash.new(0)
STDIN.each_line do |line|
words = line.downcase.split
words.each do |word|
counts[word] += 1
end
end
counts.sort_by { |_k, v| -v }.each do |word, count|
p "#{word} #{count}"
end STDIN.each
.map(&:downcase)
.map(&:split)
.flatten
.tally
.sort_by { _2 }
.reverse
.each { |word, count| puts "#{word} #{count}" }
For accurate diff results, one would need `puts` instead of `p` to get rid of the additional quotes.
crystal build --no-debug --release simple.cr -o simple-cr> To reduce the allocations, we’ll use a map[string]*int instead of map[string]int so we only have to allocate once per unique word
Does the `map[string]int` approach allocate a new integer each time it is incremented?
For starters, ints aren't normally "allocated" (in the heap sense). Go is value oriented, not reference oriented.
I feel like the bigger change in the code was avoiding the allocation of the string for each word in the source text, and instead passing the byte slice directly to the point where it was used as the map key. Since strings are immutable, when you convert a byte slice to a string, it must be copied into a new immutable backing slice so that no one else can touch it. The exception is that the compiler optimizes map accesses where the key is a string type, but the code is converting a byte slice into a string at the point of using it as a map key. The compiler elides that conversion, making it a no-op, and instead just passes the byte slice in, since the hash map can make a copy of the string if it needs to, but it will otherwise avoid the unnecessary allocation and copying that converting this would otherwise require.
I feel like that's where the big difference is. Heap allocating the integers and storing references in the map doesn't feel like it's actually doing anything here, although storing pointer values in maps can be useful in certain scenarios.
1. When filling the map with `word`, can use emplace with std::move of the `word` instead of operator[] because `word` is no longer needed after that, after checking that it's not already in the map
2. Constructing the vector (for sorting) with all pairs from the map ends up copying the whole map's worth of data! Can use iterator of the map instead of the key of the map in the pair. This may also make sort() faster because it may be cheaper to move an iterator than a string during sort()
https://chrispenner.ca/posts/wc
Edit: upon reading the article it seems like the'yre different kinds of word counting. Still interesting and worth reading nevertheless!
$ apt install logtop
$ printf "%s\n" the foo the foo the defenestration the | logtop
7 elements
1 4 the
2 2 foo
3 1 defenestration
I'm not a linux/shell expert, but I really, really thought the shell script was the most performant answer... it was quite legible even to this Windows user, and only had one line of text.
(defmethod performance-count ((path-file string))
(let ((map (make-hash-table :test 'equal)))
(with-open-file (stream path-file :direction :input :if-does-not-exist nil)
(when stream
(loop for line = (read-line stream nil 'end)
until (eq line 'end)
do
(let ((split (split-string #\space (string-downcase line))))
(dolist (word split)
(let ((index (gethash word map)))
(if index
(setf (gethash word map) (incf index))
(setf (gethash word map) 1))))))
(let ((keys (sort (alexandria:hash-table-keys map) (lambda(x y)(> (gethash x map)(gethash y map))))))
(dolist (key keys)
(format t "~A ~A~%" key (gethash key map))))))))
the 4
foo 2
defenestration 1
Evaluation took:
0.000 seconds of real time
0.000294 seconds of total run time (0.000000 user, 0.000294 system)
100.00% CPU
757,077 processor cycles
0 bytes consed
"The foo the foo the defenestration the"
.split(/\s/)
.map(w => w.toLowerCase())
.reduce((m, w) => { m[w] = m[w] ? m[w] + 1 : 1; return m }, {})
edit: Also the C implementation will get in infinite loop with more than 64k words
The constraints do say that it is okay to assume lines are reasonable length. But yes, if you were making GNU wordfreq, that might not be something you want to assume. But at that point, it just depends on what you want your failure mode to be I suppose. greps for example will happily just gobble up as much memory as they can to fit a line into memory. :-)
The comments explain why I think it's slower. The TL;DR is that using a trie requires more memory accesses than a hash table (per byte of input), and those memory accesses slow the whole enterprise down.
But, not all memory accesses are created equal. So perhaps a more clever representation that exploits locality better would do better. "Obvious" techniques for shrinking memory usage made a big difference and brought the performance of the trie program closer to C: https://github.com/benhoyt/countwords/pull/2
#!/usr/bin/env ruby counts = Hash.new(0) STDIN.each_line { |line| line.downcase.split.each { |w| counts[w] += 1 }} counts.sort_by { |k,v| -v }.each { |k,v| puts "#{k} #{v}" }
Also, not knowing Forth is a Bad Thing(tm) if you want to be a Real Programmer(tm). Same goes for Lisp and Prolog. They all have a place in the history and practice of programming, and dismissing them as "obscure languages that no one uses" misses the point.
This article shows exactly why C is nearly twice as fast as C++ and why rust is in between. This is a good article, the other thread was a bad article and bad thread
Hacker News tends to frown on very short, negative posts from new accounts.
> With the lack of context in this article I'm willing to bet rust isn't actually faster than C
into something like
> Without benchmarks, I'm willing to bet rust isn't actually faster than C.
Now that I write that out, it's even shorter! But it's way better. You've made a specific criticism of the article, rather than making a very abstract one.
I'd still say that something this short isn't likely to be upvoted, but it isn't likely to be downvoted like you were. Compare your comment to this one: https://news.ycombinator.com/item?id=26446830
While it does start with some "I like Rust," imho that wasn't even necessary. It's also more aggressive than you were, but even got some replies. Even though it contained the same sentiment.
I'm not sure if people think benchmarks is just the numbers but I wanted to see source so I can why it was faster and what conditions. C++ is basically an assembler and saying you're faster than assembly is either completely untrue or it'd be poorly written assembly which should be examined.
So essentially I wanted actual code with example numbers
Nobody is obliged to learn that many languages well, but truth demands scruples. If you are not interested in truth, what is the point of publishing one of these?
The result is predictable: whichever language the author already knows best wins.
In this one, for example, we get:
"There’s obviously a lot more pushing you could do with C++. However, I suspect it would end up getting more and more low-level and more C-like"
which is just wholly false.
There are reasons why the people who are the most serious about performance use C++: you can get C++ code as fast as the machine can physically do without giving up any abstraction or readability, and without avoiding powerful features. That is the core value proposition of the language: abstraction without penalty. It delivers, and not by accident.
To get top performance does require awareness of what operations are inherently slow for machines, and not writing code to require such operations. But that doesn't mean giving anything up.
Code that looks like C is no faster than C. Hence, to make code faster than C, you must make it look less like C.
Really, we don't expect an article like this to reveal all about how to write the best code in all languages.
Just don't report lazy falsehoods.
And FWIW, I ported the optimized C program to Rust. They look very similar even though Rust is all about zero cost abstractions as well. So the OP's claim really isn't that ridiculous.
Idiomatic Rust, like idiomatic C++, is faster than C. So, your C-in-Rust runs like C, slow, not like good Rust or good C++, fast.
#include <algorithm>
#include <cctype>
#include <iostream>
#include <iterator>
#include <string>
#include <unordered_map>
#include <vector>
int main() {
std::unordered_map<std::string, int> counts;
std::string word; word.reserve(1024);
for (std::istreambuf_iterator<char>
in(std::cin), end; in != end; ++in) {
unsigned char c = *in;
if (std::isspace(c)) {
if (!word.empty()) {
++counts[word];
word.clear();
}
} else word.push_back(std::tolower(c));
}
if (!word.empty()) { // in case no EOL
++counts[word];
}
std::vector<std::pair<const std::string,int> const*>
vec;
vec.reserve(1<<15);
for (auto const& word : counts) {
vec.push_back(&word);
}
std::sort(vec.begin(), vec.end(),
[](auto const* a, auto const* b) {
return a->second > b->second;
});
for (auto const* word : vec) {
std::cout << word->first
<< ' ' << word->second << '\n';
}
} $ hyperfine './optimized-ncmncm-cpp < kjvbible_x10.txt'
Benchmark #1: ./optimized-ncmncm-cpp < kjvbible_x10.txt
Time (mean ± σ): 1.396 s ± 0.023 s [User: 1.381 s, System: 0.012 s]
Range (min … max): 1.357 s … 1.429 s 10 runs
$ hyperfine './optimized-cpp < kjvbible_x10.txt'
Benchmark #1: ./optimized-cpp < kjvbible_x10.txt
Time (mean ± σ): 407.6 ms ± 9.1 ms [User: 400.3 ms, System: 6.3 ms]
Range (min … max): 396.6 ms … 423.6 ms 10 runs
$ hyperfine './optimized-c < kjvbible_x10.txt'
Benchmark #1: ./optimized-c < kjvbible_x10.txt
Time (mean ± σ): 203.6 ms ± 4.7 ms [User: 195.1 ms, System: 8.0 ms]
Range (min … max): 196.2 ms … 211.1 ms 14 runs
$ hyperfine './rust/optimized/target/release/countwords < kjvbible_x10.txt'
Benchmark #1: ./rust/optimized/target/release/countwords < kjvbible_x10.txt
Time (mean ± σ): 314.6 ms ± 6.9 ms [User: 302.0 ms, System: 12.1 ms]
Range (min … max): 307.6 ms … 328.9 ms 10 runs
$ hyperfine './rust/optimized-customhashmap/target/release/countwords < kjvbible_x10.txt'
Benchmark #1: ./rust/optimized-customhashmap/target/release/countwords < kjvbible_x10.txt
Time (mean ± σ): 239.0 ms ± 2.8 ms [User: 231.0 ms, System: 7.5 ms]
Range (min … max): 236.0 ms … 245.0 ms 12 runs
g++ -O3 optimized-ncmncm.cpp -o optimized-ncmncm-cpp
Care to retract any of your statements? Did you even bother to look at the programs before spouting off a bunch of nonsense here?
Surely the correct answer is "use some established third party library that does this exact task well". Probably faster, better battle tested against weird input, localised etc etc than anything you can come up with by yourself.
Though obviously that is then useless if the interview is actually for someone who can actually write something like this for some super secret internal thing that is nothing like the problem solved by the standard solutions available. But is that really the case? I have doubts.
If I was hiring though, I'd be more worried about hiring people who would write standard elements from scratch (mostly because that's more fun) than re-use standard building blocks to get a task done correctly.
e.g. for this specific question I'd be most impressed with someone who could talk about Spacy or some other standard library that can do a lot of things in this area very quickly but also be repurposed to do interesting new things. Do I need someone who can write Spacy from scratch or someone who can use Spacy?
Immediately optimising by ignoring non-ASCII and punctuation seems mor elike a red flag than a positive.
The point of questions like this is to test basic understanding, not the ability to search on SO or similar.
> If I was hiring though, I'd be more worried about hiring people who would write standard elements from scratch (mostly because that's more fun) than re-use standard building blocks to get a task done correctly.
As long as the hire is able to fix/extend the 3rd party library on their own... this is exactly what you figure out with "these types of questions".
There are cases where your task is nice, simple, and well-defined, but word counting is not one of them. It's better to give a more realistic task that allows people to think about the premises.
If it's instead explicitly given that this is a quick litmus test, the choices of the first pass implementation don't really matter, and you'll talk about complexities after, then sure.
I seriously doubt such a library exists for most languages (Javascript might be an exception because they seem to love writing one-line libraries). It's such a simple task!