LINQ Ruined My Favorite Interview Question
scottchamberlin.tumblr.com
scottchamberlin.tumblr.com
(take 10 (reverse (sort-by (comp first rest) (frequencies (string/split ... #"\+s"))))
Edit: remember to sort! (Forgot my coffee this morning...)
Edit again: and aphyr's solution is even more concise and idiomatic, where `s` is the first paragraph of the blog post:
=> (->> (string/split s #"\s+") frequencies (sort-by val) reverse (take 10))
(["I" 7] ["to" 6] ["the" 5] ["a" 5] ["of" 4] ["candidates" 3] ["is" 3] ["question" 3] ["in" 3] ["their" 2])In practice, if you use ->> and write neater LINQ, the code is very close. Although LINQ only has group and sum/reduce, no dedicated frequencies function.
from collections import Counter
Counter(s1.split(' ')).most_common(10) doStuff = require("doStuff");
var result = doStuff(theString);
I would expect a good candidate to:
1. know that a heap is optimal here (remembering whether Counter uses it is optional)
2. express reluctance to implement it from scratch because surely the stdlib can do it better.
So IMHO using libraries like this ruins the question only in the sense of showing its not a challenge for the candidate. Something so simple should not take a page of code.
Creating a heap is O(n) if I remember correctly, so that may well be the most efficient solution.
Just like math homework back in the day, the teacher didn't do it to check your answer, they did it to check how you arrived at your answer.
Therefore, I am unlikely to take the job.
Just to give another example, our consulting company still gets requests for projects to be deployed against Java 1.4!
d = {}
for word in s1.split(' '):
try:
d[word] += 1
except KeyError:
d[word] = 1
print [(x, d[x]) for x in sorted(d, key=d.get, reverse=True)][:10] d3.entries((s.split(" ").reduce(function(p, v){
v in p ? p[v]++ : p[v] = 1;
return p;}, {})))
.sort(function(a, b){ return a.value > b.value; })
.map(function(d){ return d.key;})
.slice(-10); def top_ten(s):
words = s.split(' ')
word_list = set(words)
return sorted(word_list, key=lambda x: words.count(x))[:10]
But I like the readability of this solution and there's a strong argument to be made for it on that basis, especially if the string is short. If this were a job interview, this would be a totally acceptable solution, though it'd be important to be able to discuss why other solutions might be faster and why you prefer this one anyway.
I thought the whole point of Big O / asymptotic analysis is that you can ignore lower-order terms and constant factors because they are insignificant for any appreciably large input size. And also because the lower order terms and constant factors vary too much depending on the programming language, the compiler or VM, the hardware, etc.
At any rate, I wanted to test this out, so I made a naive benchmark for running these functions. The dict solution was ten times faster (0.0011s vs 0.015s) than the list version with ~1350 words. The dict solution ran in 0.13s at ~162,000 words, while I waited a couple minutes before killing the list version on that input.
def top_ten(s):
words = s.split()
return sorted(set(words), key=words.count, reverse=True)[:10]
You should always compare your results to what is expected. For a problem like this, use a small set of test data that can easily be counted and sorted in your head or on paper.
You forgot reverse=True and your results show the 10 least common words. ;)
This kind of error happens to all of us. That's why we have unit tests and QA teams. If you made this mistake during an interview I wouldn't give it much importance and we would have a good laugh about it.
In Ruby, without imports/requires:
def toptenwords(str)
words = str.split
words.sort_by{|word| words.count(word)}.uniq.reverse.take(10)
end
def toptenwords(str) str.split.sort_by{|word| str.split.count(word)}.uniq.reverse.take(10) end counts = Hash.new { 0 }
IO.read('bible-pg10.txt').split.each { |w| counts[w] += 1; }
counts.keys.sort_by { |w| -counts[w] }.take 10
I'd be interested to see if there's a way to do this in a single expression in Ruby.
You definitely can do a hash-based solution in a single expression in Ruby. Here's a very ugly and kludgy example that you could probably improve on if you wanted to. I don't think it's n^2 because the group_by just counts the occurrences of each word and returns a hash where the count is the key:
str.split.group_by{|w| str.split.count(w)}.sort_by{|k,v| k}.reverse.flatten.uniq.keep_if{|w| w.is_a?(String)}.take(10)
I'm also trying to work out a better way to do this using "chunk" because although hashes are fast to access, they are not fundamentally sortable, and sort_by returns a 2d array just like chunk does anyway.
d[word] = d.get(word,0) + 1
dictionary.get is quite useful.
Also, I'd consider s.split(None), instead of s.split(' '). It will group whitespace, so that any double space or other whitespace is collapsed into one delimiter.
s1 = 'a a a a a a a a a a a a a a a a A A A A A A A A A A A A A a. a. a. a. a. a. a. a. a.'
those two lines give:
[('a', 16), ('A', 13), ('a.', 9), ('', 1)
(HN might collapse the double space in s1)
Like this here Commonest[StringSplit[string], 10] array_count_values(...)
arsort(...)
array_slice(...)
Seriously built in method naming and argument ordering don't follow any consistent convention in PHP, that's always been the most irksome thing to me...
sorted([(i,sum([i==j for j in s.split()])) for i in list(set(s.split()))],key=lambda x:x[1],reverse=True)[:10]
In Common Lisp you'd usually supply a TEST keyword parameter (#'STRING= or #'EQUAL in this case). Does Clojure use Java's type system to infer?
(defn frequencies
"Returns a map from distinct items in coll to the number of times
they appear."
{:added "1.2"
:static true}
[coll]
(persistent!
(reduce (fn [counts x]
(assoc! counts x (inc (get counts x 0))))
(transient {}) coll)))Uh, no. Your solution just uses a bunch of standard library functions (at least, I hope they're not syntactic forms… and why a function as specific as "frequencies" not in some namespace boggles my mind). I could write that in C with an appropriate standard library.
Ironically, expressing this in something like SQL actually speaks to the expressivity of the language because the solution was produced entirely with syntactic forms.
The fact that such functions are at a Clojure programmer's fingertips is exceptionally important. If you strip out the core functions (and macros) from any lisp or Clojure you are left with almost nothing( see http://stackoverflow.com/questions/3482389/how-many-primitiv...).
Imports System.LinqConsider that in Clojure, a common act transforming one datastructure into another. Typically only a few kinds: maps, sets, and some kind of sequence.
What is a highly common generic transformation of a sequence to a map? `frequencies`. Particularly since you'd only be doing this with pretty finite sequences, given the finite nature of the built-in maps. Finite here means countable. What generic, domain-independent thing would you be counting? Often, the items themselves.
What? Pure untyped lambda calculus does things "only with syntactic forms". Don't tell me that your solution wouldn't use some basic operators other than rudimentary syntax. Keep in mind that in Lisp-family languages, many things that are a part of syntax in other languages are actually simple functions - arithmetic operations being a case in point - so you really can't avoid using library functions.
I believe that the major point of Clojure etc. is that they provide concise basic operations that you can use everywhere to compose complex operations better than you can do with, say, C#.
(->> (string/split ... #"\s+")
frequencies
(sort-by val)
reverse
(take 10))Edit: Actually, we can also get rid of the 'reverse' by replacing (sort-by val) with (sort-by (comp - val)). Not going to be shorter in terms of character count, though we win on line count.
(->> (string/split s #"\s+")
frequencies
(sort-by val >)
(take 10))
Much as I love Clojure, it's worth pointing out that Clojure's lazy sequences (include intermediate sequences) get cached, while LINQ evaluates more like Clojure's reducers. (You can even get it to do so in parallel.)
On a side note, I'd like to point out a key difference between this Clojure example and corresponding variants in C#, Python, &c: the Clojure variant has no variables. This isn't just a matter of concision: coming up with descriptive names is hard, and usually means duplicated information, either in the name or the type declaration. Often, those names are re-used over and over again with subtly different meanings, for each stage in a pipeline--forcing the reader to reason carefully about the declaring scope at each use.
Consider, for example:
Swimmer swimmer = new Swimmer("foo");
swimmer.setStyle("butterfly");
swimmer.swim();
return swimmer;
(doto (Swimmer. foo)
(.setStyle "butterfly")
.swim)
var words = s.Split(' ');
var wordCounts = words.GroupBy(x => x).Select(x => new { Name = x.Key, Count = x.Count() }).OrderByDescending(x => x.Count);
var countedWords = wordCounts.Select(x => x.Name).Take(10).ToList();
return ExtractTopTen(countedWords);
var top = (from w in text.Split(' ')
group w by w into g
orderby g.Count() descending
select g.Key).Take(10);
This is the real power of ->, .., ->>, doto, and friends: eliminating names for things. By thinking about the composition of transformations, instead of the intermediate results, you can make an algorithm easier to understand and change.
return s.Split(' ')
.GroupBy(word => word)
.OrderByDescending(wordGroup => wordGroup.Count())
.Select(wordGroup => wordGroup.Key)
.Take(10).ToList();I totally agree with you on this.
>Even in the more compact C# example from this thread, consider: var top = (from w in text.Split(' ') group w by w into g orderby g.Count() descending select g.Key).Take(10);
> This variant refers to the temporary variables w (for "words") and g (for "groups"?) six times. Both authors felt the need to reduce the repetition of variables, but the best they could do was to choose single-character names.
> This is the real power of ->, .., ->>, doto, and friends: eliminating names for things. By thinking about the composition of transformations, instead of the intermediate results, you can make an algorithm easier to understand and change.
You lost me somewhere along the way....are you saying the "from w in text"... snippet is bad, and something more along the lines of "This is the real power of ->, .., ->>" is more appropriate?
I ask because that code seems extremely readable to me. Personally, I don't give a shit if it's 30% more verbose or runs 50% slower, for 99% of code (written in the world), optimum performance doesn't matter. Maintainability does matter though. All of this software being written today has to be either maintained by someone, or replaced by something else. And the top ~2% of programmers like you sure as hell aren't going to be taking maintenance jobs any time soon.
I'm curious what the thoughts of a technically smart person such as yourself are on the subject of what companies will be left with 5 to 10 years down the road when consultants have come through and implemented using the currently most optimum platform/language/algorithms?
And I honestly don't mean for this question to be disrespectful. I'm just coming from a situation where I'm a former developer but on a project where I'm not coding, and I ask for features and the developers say they can't do it, or it will be a performance problem. And I know these guys are far more like you than me intelligence/education wise, but the things I ask for I've done tons of times in the past with 10 to 1000 times the data size, without a problem, on far older hardware.
I'm just curious what kind of a support problem you ultra smart people are leaving behind, or if you ever think about the idea that almost no on else is as smart as you?
1. As simple as humanly possible, so that the algorithm is easy to understand and change. Each component is isolated and can be understood and modified in isolation. Boundaries between component and environment are clearly thought-out.
2. As simply expressed as possible: broken up into distinct, well-organized functions, with descriptive, regular names for functions and variables, in context.
3. Well-documented; each function and each namespace come with contextual docs explaining their motivation, arguments, consequences, invariants, etc., with examples.
4. As short as possible, because humans have trouble holding large amounts of context in their head. Minimize the amount of scrolling or jumping between files necessary to understand the algorithm.
5. Well-tested, so that changes can be made freely. The test suite needs to be fast, so one can get feedback within seconds of making a change to the file; ideally a few milliseconds. A balance of typechecking, logical tests with mocks, integration tests, and full stress tests provides a continuum of safety.
I don't see these goals as particularly constrained to any language, but I will say that I feel best able to achieve them in a Lisp. Dunno whether that helps you project your problems at work onto me personally, though. ;-)
return new Swimmer("foo")
{ Style = "butterfly"}
.swim();
var result = new Swimmer("foo")
{ Style = "butterfly"};
result.swim();
return result;I speak as someone who codes in C# for work and Clojure as a hobby. There's lots of things that just aren't quite possible in C#, and _everything_ is possible in Clojure.
Pointfree is beautiful to read and grok. It's a little trickier to debug, since pretty much every debugger on earth needs points to inspect values.
def top10(s: String) = s.split(' ').groupBy(identity).mapValues(_.size).toList.sortBy(-_._2).take(10).map(_._1)
def mostCommon(str: String, num: Int) = { str.split(" ").groupBy { s => s} .map { case (k,v) => k -> v.length }.toList .sortBy { _._2 }.reverse.take(num).map { _._1 } }
var top = (from w in text.Split(' ')
group w by w into g
orderby g.Count() descending
select g.Key).Take(10); from word in text.Split(' ')
group word by word into g
let count = g.Count()
orderby count descending
select g.Key
tr 'a-z' 'A-Z' | sed 's/[^A-Z][^A-Z]*/\ /g' | grep -v '^$' | sort | uniq -c | sort -nrk1 | head -10
edit: I don't know enough about HN,there should be a newline after the backslash in the sed command.
echo $sentence | rs -T | sort | uniq -c | sort -rn|head -10
$ echo Foo foo foo. | rs -T
Foo
foo
foo.
$ echo Foo foo foo. | tr 'a-z' 'A-Z' | sed 's/[^A-Z]/\
/g' | grep -v '^$'
FOO
FOO
FOO
$ cat foo
Foo
foo
foo.
$ <foo rs -T
Foo foo foo.
$ <foo tr 'a-z' 'A-Z' | sed 's/[^A-Z]/\
/g' | grep -v '^$'
FOO
FOO
FOO
What's interesting though is how UNIX shell, being essentially a symbolic FP language, allows one to solve the problem in a clear and concise way. And if one desires, the program can be easily modified to read sentences, say, from network from ssh tunneled via HTTPS. That kind of flexibility can rarely be achieved in languages like C# with tight coupling between the units of abstraction.
$ < bible-pg10.txt tr -cs a-zA-Z '\n' | sort | uniq -c | sort -nr | head
62265 the
38915 and
34588 of
13474 to
12846 And
12589 that
12387 in
9762 shall
9668 he
8942 unto
Unix for Poets, man. It's the shit.
s.Split(' ').GroupBy(x => x).OrderByDescending(x => x.Count()).Select(x => x.Key).Take(10).ToList();
There's a lot of benefits to Clojure over C#, but the ability to chain functional list operators really isn't one of them.
I almost feel a bit gross when I have to write a "foreach" loop at this point, because there's almost always an equivalent way to do it in LINQ (although it's a tradeoff, as the one downside of LINQ is that given it's deferred nature, it's harder to debug).
I think this is mostly because I'm still not comfortable with the syntax, and partly because of the set/iterative impedance mismatch that is there. "It's not you, LINQ. It's me."
Agreed. I've transitioned to spending most of my time in JS, and wherever I can I use .map(), but the chaining it's not quite the same as LINQ. Someday I intend to write a library of Array addons to provide GroupBy and so on, but I can't imagine it'll be super efficient.
_([ ... ]).map( ... ).filter( ... ).value()
I wish they would at least alias map, reduce, some, etc.
It makes sense to use terms that many C# (and SQL) users are already familiar with, rather than terms that they may not know of.
Actually........
It's more generic than that. It's C#'s monadic comprehension syntax ala Scala's for and Haskell's Do.
It works on more than just IEnumerable, it actually works on anything that implements Select, SelectMany, and Where...
It makes a 'whole class of things that you would have to do with loops' go away. Once you are comfortable with the syntax, it makes code a lot more readable.
But you lose track of when and where things are getting executed.
For example, it is easy to make something that you intend to execute inside of SQL server run inside of C# code. And then, all of a sudden, string comparison is case-sensitive.
You wind up having to context switch between procedural and set mentality without the same kind of visual cues you used to get.
You do have to be careful for hidden pitfalls, like an accidental N+1 select against the database, but even that is usually fixed pretty easily if you simple call .Include() in the original expression.
From the article, I really don't understand the concept of not updating your knowledge of Linq. It's been out for years now. And then the first concern is with performance? That sounds very problematic to me. There are a lot of .NET developers out there that are still stuck in the previous decade. There are a lot of shops that are still on 2.5, which is too bad, because that was when the framework and C# really started to take off.
Four killer issues I've seen so far:
We had a major production performance issue which turned out to be a stray ToList which was causing a massive memory ballooning. Didn't get noticed in test as the test cases passed but it hit prod and 2000 users bashed it and tried to allocate 20Mb each causing our cluster to shit a brick.
Null reference exceptions! There are so many dereferencing operations in an average LINQ expression that you really have no idea which one is blowing if it goes pop in production in release config.
People using .Single(...) and getting more or less than one result back. So frustrating.
If you push an IEnumerable<T> over an interface boundary the performance and memory semantics are not preserved and you end up with a leaky abstraction. These are shits to resolve. Example: queries executing inside the view which is outside the transaction scope.
We've had to ban it in some circumstances.
We try to use ICollection in our APIs instead of IEnumerable, since the latter can have surprising semantics like being a wrapper for some operation which may not be valid anymore, or might be slower than you expect to do things like .Count(). IMO it's really not best for transporting across interface boundries in the most common case; only when you're specifically trying to avoid having the whole collection in memory or something like that.
Another thing that can help is this wonderful May<T> library[1]. It a great option type[2] for .NET. It helps make operations more composable.
That is the curse of modern software development. But LINQ is minor offender here compared to some kinds of remoting, code reflects from endless xml configs wrapped in gazillion interfaces and some inventive orm-s. Sadly our tools have not kept up with the complexity that is external to the executing code. The debugger in the pre- inversion of control/bean injection days was almighty because you had all of the program state in front of you.
A little known fact is that if you wrap Java/C# code in enough unit tests you get Haskell.
It is enough to make you want to switch to a pure message passing system, isn't it?
If you had to tell the compile if you wanted code or a tree, that problem would be solved. It'd also be one step closer to allowing type inference for lambdas assigned to locals.
I'm tired of seeing people answer interview questions with anything _other_ than LINQ.
I'd like to aee people using the right tools for the job.
That said, not all questions are best solved with it, but it certainly has made working with enumerables much easier.
That a C# developer can seriously say they're not up to speed with "new stuff" like LINQ (2005) and dynamic (2008) in 2013 is kind of ridiculous. What they mean is that they figured out how to map what they learned about Java in school into C#'s syntax and now they're done learning.
You don't sound arrogant, but rather sound naive. I agree that someone who recruits surely should have known about and have experienced LINQ significantly by now, but the notion that you should be using it "all the time" is absolute nonsense.
I avoid LINQ. I encourage others to avoid LINQ. It is almost always a sign of bad code.
LINQ is syntactical sugar over basic set operations. It is perfectly fine if you're doing naive activities, such as the example give -- a contrived example of brute forcing a problem, where two approaches of the same very basic need unsurprisingly yield the same complexity -- but it falls apart in real-world persistent code with considered algorithms and storage. In real long term code, it is usually the canary in the mineshaft telling you that the developers aren't using proper algorithms or storage.
In many large-scale projects it invariably turns into the performance nightmare that ends up causing whole rewrites.
Again, not because of LINQ itself, which of course can do basic operations like grouping and sorting as quickly as you could do "by hand", but that it makes it so easy to do those things that developers start to resort to that as a catch-all magical solution that is costless because how much could a line or two of code cost? -- a master List<stuff> that they just sort and group by and select from all over the code (O(n) * O(n) * O(n)...why not?). The end result is that the data structures and encapsulation that should have happened never did, so while it might seem more concise and obvious on a one-to-one comparison with a loop perspective, neither case should ever have happened.
LINQ is the gun by which a lot of terrible programmers are repeatedly shooting themselves in the foot with, all while gloating about their concise code.
The problem is that it makes it so conveniently easy to do brute-force tactics that....oh the horrible things I've seen...code gets littered with LINQ doing naive queries repeatedly over massive sets of data. Of course you need good coders and good code audits, but LINQ, I think, gives a unsupported sense of comfort that one is making good code (where if people had to code these as loops, it would become very evident very early on that maybe they should rethink their approach).
Correct me if I'm wrong, but the world is moving towards functional programming (i.e. LINQ) not away from it. Personally, I find LINQ far, far easier to read, write, and analyze. (On the other hand, I understand the deferred semantics and watch for warning signs like enumerating a sequence more than once.)
Honestly, a C# company avoiding LINQ sounds to me like the canary in the mineshaft telling you the company has programmers falling behind the times and doing things the hard way.
LINQ encourages the belief that set operations are free, such that you no longer have to concern yourself with concepts like memoization or appropriate structures. This has nothing to do with functional programming. Literally at all.
Basically, you're saying it's too easy to accidentally perform a computation twice by iterating through a sequence twice. Recurse on that problem and you get an exponential blowup.
That is actually a problem mostly unique to LINQ and it is really important that a programmer understand the deferred semantics. Some tools, e.g. ReSharper, will detect multiple enumerations and warn you about it. Judicious use of ToArray/ToList solves most issues with defer-splosion.
> "it falls apart in real-world persistent code with considered algorithms and storage"
> "it makes it so easy to do [grouping / sorting / etc] that developers start to resort to that as a catch-all magical solution that is costless because how much could a line or two of code cost"
With hindsight I know your issue is with LINQ's deferred non-memoized execution. It's really easy to enumerate a sequence twice, but every place you do that might add a factor of 2 to the runtime and so there's a danger of exponential blowup that must be avoided. But that's not what I see when I read the above.
The above reads as complaints about easy composability and worries about code monkeys failing to consider what the code they're writing will do. Those are standard "functional / high-level programming is bad" complaints. With hindsight I know that's not what you meant, but that's what it reads like.
LINQ's deferred execution was the right choice for performance, but it's the hardest one. You've got to know when to call .ToList(). I'm not denying that I've seen people evaluate the same expensive list 100 times, use a join when precomputing a Dictionary would have been much faster, close a connection before the result is actually evaluated. But I've never seen C++ programmers say you can make mistakes with pointers, so don't them.
Clojure's lazy sequences are guaraanteed to evaluate once, but that comes at the expense of storage. In particular, a query like the one in the original blog post will evaluate multiple intermediate lists that then need to be thrown away. And indeed, they've introduced reducers to address this, which behaves more like LINQ.
Why not fuse the operations, if the values are immutable?
It wouldn't be hard to make LINQ behave like Clojure, either.
Oh give me a break. My exact complaint about LINQ is the opposite -- that it makes expensive operations seem simple, which with all innovations in language means that mediocre programmers will start abusing it in all codebases.
LINQ has a place. To say that one should be using it "all the time", however, is exactly as I said -- a canary in a coal mine. The only time I've ever come across the extensive use of LINQ it has been in horrific code.
In fact, LINQ on Rx is a symptom of the classic LINQ issue -- don't have subscriber or source filtering, or coherent, useful events -- simply broadcast everything and through the magic, free filtering of LINQ all is solved in the consumer.
The HR interviewer followed up my answer asking about runtime and memory usage. While these are good questions, I got the feeling they didn't want to receive an answer using a single line of LINQ.
are candidates allowed to chose their favorite language?
man bash | tr '[:upper:] ' '[:lower:]\n' | sed '/^$/d' | sort | uniq -c | sort -rn | head | awk '{ print $2 }' | fmt
or do you only hire windows coders?
man bash | tr '[:upper:] ' '[:lower:]\n' | awk '/./ { bag[$1]++ } END { for(word in bag) { print bag[word], word } }' | sort -rn | awk '{ print } NR>=10 { exit(0) }'
But it would require more typing and thinking.
sure one could do this in awk completely,
man bash | awk '
/./ {
for (i = 1; i<=NF; i++) {
bag[tolower($i)]++
}
}
END {
for (i = 1; i<=10; i++) {
score=0;
for(word in bag) {
if (bag[word] > score) {
score=bag[word];
best=word
}
}
printf "%s ", best
delete bag[best]
}
printf "\n"
}'
Edit: and while we're on the subject, you can get rid of head and use awk 'NR<=10 {print $2}'
Edit²: I would also like to point out that your version is superior to any short C# program, because sort can sort sequences that exceed the amount of RAM you have.
If you like LINQ and wish it were available in JS, look at underscore or lo-dash.
performance of Linq 2 Objects is not that great either and it doesn't add much value readability-wise over rewriting the same task in other dynamic languages: https://github.com/dartist/sudoku_solver
>>> from collections import Counter
>>> Counter("here are some words here are".split()).most_common(3)
[('are', 2), ('here', 2), ('words', 1)]As someone who doesn't do C# or LINQ, that second solution seems to me like someone really wanted to have as few lines of code as possible.
I don't claim to have an impressive programming pedigree, but while I take simplicity and performance into account, I never take "conciseness" into account. Conciseness usually means "this is opaque as shit but at least it's short". And who really cares about short? What's the purpose of "short"? None that I can find, other than impressing interviewers. Anyone who believes otherwise should probably be writing in Clojure or Haskell (and probably is), but I personally just don't see the point.
But that's just my opinion. My favorite language is Python.
1. Iterate through all key,value pairs once, keeping track of the 10 most common 2. Run quickselect 10 times 3. Coolest (and an interview question in it's own right) - modify quickselect to return the top 10!
...
> var words = s1.Split(' ');
Wrong. Yet another example where an interviewer cannot correctly solve his own questions.
You named more than I detected.
The C# isn't even that succinct compared to doing a similar thing in other popular languages. For example, in Haskell:
topTenWords :: String -> [String]
topTenWords = take 10 . map fst . sortBy (flip (comparing snd)) . map (\l -> (head l, length l)) . group . sort . wordsFor \l -> (head l, length l) I tend to use head * * * length (without the spaces between those stars).
private static int CompareKVPByCount(KeyValuePair<string, int> a, KeyValuePair<string, int> b)Or even:
kvpList.Sort(kvp => kvp.Value)
import qualified Data.Map as M
import Data.List
import Data.Ord
countWords :: String -> [String]
countWords = map fst . take 10
. sortBy (comparing snd)
. M.toList . M.fromListWith (+) . map (\w -> (w, 1))
. words foreach (var item in list)
if (SomeCondition(item)) return item;
return null;
list.FirstOrDefault(item);But it's funny, the first time I read the paragraph, I'd assume the words were not separated by space, and you had to find occurrences of combinations than.
I instinctively made the test much harder than it would be. I'm damaged.
count = take 10 . map head . reverse . sortBy (comparing length) . group . sort . words
input_string.split(/\W+/).inject(Hash.new(0)) {|acc, w| acc[w] += 1; acc}.sort {|a,b| b.last <=> a.last }[0,10]Reverse[SortBy[Tally[StringSplit[#]], #[[2]] &]][[;; 10, 1]] &
.say for (bag($text.words) ==> sort {-*.value})[^10] def topx(str,x)
c = Hash.new(0)
str.split(/\s+/).each { |s| c[s] += 1 }
c.sort_by {|k,v| -v}.take(x)
end- deferred ejecution by default, saving memory and time
- step by step syntax, each new operation is at the end, not the beginning
- excellent type inference and intellisense, js? ruby?...
- it works with the same syntax on the database!!! Haskell?
- map and filter where there, but groupby and join where not so common in previous query comprehensions APIs.
- the most important: it's actually usable in jobs you get paid for, not experiments you can make at home or university.
There are however two things that doesnt make it 100% perfect:
- expression tree lambas are identical to non expression ones, making it hard for developers to know if one step is going to be translated or executed. I would have chosen => for non expression and -> for expressions for example or something like that.
- having two syntax, method chain and query comprehensions, produces a frequent anoying back and forth since some operators are better written in one (let, join, group by) while others are only available in method chain (take, toDictionary...)
(take 10 (reverse (sort-by (comp first rest) (frequencies (string/split ... #"\+s")))) ; llambda Clojure
// haakon Scala
s.split(' ').groupBy(identity).mapValues(_.size).toList.sortBy(-_._2).take(10).map(_._1)
(->> (string/split s #"\s+") frequencies (sort-by val) reverse (take 10)) ; aphyr Clojure
var top = (from w in text.Split(' ') // louthy C# LINQ
group w by w into g
orderby g.Count() descending
select g.Key).Take(10);
collections.Counter(s1.split()).most_common(10) # shill Python
d = {} # shill Python without collections library
for word in s1.split(): d[word] = d.get(word, 0) + 1
print [(x, d[x]) for x in sorted(d, key=d.get, reverse=True)][:10]
words = s.split() # spenuke and abecedarius probably O(N²) Python
sorted(set(words), key=words.count, reverse=True)[:10]
d3.entries((s.split(" ").reduce(function(p, v){ // 1wheel JS with d3
v in p ? p[v]++ : p[v] = 1;
return p;}, {})))
.sort(function(a, b){ return a.value > b.value; })
.map(function(d){ return d.key;})
.slice(-10)
# kenuke O(N²) Ruby:
str.split.sort_by{|word| str.split.count(word)}.uniq.reverse.take(10)
counts = Hash.new { 0 } # my Ruby
str.split.each { |w| counts[w] += 1; }
counts.keys.sort_by { |w| -counts[w] }.take 10
# aaronbrethorst ruby
str.split(/\W+/).inject(Hash.new(0)) {|acc, w| acc[w] += 1; acc}.sort {|a,b| b.last <=> a.last }[0,10]
Commonest[StringSplit[string], 10] # carlob Mathematica
Reverse[SortBy[Tally[StringSplit[#]], #[[2]] &]][[;; 10, 1]] & # superfx old Mathematica
$a = array_count_values(preg_split('/\b\s+/', $s)); arsort($a); array_slice($a, 0, 10) // Myrth PHP
tr -cs a-zA-Z '\n' | sort | uniq -c | sort -nr | head # mzs and me sh
-- lelf in Haskell
take 10 . map head . reverse . sortBy (comparing length) . group . sort . words
# prakashk Perl6
.say for (bag($text.words) ==> sort {-*.value})[^10]
# navinp1912 C++
string s,f;
map<string,int> M;
set<pair<int,string> > S;
while(cin >> s) {
M[s]++;
int x=M[s];
if(x>1) S.erase(make_pair(x-1,s));
S.insert(make_pair(x,s));
}
set<pair<int,string> >::reverse_iterator it=S.rbegin();
int topK=10;
while(topK-- && (it!=S.rend())) {
cout << it->second<<" "<<it->first<<endl;
it++;
}
Although I haven't tested it, I think the Afterquery program to solve this, assuming you first had something to tokenize your text into one word per row, would be something like
&group=word;count(*)
&order=-count(*)
&limit=10
Perl 5, perhaps surprisingly, is not simpler:
perl -wle 'local $/; $_ = <>; $, = " "; $w{$_}++ for split; print @{[sort {$w{$b} <=> $w{$a}} keys %w]}[0..9]'
perl -wlne '$w{$_}++ for split; END { $, = " "; print @{[sort {$w{$b} <=> $w{$a}} keys %w]}[0..9]}'
string s,f;
map<string,int> M;
set<pair<int,string>> S;
while(cin >> s) {
M[s]++;
int x=M[s];
if(x>1) S.erase(make_pair(x-1,s));
S.insert(make_pair(x,s));
}
auto it=S.rbegin();
int topK=10;
while(topK-- && (it!=S.rend())) {
cout << it->second<<" "<<it->first<<endl;
it++;
}
Here's one.
perl -0777 -nE '$w{$_}++ for split; say for (sort {$w{$b} <=> $w{$a}} keys %w)[0..9]'
perl -0777 -nE '$w{$_}++ for split; $, = $"; say((sort {$w{$b} <=> $w{$a}} keys %w)[0..9])'(sorry)
string s,f;
map<string,int> M;
set<pair<int,string> > S;
while(cin >> s) {
M[s]++;
int x=M[s];
if(x>1) S.erase(make_pair(x-1,s));
S.insert(make_pair(x,s));
}
set<pair<int,string> >::reverse_iterator it=S.rbegin();
int topK=10;
while(topK-- && (it!=S.rend())) {
cout << it->second<<" "<<it->first<<endl;
it++;
} while (cin >> s) M[s]++;
for (map<string,int>::iterator i = M.begin(); i != M.end(); i++) {
S.insert(make_pair(i->second, i->first));
}