K Isn't Lisp (2004)
nsl.com
nsl.com
I kinda hate it how all the K advocacy always uses old-style languages with small stdlibs, _and_ always ignores the complexity.. How about matlab (which does have "unique" in stdlib), python/numpy or even R?
library(data.table)
v <- rbindlist(list(
data.table(e1=1, string='one'),
data.table(e1=1, string='two'),
data.table(e1=1, string='three'),
data.table(e1=1, string='one'),
data.table(e1=1, string='four'),
data.table(e1=1, string='two'))
)
v[, list(counter= sum(e1)),by= string]For K, I think the solution is to find unique first (nlog(n)), the for each unique value go over the list and find how many input values match (multiply by n). That's O(n^2 * log(n)), even worse than I wrote initially.
Your analysis of the algorithm you've described is also wrong: it would be O(n*log(n) + n^2), or O(n^2), because the second part loops over unique elements and not operations performed by unique.
In the random-access model, hash table construction/lookup is O(n) average case for n elements, if the elements have bounded size. More generally, it's linear in the total size of the input. If the strings are very long it can take a while to hash and compare them, but there's never a log(n) factor. Maybe you're confusing this with results from comparison sorting?
Both solutions are O(n) with the assumptions of random-access memory and bounded string size (or hash/comparison time), which I consider to be pretty standard.
(frequencies (map second v))
;; => {"one" 2, "two" 2, "three" 1, "four" 1}
;; or:
(update-vals (group-by second v) count)
;; => {"one" 2, "two" 2, "three" 1, "four" 1}
;; +(#:'=v[;1];?v[;1])
;; flip(count each group v[;1];unique v[;1])
So, shorter than the readable K version in a Lisp.What did we learn here? Probably nothing.
K is great for sequence operations--not sure what we're trying to imply about Lisps.
Here's the result (if you wanted to further optimize and know that the result will fit in a fixnum (i.e. +/- 2^62 on modern implementations, you can change the "'number" to "'fixnum"):
(defun unique-sum (list)
(multiple-value-bind (keys values)
(scan-hash
(collect-hash-sum
(map-fn 'string #'cadr (scan list))
(map-fn 'number #'car (scan list))
:test #'equal))
(map-fn 'list #'list values keys)))
And the extension to SERIES: (series::defS collect-hash-sum (keys values &rest option-plist)
"(collect-hash keys values :test :size :rehash-size :rehash-threshold)
Combines a series of keys and a series of values together into a hash
table. Duplicate keys sum the values. The keyword arguments specify
the attributes of the hash table to be produced. They are used as
arguments to MAKE-HASH-TABLE"
(fragl ((keys t) (values t) (option-plist))
((table))
((table t (apply #'make-hash-table option-plist)))
()
()
((incf (gethash keys table 0) values)) () () nil)
:optimizer
(series::apply-literal-frag
(list '(((keys t) (values t) (table)) ((table)) () ()
() ((incf (gethash keys table 0) values)) () () nil)
keys values `(make-hash-table ,@ option-plist)))
:trigger t) (let ((test-input '((1 "one") (1 "two") (1 "three") (1 "one") (1 "four") (1 "two"))))
(mapcar (lambda (string)
(list (count string test-input :test #'string= :key #'second)
string))
(remove-duplicates (mapcar #'second test-input)
:test #'string= :from-end t)))
I will admit it's not as short as: v:((1;"one");(1;"two");(1;"three");(1;"one");(1;"four");(1;"two"))
+(#:'=v[;1];?v[;1]) (let ((in '((1 "one") (1 "two") (1 "three") (1 "one") (1 "four") (1 "two"))))
(loop with hash = (make-hash-table :test #'equal)
for v in in
do (incf (gethash (cadr v) hash 0))
finally (return (loop for key being the hash-keys of hash
collect (list (gethash key hash) key))))) (defun doit (elts)
(mapcar (lambda (elt) `(,(count elt elts :test 'equal :key 'second) ,elt))
(delete-duplicates (mapcar 'second elts) :test 'equal)))))
or maybe (defun doit (elts)
(loop for elt in (delete-duplicates (mapcar 'second elts) :test 'equal)
collect `(,(count elt elts :test 'equal :key 'second) ,elt))) echo '1 "one"\n1 "two"\n1 "three"\n1 "one"\n1 "four"\n1 "two"' | awk '{ counters[$2] += $1 } END { for (id in counters) { print counters[id], id } }'
2 "two"
1 "four"
1 "three"
2 "one"But no, it doesn't. It completely ignores the digit, just groups by the string and counts that.
The problem is very weird and stupid though. The input is clearly logically a list of dictionaries with a string as a key and a count as the value. If you have that list (let's call is myList), in K you just need to sum the dictionaries together and it will automatically adds all the counts together. So the code would literally just be "+/myList".
I would just convert the list of lists to a list of dictionaries and do that.
Even my contribution (using the SERIES library) keeps the hash-table explicit.
I don't know how much of this is habit, how much is lisp programmers being control freaks, and how much of it is "If I need to do grouping, having the hash-table explicitly available will probably be needed at some point anyways"
1: Static grouping (i.e. where all group keys are known in advance) is trivial.
But no, it doesn't. It completely ignores the digit, just groups by the string and counts that.
The problem is very weird and stupid though. The input is clearly logically a list of dictionaries with a string as a key and a count as the value. This is K, not Bash, you should be using a list of dictionaries, not a string. If you have that list (let's call is myList), in K you just need to sum the dictionaries together and it will automatically adds all the counts together. So the code would literally just be "+/myList".
> BC You had used APL, and then you explored these other languages—Prolog variants and so on—but when you got to Morgan Stanley you came back to APL. What brought you back?
> AW I much preferred implementing and coding in LISP, but once I was dealing with big data sets and then having to do fairly simple calculations, APL just seemed to have the better vocabulary.
> It had to come up one level. Common LISP even then had about 2,000 primitives. I didn't like that. What I liked was the original LISP, which had car, cdr, cons, and cond, but that was too little. Common LISP was way too big, but a stripped-down version of APL was in the middle with about 50 operations. It's about the same size as C. But the thing about the languages that I implement is that there are no libraries: those 50 operations are it. Everybody builds from there, and the resulting programs are extremely short.