Transducers.jl: Efficient Transducers for Julia | Hacker News Reader

Transducers.jl: Efficient Transducers for Julia

github.com

23 pointsthetwentyone4 comments

Can someone ELI5 what is unique about transducers? They've always sounded interesting but I'm not willing to invest the time to learn Clojure just to understand them. The example here basically makes it look like piping, which doesn't seem new enough to warrant the new terminology, but the README alludes to the protocol being different from typical iterators and more efficient somehow which sounds interesting.

Say you want transform a collection of data. Doing it using chains of map, filter and friends will create intermediate collections in an eager unpure language.

Transducers are oblivious to the context in which they are being used, and transforms each element in turn. They can be extended at each end and do not create intermediate results. Thay are of course not only useful when tranforming collections, but also in places where data flows in one direction like Async channels.

I wrote this (and the accompanying code) some time ago. Some people liked it as an explainer to transducers: https://srfi.schemers.org/srfi-171/srfi-171.html

But shouldn't not generating intermediate collections be the obvious way to do it? In Rust for example map/filter/etc use normal iterators and don't create intermediate collections. C++ ranges have this property too.

Well, in scheme (my language of choice), if you chain map and filter like so:

    (map square (filter even? lst))

filter will return a new list that map will map over and return yet another list (map and filter doesn't modify its arguments). With transducers you can avoid that.

And transducers can be passed to other functions that can use them in any context. You can create one transducer that filters even numbers and squares the result and re-use it either by transducing over a list, or as a processing part of an async channel.

It is like a streaming data protocol, in that way. Transducers transform data.

Reply on news.ycombinator.com