Comment by KirinDave | Hacker News Reader

Comment by KirinDave | Hacker News Reader

Parent Full thread

> In the past I also did some other work on bringing Clojure Transducers to C++ ... consider it a much less important effort---without the data-structures the other parts of Clojure feel almost like just sugar.

Very few clojure programmers consider Transducers very important even in Clojure. While they're convenient for implementing Clojure's stdlib and they have some nice properties, they're very difficult to work with. It takes a very experienced clojure programmer to use transducers. Heck, I know some famous clojure developers who admit they themselves don't feel confident using them.

But the main challenge for them is that transducers just scream for some basic type validation so that you don't accidentally chain together nonsense. C++ could provide that, and you'd end up with all the same properties. In particular, they offer something like half the monad laws and a more easily composed form of CPS (how your continuation is called is more predictable).

I'd think that might be a really valuable thing in C++, as it's a construction that reproduces stream fusion.

Yeah, I agree!

Actually one of the things I found during my work on transducers is that actually they seem to me more implementable in C++ than in Haskell. For example, you can implement state-less transducers with almost zero overhead, something that in Clojure is definitely not true (so the standard transducers do hide state inside the closure of the reducing functions). It is also easier to do n-ary transducers for zipping and unzipping and everything, as you intuit, is type checked.

Still, I agree that transducers are most useful when you have libraries that make sense of them. For example, you could implement a lot of "Rx" using transducers. Ideally I see a user never writing a transducer, and occasionally passing one to some library that provides "higher order" reactive collections (channels, observables, etc.) that can be transformed with transducers. For example, my motivation to implement them in C++ was to build "observable/reactive lenses" (something like "cursors" in Om) [1] where you could use transducers to specify how you "zoom" into the virtual sequence of state values...

Anyways, they are an interesting coding tool, but they don't really influence the architecture of the application.

[1] atria::funken https://github.com/Ableton/atria

> they seem to me more implementable in C++ than in Haskell

If you don't mind the overhead, it's ContT wrapping some base monad. Or you could make it integral. That is more specific than transducers, I grant. But there's not much they can do that ContT over like Array does not.

I think it's too bad they were invented in Clojure, because they're brutally hard to use right and write in clojure.

I have some interesting work I could share in private on using them to provide stream fusion properties to pipelines of channels. Hit me up on keybase if you'd like to see. I don't want to open source it because then people will expect me to explain them and maintain them and I don't feel like it :)

Sorry, I actually meant "than in Clojure". You are right about the ContT. I'm actually curious about your transducer based stream fusion stuff... I'll hit you up later about it :-)

You might find this work on composable well-typed transducers in Idris interesting: https://deque.blog/2017/08/04/transducers-in-idris-2/

Know what you mean RE transducers. I still haven't come to a point where using the transducer pattern is more convenient than just (partial map f) tbh.

Yet to dare clojure spec but it feels likely that the next mind blowing journey will be therein.

Writing your own transducers is hard and error prone, but I think using and composing the ones given you by clojure.core is actually rather easy (Especially using the 3 arity of `into`).

I don't use so many `->>` anymore thanks to them.

I dunno. Except for one very specific project, I've yet to see a "Wow!" use case for transducers. The only place where they really shine is working with channels, but I think channels are a big design mistake (after extensive work trying to tame them).

Interesting! Why do you think they are a big design mistake? I can understand how they can be misused, but I find myself missing them in every language I used that doesn't have them...

So either I'd rather talk about the COMPUTATION and resource safety rather than the channel, as in Pipes/Conduit/Streaming from haskell.

OR, I'd rather deal with a direct actor model a la Erlang, Pony and Cloud Haskell.

Go's experience is so goddamn miserable it should just put the entire notion of async channels as a higher level programming library to bed. As a low-level primitive, it's okay but not fundamental.

Reply on news.ycombinator.com