Can someone explain Clojure Transducers to me in Simple terms?

I have tried reading up on this but I still don't understand the value of them or what they replace. And do they make my code shorter, more understandable or what?

Update

Alot of people posted answers, but it would be nice to see examples of with and without transducers for something very simple, which even an idiot like me can understand. Unless of course transducers need a certain high level of understanding, in which case I will never understand them :(

Transducers are recipes of what to do with a sequence of data without knowledge of what the underlying sequence is (how to do it). It can be any seq, async channel or maybe observable.

They are composable and polymorphic.

The benefit is, you don't have to implement all standard combinators every time a new data source is added. Again and again. As a result, you as user are able to reuse those recipes on different data sources.

Prior to version 1.7 of Clojure you had three ways to write dataflow queries:

1. nested calls

   (reduce + (filter odd? (map #(+ 2 %) (range 0 10))))
   

2. functional composition

   (def xform
     (comp
       (partial filter odd?)
       (partial map #(+ 2 %))))
   (reduce + (xform (range 0 10)))
   

3. threading macro

   (defn xform [xs]
     (->> xs
          (map #(+ 2 %))
          (filter odd?)))
   (reduce + (xform (range 0 10)))
   

With transducers you will write it like:

(def xform
  (comp
    (map #(+ 2 %))
    (filter odd?)))
(transduce xform + (range 0 10))

They all do the same. The difference is that you never call transducers directly, you pass them to another function. Transducers know what to do, the function that gets a transducer knows how. The order of combinators is like you write it with threading macro (natural order). Now you can reuse xform with channel:

(chan 1 xform)

Transducers improve efficiency, and allow you to write efficient code in a more modular way.

This is a decent run through.

Compared to composing calls to the old map, filter, reduce etc. you get better performance because you don't need to build intermediate collections between each step, and repeatedly walk those collections.

Compared to reducers, or manually composing all your operations into a single expression, you get easier to use abstractions, better modularity and reuse of processing functions.

Say you want to use a series of functions to transform a stream of data. The Unix shell lets you do this kind of thing with the pipe operator, e.g.

cat /etc/passwd | tr '[:lower:]' '[:upper:]' | cut -d: -f1| grep R| wc -l

(The above command counts the number of users with the letter r in either upper- or lowercase in their username). This is implemented as a set of processes, each of which reads from the previous processes's output, so there are four intermediate streams. You could imagine a different implementation that composes the five commands into a single aggregate command, which would read from its input and write its output exactly once. If intermediate streams were expensive, and composition were cheap, that might be a good trade-off.

The same kind of thing holds for Clojure. There are multiple ways to express a pipeline of transformations, but depending on how you do it, you can end up with intermediate streams passing from one function to the next. If you have a lot of data, it's faster to compose those functions into a single function. Transducers make it easy to do that. An earlier Clojure innovation, reducers, let you do that too, but with some restrictions. Transducers remove some of those restrictions.

So to answer your question, transducers won't necessarily make your code shorter or more understandable, but your code probably won't be longer or less understandable either, and if you're working with a lot of data, transducers can make your code faster.

This is a pretty good overview of transducers.