## Friday, July 29, 2011

### Map Reduce Example in F#

A while back I blogged about the canonical map reduce example (as seen in the Hadoop user manual) of counting words. Today I noticed that an alias for F#'s List.fold function is List.reduce*. I had already seen List.map, so I put two and two together, and pretty soon I had a tiny program up and running in F# Interactive. Enjoy:
/// maps an input into a tuple of input and countlet mapper x =    (x, 1)/// reduces a list of input and count tuples, summing the countslet reducer (a:(string * int) list) (x:string * int) =    if List.length a > 0 && fst (List.head a) = fst x then        (fst (List.head a), snd (List.head a) + snd x)::List.tail a    else        x::a            /// maps the mapper function over a list of inputlet map xs =    List.map (mapper) xs/// maps the reducer function over a list of input and count tupleslet reduce (xs:(string * int) list) =     List.fold (reducer) [] xs/// maps the input, sorts the intermediate data, and reduces the resultslet mapReduce xs =     map xs    |> List.sort    |> reduce

It turns out the functional programming appears to be made for this pattern!