Empty RDD It cannot be substituted when RDD is empty: val rdd = sc.emptyRDD[Int] rdd.reduce(_ + _) // java.lang.UnsupportedOperationException: empty collection at // org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$ … rdd.fold(0)(_ + _) // Int = 0 You can of course combine reduce with condition on isEmpty but it is rather ugly. Mutable buffer Another use case for fold is … Read more
Here’s a fully parametric example: data Weird a = Weird a (a -> a) instance Foldable Weird where foldMap f (Weird a b) = f $ b a Weird is not a Functor because a occurs in a negative position.
zip2 xs ys = foldr step done xs ys where done ys = [] step x zipsfn [] = [] step x zipsfn (y:ys) = (x, y) : (zipsfn ys) How this works: (foldr step done xs) returns a function that consumes ys; so we go down the xs list building up a nested composition … Read more
The easiest way to understand foldr is to rewrite the list you’re folding over without the sugar. [1,2,3,4,5] => 1:(2:(3:(4:(5:[])))) now what foldr f x does is that it replaces each : with f in infix form and [] with x and evaluates the result. For example: sum [1,2,3] = foldr (+) 0 [1,2,3] [1,2,3] … Read more
Short answer: foldRight associates to the right. I.e. elements will be accumulated in right-to-left order: List(a,b,c).foldRight(z)(f) = f(a, f(b, f(c, z))) foldLeft associates to the left. I.e. an accumulator will be initialized and elements will be added to the accumulator in left-to-right order: List(a,b,c).foldLeft(z)(f) = f(f(f(z, a), b), c) fold is associative in that the … Read more
We begin with the type of foldMap: foldMap :: (Foldable t, Monoid m) => (a -> m) -> t a -> m foldMap works by mapping the a -> m function over the data structure and then running through it smashing the elements into a single accumulated value with mappend. Next, we note that, given … Read more
In general, all 6 fold functions apply a binary operator to each element of a collection. The result of each step is passed on to the next step (as input to one of the binary operator’s two arguments). This way we can cumulate a result. reduceLeft and reduceRight cumulate a single result. foldLeft and foldRight … Read more
You can’t really use map to sum up a list, because map treats each list element independently from the others. You can use map for example to increment each value in a list like in map (+1) [1,2,3,4] — gives [2,3,4,5] Another way to implement your addm would be to use foldl: addm’ = foldl … Read more
The recursion for foldr f x ys where ys = [y1,y2,…,yk] looks like f y1 (f y2 (… (f yk x) …)) whereas the recursion for foldl f x ys looks like f (… (f (f x y1) y2) …) yk An important difference here is that if the result of f x y can … Read more
The method fold (originally added for parallel computation) is less powerful than foldLeft in terms of types it can be applied to. Its signature is: def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1 This means that the type over which the folding is done has to be a supertype of the collection element … Read more