written 5.6 years ago by |
Combiner:
i. A Reduce function is associative and commutative. That is, the values to be combined can be combined in any order, with the same result.
ii. The addition performed in Example 1 is an example of an associative and commutative operation. It doesn’t matter how we group a list of numbers v1, v2, . . . , vn; the sum will be the same.
iii. When the Reduce function is associative and commutative, we can push some of what the reducers do to the Map tasks
iv. These key-value pairs would thus be replaced by one pair with key w and value equal to the sum of all the 1’s in all those pairs.
v. That is, the pairs with key w generated by a single Map task would be replaced by a pair (w, m), where m is the number of times that w appears among the documents handled by this Map task.