What is the use of grouping comparator in hadoop map reduce

In support of the chosen answer I add:

Following on from this explanation

**Input**:

    symbol time price
    a      1    10
    a      2    20
    b      3    30

**Map output**: create composite key\values like so:

> symbol-time time-price
>
>**a-1**         1-10
>
>**a-2**         2-20
>
>**b-3**         3-30

The Partitioner: will route the a-1 and a-2 keys to the same reducer despite the keys being different. It will also route the b-3 to a separate reducer.

GroupComparator: once the composites key\value arrive at the reducer instead of the reducer getting

>(**a-1**,{1-10})
>
>(**a-2**,{2-20})

the above will happen due to the unique key values following composition.

the group comparator will ensure the reducer gets:

(a-1,{**1-10,2-20**})

The key of the grouped values will be the one which comes first in the group. This can be controlled by Key comparator.

**[[In a single reduce method call.]]**

Leave a Comment