hadoop: difference between 0 reducer and identity reducer?

by

You understanding is correct. I would define it as following:
If you do not need sorting of map results – you set 0 reduced,and the job is called map only.

If you need to sort the mapping results, but do not need any aggregation – you choose identity reducer.
And to complete the picture we have a third case : we do need aggregation and, in this case we need reducer.

More Related Contents:

  1. How does Hadoop process records split across block boundaries?
  2. merge output files after reduce phase
  3. Container is running beyond memory limits
  4. Setting the number of map tasks and reduce tasks
  5. Is it better to use the mapred or the mapreduce package to create a Hadoop Job?
  6. Chaining multiple MapReduce jobs in Hadoop
  7. When do reduce tasks start in Hadoop?
  8. How to get the input file name in the mapper in a Hadoop program?
  9. Hadoop speculative task execution
  10. Hive unable to manually set number of reducers
  11. how many mappers and reduces will get created for a partitoned table in hive
  12. hadoop map reduce secondary sorting
  13. Where does hadoop mapreduce framework send my System.out.print() statements ? (stdout)
  14. How does Hadoop perform input splits?
  15. What is the use of grouping comparator in hadoop map reduce
  16. Hadoop input split size vs block size
  17. Default number of reducers
  18. What is Hive: Return Code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
  19. What is Google’s Dremel? How is it different from Mapreduce?
  20. Hadoop WordCount example stuck at map 100% reduce 0%
  21. Can Hive recursively descend into subdirectories without partitions or editing hive-site.xml?
  22. Failed to locate the winutils binary in the hadoop binary path
  23. What is the difference between partitioning and bucketing a table in Hive ?
  24. Oozie: Launch Map-Reduce from Oozie action?
  25. Easiest way to install Python dependencies on Spark executor nodes?
  26. Change File Split size in Hadoop
  27. Difference between Pig and Hive? Why have both? [closed]
  28. Datanode process not running in Hadoop
  29. Behavior of the parameter “mapred.min.split.size” in HDFS
  30. Hive Data Retrieval Queries: Difference between CLUSTER BY, ORDER BY, and SORT BY

Leave a Comment