Is it better to use the mapred or the mapreduce package to create a Hadoop Job?

by

Functionality wise there is not much difference between the old (o.a.h.mapred) and the new (o.a.h.mapreduce) API. The only significant difference is that records are pushed to the mapper/reducer in the old API. While the new API supports both pull/push mechanism. You can get more information about the pull mechanism here.

Also, the old API has been un-deprecated since 0.21. You can find more information about the new API here.

As you mentioned some of the classes (like MultipleTextOutputFormat) have not been migrated to the new API, due to this and the above mentioned reason it’s better to stick to the old API (although a translation is usually quite simple).

More Related Contents:

  1. How does Hadoop process records split across block boundaries?
  2. merge output files after reduce phase
  3. Container is running beyond memory limits
  4. Setting the number of map tasks and reduce tasks
  5. Chaining multiple MapReduce jobs in Hadoop
  6. When do reduce tasks start in Hadoop?
  7. How to get the input file name in the mapper in a Hadoop program?
  8. Hadoop speculative task execution
  9. Hive unable to manually set number of reducers
  10. how many mappers and reduces will get created for a partitoned table in hive
  11. hadoop map reduce secondary sorting
  12. Where does hadoop mapreduce framework send my System.out.print() statements ? (stdout)
  13. How does Hadoop perform input splits?
  14. What is the use of grouping comparator in hadoop map reduce
  15. Hadoop input split size vs block size
  16. Default number of reducers
  17. What is Hive: Return Code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
  18. What is Google’s Dremel? How is it different from Mapreduce?
  19. Hadoop WordCount example stuck at map 100% reduce 0%
  20. Can Hive recursively descend into subdirectories without partitions or editing hive-site.xml?
  21. hadoop: difference between 0 reducer and identity reducer?
  22. Failed to locate the winutils binary in the hadoop binary path
  23. What is the difference between partitioning and bucketing a table in Hive ?
  24. Sqoop import : composite primary key and textual primary key
  25. Hadoop java.io.IOException: Mkdirs failed to create /some/path
  26. data block size in HDFS, why 64MB?
  27. How can I access S3/S3n from a local Hadoop 2.6 installation?
  28. java.lang.NoClassDefFoundError: org/apache/hadoop/fs/StorageStatistics
  29. Is it better to have one large parquet file or lots of smaller parquet files?
  30. Parallel Algorithms for Generating Prime Numbers (possibly using Hadoop’s map reduce)

Leave a Comment