How to get the input file name in the mapper in a Hadoop program?

by

First you need to get the input split, using the newer mapreduce API it would be done as follows:

context.getInputSplit();

But in order to get the file path and the file name you will need to first typecast the result into FileSplit.

So, in order to get the input file path you may do the following:

Path filePath = ((FileSplit) context.getInputSplit()).getPath();
String filePathString = ((FileSplit) context.getInputSplit()).getPath().toString();

Similarly, to get the file name, you may just call upon getName(), like this:

String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();

More Related Contents:

  1. How does Hadoop process records split across block boundaries?
  2. merge output files after reduce phase
  3. Container is running beyond memory limits
  4. Setting the number of map tasks and reduce tasks
  5. Is it better to use the mapred or the mapreduce package to create a Hadoop Job?
  6. Chaining multiple MapReduce jobs in Hadoop
  7. When do reduce tasks start in Hadoop?
  8. Hadoop speculative task execution
  9. Hive unable to manually set number of reducers
  10. how many mappers and reduces will get created for a partitoned table in hive
  11. hadoop map reduce secondary sorting
  12. Where does hadoop mapreduce framework send my System.out.print() statements ? (stdout)
  13. How does Hadoop perform input splits?
  14. What is the use of grouping comparator in hadoop map reduce
  15. Hadoop input split size vs block size
  16. Default number of reducers
  17. What is Hive: Return Code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
  18. What is Google’s Dremel? How is it different from Mapreduce?
  19. Hadoop WordCount example stuck at map 100% reduce 0%
  20. Can Hive recursively descend into subdirectories without partitions or editing hive-site.xml?
  21. hadoop: difference between 0 reducer and identity reducer?
  22. Failed to locate the winutils binary in the hadoop binary path
  23. What is the difference between partitioning and bucketing a table in Hive ?
  24. Sqoop import : composite primary key and textual primary key
  25. Oozie: Launch Map-Reduce from Oozie action?
  26. Hadoop java.io.IOException: Mkdirs failed to create /some/path
  27. Is it better to have one large parquet file or lots of smaller parquet files?
  28. Parallel Algorithms for Generating Prime Numbers (possibly using Hadoop’s map reduce)
  29. Difference between hadoop fs -put and hadoop fs -copyFromLocal
  30. Hive Data Retrieval Queries: Difference between CLUSTER BY, ORDER BY, and SORT BY

Leave a Comment