Hadoop DistributedCache is deprecated – what is the preferred API?

by

The APIs for the Distributed Cache can be found in the Job class itself. Check the documentation here: http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html
The code should be something like 

Job job = new Job();
...
job.addCacheFile(new Path(filename).toUri());

In your mapper code: 

Path[] localPaths = context.getLocalCacheFiles();
...

More Related Contents:

  1. Calling a mapreduce job from a simple java program
  2. Oozie: Launch Map-Reduce from Oozie action?
  3. Hadoop Word count: receive the total number of words that start with the letter “c”
  4. Reading file as single record in hadoop
  5. MultipleOutputFormat in hadoop
  6. Change File Split size in Hadoop
  7. hadoop java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.1-hadoop2.jar
  8. Hadoop “Unable to load native-hadoop library for your platform” warning
  9. Hive: Best way to do incremetal updates on a main table
  10. Read whole text files from a compression in Spark
  11. hadoop No FileSystem for scheme: file
  12. Write a file in hdfs with Java
  13. Should I call ugi.checkTGTAndReloginFromKeytab() before every action on hadoop?
  14. How to use Sqoop in Java Program?
  15. Hadoop cluster setup – java.net.ConnectException: Connection refused
  16. How to convert .txt file to Hadoop’s sequence file format
  17. Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
  18. Iterate twice on values (MapReduce)
  19. Accessing a mapper’s counter from a reducer
  20. NotSerializableException on anonymous class
  21. How do I build/run this simple Mahout program without getting exceptions?
  22. Java8: HashMap to HashMap using Stream / Map-Reduce / Collector
  23. Is gzip format supported in Spark?
  24. Change output filename prefix for DataFrame.write()
  25. out of Memory Error in Hadoop
  26. How can I force Spark to execute code?
  27. How to list all files in a directory and its subdirectories in hadoop hdfs
  28. How to run a jar file in hadoop?
  29. Exception “: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length” from java
  30. Java GC (Allocation Failure)

Leave a Comment