Is it better to use the mapred or the mapreduce package to create a Hadoop Job?
To create MapReduce jobs you can either use the old org.apache.hadoop.mapred package or the newer org.apache.hadoop.mapreduce package for Mappers and Reducers, Jobs ... The first one had been marked as deprecated but this got reverted meanwhile. Now I wonder whether it is better to use the old mapred package or the new mapreduce package to create a job and why. Or is it just dependent on whether you need stuff like the MultipleTextOutputFormat which is only available in the old mapred package?
Solution 1:
Functionality wise there is not much difference between the old (o.a.h.mapred) and the new (o.a.h.mapreduce) API. The only significant difference is that records are pushed to the mapper/reducer in the old API. While the new API supports both pull/push mechanism. You can get more information about the pull mechanism here.
Also, the old API has been un-deprecated since 0.21. You can find more information about the new API here.
As you mentioned some of the classes (like MultipleTextOutputFormat) have not been migrated to the new API, due to this and the above mentioned reason it's better to stick to the old API (although a translation is usually quite simple).
Solution 2:
Both the old and new APIs are good. The new API is cleaner though. Use the new API wherever you can, and use the old one wherever you need specific classes that are not present in the new API (like MultipleTextOutputFormat)
But do take care not to use a mix of the old and new APIs in the same Mapreduce job. That leads to weird problems.
Solution 3:
Old API (mapred)
Exists in Package org.apache.hadoop.mapred
Provide A map/reduce job configuration.
- Reduces values for a given key, based on the Iterator
- Package Summary
New API (mapreduce)
Exists in Package org.apache.hadoop.mapreduce
Job configuration is done by separate class, Called JobConf which is extension of Configuration
ClassReduces values for a given key, based on the Iterable
Package Summary