java.lang.IllegalArgumentException: Unable to parse - check settings for mapreduce.application.framework.path

Problem

After an update or change to the Hadoop version in a Hortonworks cluster, Datameer jobs fail in the MapReduce Framework. The following stacktrace is generated in the Job log: 

2015-08-06 18:14:53,435 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread 
java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path 
at org.apache.hadoop.mapreduce.v2.util.MRApps.getMRFrameworkName(MRApps.java:178) 
at org.apache.hadoop.mapreduce.v2.util.MRApps.setMRFrameworkClasspath(MRApps.java:203) 
at org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:248) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.getInitialClasspath(TaskAttemptImpl.java:620) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:755) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:812) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1527) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1504) 
at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362) 
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) 
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) 
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1069) 
at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:145) 
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1311) 
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1303) 
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173) 
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106) 
at java.lang.Thread.run(Thread.java:745) 
Caused by: java.net.URISyntaxException: Illegal character in path at index 11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework 
at java.net.URI$Parser.fail(URI.java:2829) 
at java.net.URI$Parser.checkChars(URI.java:3002) 
at java.net.URI$Parser.parseHierarchical(URI.java:3086) 
at java.net.URI$Parser.parse(URI.java:3044) 
at java.net.URI.<init>(URI.java:595) 
at org.apache.hadoop.mapreduce.v2.util.MRApps.getMRFrameworkName(MRApps.java:176) ... 18 more 

Of note, the Hadoop variable of mapreduce.application.framework.path contains a variable for hdp.version.

Cause

This is a configuration problem in Datameer. The hdp.version variable is not defined.

Solution

To resolve this issue, add or edit the following two parameters in the Datameer environment on the Administration tab in the Hadoop Cluster section's Custom Properties field: mapreduce.application.framework.path and mapreduce.application.classpath. 

The value of mapreduce.application.framework.path should be set to the HDFS path of the mapreduce.tar.gz file. The format most commonly looks similar to this /hdp/apps/<version>/mapreduce/mapreduce.tar.gz#mr-framework. Please verify that the characters before the # character are the path to a valid file in HDFS.

The value of mapreduce.application.classpath should be set to the output of the command hadoop classpath when run on any cluster member host.