RuntimeException: Task: Error: Unsupported major.minor version

Problem

After creating jobs and deploying to the cluster, all jobs will fail instantly. 

Error Message

...
INFO [ConcurrentJobExecutor-0] (DefaultMrJobClient.java:121) - Waiting on completion of job: job_<id>_0001
INFO [ConcurrentJobExecutor-0] (DefaultMrJobClient.java:140) - job_<id>_0001: map 100% reduce n/a (total 100%)
WARN [ConcurrentJobExecutor-0] (DefaultMrJobClient.java:149) - Task Id : attempt_<id>_0001_m_000001_0, Status : FAILED
WARN [ConcurrentJobExecutor-0] (DefaultMrJobClient.java:154) - attempt_<id>_0001_m_000001_0: Error: datameer/dap/common/graphv2/hadoop/MrJobOutputFormat : Unsupported major.minor version 51.0
...

Troubleshooting

Check the Java version and path on Datameer node as well on cluster nodes. 

[datameer@hostname bin]$ ps -ef | grep java | grep -i datameer 
datameer 26627 55598 0 11:11 pts/7 00:00:00 grep java 
datameer 31315 1 1 06:56 pts/7 00:04:38 /usr/java/jdk1.7.0_51/bin/java -Xmx4096m -XX:MaxPermSize=384m -Xms256m -XX:MaxNewSize=448m -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -Dfile.encoding=utf-8 -server -Ddeploy.mode=live -Djetty.port=8080 -XX:HeapDumpPath=/var/app/datameer/Datameer-<version>-<dist>/logs/heapdump.log -Dlog4j.configuration=file:///var/app/datameer/Datameer-<version>-<dist>/conf/log4j-production.properties -Djetty.home=/var/app/datameer/Datameer-<version>-<dist> -Djava.io.tmpdir=tmp -Djava.library.path=/var/app/datameer/Datameer-<version>-<dist>/lib/native -jar /var/app/datameer/Datameer-<version>-<dist>/start.jar /var/app/datameer/Datameer-<version>-<dist>/etc/jetty-logging.xml /var/app/datameer/Datameer-<version>-<dist>/etc/jetty-started.xml 
[datameer@hostname bin]$ java -version 
java version "1.7.0_51" 
Java(TM) SE Runtime Environment (build 1.7.0_51-b13) 
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode) 

[namenode@hostname jdk]$ ps -ef | grep java | grep namenode 
hdfs 11642 4517 1 06:46 ? 00:03:03 /usr/java/1.6.0_81l64/bin/java -Dproc_namenode -Xmx1000m -Dhdfs.audit.logger=INFO,RFAAUDIT -Dsecurity.audit.logger=INFO,RFAS -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=<nameservice>.log.out -Dhadoop.home.dir=/opt/cloudera/parcels/CDH/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/opt/cloudera/parcels/CDH/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xms4294967296 -Xmx4294967296 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.NameNode 
[namenode@hostname jdk]$ java -version 
java version "1.7.0_51" 
Java(TM) SE Runtime Environment (build 1.7.0_51-b13) 
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

Cause

In this case it is a mismatch in Java versions between the Datameer node 

/usr/java/jdk1.7.0_51/bin/java

and the Data Nodes (DN) in the cluster 

/usr/java/1.6.0_81l64/bin/java

and/or a not correct set path to theJava Runtime Environment (JRE).

Solution

Have same Java version on all nodes. Set JRE path correct.