How to Save Hadoop Task Logs in a Job Trace

Goal

This article will explain how to force Datameer to save Hadoop task logs so that they are accessible by a Job Trace. 

This is a common practice when troubleshooting a Datameer job failure. 

Learn

Follow these steps to enable task collection for a Datameer job:

• In the Datameer GUI, configure the desired job. 
• Navigate to the Custom Properties section of the configuration
• For Import Jobs, Data Links and Export Jobs this is found on the Schedule tab of the configuration in the Advanced section
• For Workbooks, this is found in the Advanced section of the configuration.
• Add the following line to the Custom Properties:

  das.debug.tasks.logs.collect.force=true

• Re-run the job to activate the new parameter.

Once these steps have been completed, the Hadoop task logs will automatically be saved by Datameer and will be available in a Job Trace. The steps to collect a Job Trace are documented here: How To Collect a Job Trace.