Container failed, exitCode=-1000. No space available in any of the local directories.

Problem

Container failed, exitCode=-1000. No space available in any of the local directories.

Cause

Out of local disc space on executing datanodes.

Collect Details:

• Locate 'core-site.xml' to determine value for 'hadoop.tmp.dir'.
• Locate 'yarn-site.xml' to determine value for 'yarn.nodemanager.local-dirs'. 
• Identify which node(s) within your cluster executed the failed job.
• Identify which Hadoop user ID executed the failed job.

Solution

Ensure the fully interpolated path for 'yarn.nodemanager.local-dirs' has sufficient space to store intermediate execute data.

If purging that path is required, you must bring that datanode offline before doing so.