Goal
This guide will explain the available Custom Properties in Datameer 5.6 and later for adjusting the memory usage of Hadoop tasks.
Learn
The following parameters were introduced in Datameer 5.6 and are mentioned in the New and Noteworthy section of this release:
|
Property name
|
Default value
|
Description
|
|---|---|---|
| das.job.map-task.memory | 2048 | Controls the container size of map-like tasks across execution frameworks. |
| das.job.reduce-task.memory | 2048 | Controls the container size of the reduce-like tasks across execution frameworks. |
| das.job.application-manager.memory | 2048 | Controls the container size of the Application Master across execution frameworks. |
These parameters are to be used universally in Datameer to update the memory usage for Map tasks, Reduce tasks and YARN Application Manager.
The value of each of these parameters is defined in MB. It is worth noting that the value is the container size, not the JVM memory sizing.
These parameters affect all execution frameworks of Datameer. No longer do separate custom properties need to be defined for MapReduce jobs and TEZ jobs.
Lastly, it is worth noting that these new settings override the execution framework existing settings.
Comments
0 comments
Please sign in to leave a comment.