Problem
An ExportJob runs with a single tasks even if there are sufficient cluster resources and the exported data could be easily split. Custom properties such as das.splitting.min-split-count / das.splitting.max-split-count do not change any behavior of the Export Job's execution.
Cause
A possible cause for this behaviour is a Sort or a Filter on the sheet being used as the Export Job's source. In this case the export will use a single task to merge and output a sorted result.
Solution
Create a duplicate of the filtered or sorted Sheet using the Copy data only option and use this new copied sheet as the source for the Export Job. This will clear the condition for a single task execution and allow parallelization during the export process.
Comments
0 comments
Please sign in to leave a comment.