Problem
When trying to export a job, multiple files are created, even though there is no specification for splitting files. Desired outcome would be for a single file.
Cause
This is standard behavior. By default, Hadoop will distribute the export job, resulting in split files.
Resolution
Datameer can override these settings, by entering the following parameter to your Custom Hadoop Properties:
das.splitting.max-split-count=1
Note, this may cause performance slowdowns with large exports.
Comments
0 comments
Please sign in to leave a comment.