Troubleshoot problematic Import Jobs and Export Jobs using an sFTP connection by implementing additional logging to debug and to determine what is preventing a successful job run.
First, occasionally there is a caching problem depending on the version of SSH/sFTP running on the host. Attempt re-running the job with the following Custom Property.
If this does not resolve the issue, remove the above parameter.
To begin debug troubleshooting, configure the artifact in question for a specific execution framework. Then implement the enhanced logging.
For Tez, related jobs should additionally be set within the import-specific
log4j.category.datameer=TRACE log4j.category.datameer.awstasks=DEBUG log4j.category.awstasks.com.jcraft=DEBUG log4j.category.org.apache.hadoop=DEBUG
Often, comparing the sshd_config and ssh_config files from the Datameer Host, Data Nodes, and sFTP Host can be a quick path to resolving sFTP issues. Notably, supported authentication and encryption mechanisms should be identical on all machines.