Articles in this section

See more

SSH/SFTP Import Job Error - Cannot parse input. Unable to determine available size

Joel Stewart
  • Updated
Follow

Environment

DM: 4.5.5, OS: Linux, DIST: -, COM: SSH, SFTP, Connection

Problem

As a user, I'm unable to import an ISO-8859-1 encoded file successfully using an SSH/SFTP connection. I receive the following error in the GUI when trying to generate the preview data to define the columns: 

Cannot parse input. Unable to determine available size.

Additionally, the following stacktrace is logged in the conductor.log file:

[superuser]  WARN [2015-02-04 09:56:59.281] [1624142202@qtp-632674997-3] (DataSourceCommand.java:698) - failed to guess field definitions
java.lang.RuntimeException: Unable to determine available size
        at datameer.dap.hadoop.filesystem.SftpFileInputStream.available(SftpFileInputStream.java:43)
...
Caused by: java.io.IOException: Unable to stat remote file
        at datameer.dap.hadoop.filesystem.SftpFileInputStream.getRemoteFileSize(SftpFileInputStream.java:157)
        at datameer.dap.hadoop.filesystem.SftpFileInputStream.available(SftpFileInputStream.java:41)
        ... 167 more
Caused by: 1: End of file
        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2569)
        at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2018)
        at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2035)
        at com.jcraft.jsch.ChannelSftp.stat(ChannelSftp.java:1992)
        at datameer.dap.hadoop.filesystem.SftpFileInputStream.getRemoteFileSize(SftpFileInputStream.java:155)
        ... 168 more

Cause

This is a known bug affecting Datameer 4.5.5. Specifically, this issue affects SSH and SFTP connections. 

Solution

To work-around this issue, attempt to import this file using another connection. For example, use an HDFS connection instead. 

This issue is resolved in Datameer 5.2 and later.

 

Related articles

Comments

0 comments

Please sign in to leave a comment.