Articles in this section

See more

SFTP Import Job Fails with "session is down" Error

Joel Stewart
  • Updated
Follow

Problem

After upgrading to Datameer 4.4, some SFTP Import Jobs fail. When they fail, the following stack trace is generated:

java.lang.RuntimeException: Cannot create connection. Reason is com.jcraft.jsch.JSchException: session is down, session is down
at datameer.dap.sdk.datastore.FileDataStoreModel.testConnect(FileDataStoreModel.java:58)
at datameer.dap.sdk.entity.DataStore.validate(DataStore.java:186)
at datameer.dap.conductor.webapp.commandObject.ConnectionCommand.validateConnection(ConnectionCommand.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springmodules.validation.bean.rule.ValidationMethodValidationRule$MethodCondition.doCheck(ValidationMethodValidationRule.java:39)
at org.springmodules.validation.util.condition.AbstractCondition.check(AbstractCondition.java:36)
at org.springmodules.validation.bean.BeanValidator.validateAndShortCircuitRules(BeanValidator.java:429)
at org.springmodules.validation.bean.BeanValidator.applyPropertiesValidationRules(BeanValidator.java:412)
at org.springmodules.validation.bean.BeanValidator.applyBeanValidation(BeanValidator.java:349)
at org.springmodules.validation.bean.BeanValidator.validateObjectGraphConstraints(BeanValidator.java:181)
at org.springmodules.validation.bean.BeanValidator.validate(BeanValidator.java:99)
at datameer.dap.conductor.webapp.controller.data.datastore.DataStoreW2DetailsController.submitConnectionDetails(DataStoreW2DetailsController.java:127)
at datameer.dap.conductor.webapp.controller.data.datastore.DataStoreW2DetailsController$$FastClassByCGLIB$$d340b755.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
at datameer.dap.conductor.webapp.controller.data.datastore.DataStoreW2DetailsController$$EnhancerByCGLIB$$2610c226.submitConnectionDetails(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:574)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at datameer.dap.conductor.webapp.filter.CsrfFilter.doHttpFilter(CsrfFilter.java:79)
at datameer.dap.conductor.webapp.filter.CsrfFilter.doFilter(CsrfFilter.java:58)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.SessionFilter.doFilter(SessionFilter.java:51)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.RestAuthenticationOriginFilter.doFilter(RestAuthenticationOriginFilter.java:52)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)
at datameer.dap.conductor.webapp.filter.GzipFilter.doFilter(GzipFilter.java:110)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.OptimisticLockRetryFilter.doFilter(OptimisticLockRetryFilter.java:31)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.PermissionDeniedExceptionFilter.doFilter(PermissionDeniedExceptionFilter.java:29)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.RequestMetaDataFilter.doFilter(RequestMetaDataFilter.java:27)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.PersistenceFilter.doFilter(Unknown Source)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at datameer.dap.conductor.authentication.CompositeAuthenticationFilter$InternalFilterChain.doFilter(CompositeAuthenticationFilter.java:67)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
at datameer.dap.conductor.authentication.CompositeAuthenticationFilter$InternalFilterChain.doFilter(CompositeAuthenticationFilter.java:70)
at datameer.dap.conductor.authentication.CompositeAuthenticationFilter.doFilter(CompositeAuthenticationFilter.java:51)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at datameer.dap.conductor.webapp.filter.AutoAdminAuthenticationFilter.doFilter(AutoAdminAuthenticationFilter.java:65)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.ClickJackingPreventionFilter.doFilter(ClickJackingPreventionFilter.java:40)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.web.multipart.support.MultipartFilter.doFilterInternal(MultipartFilter.java:113)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.OptimisticLockRetryFilter.doFilter(OptimisticLockRetryFilter.java:31)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.PermissionDeniedExceptionFilter.doFilter(PermissionDeniedExceptionFilter.java:29)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.RequestMetaDataFilter.doFilter(RequestMetaDataFilter.java:27)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.DeprecatedRestUrlsFilter.doFilter(DeprecatedRestUrlsFilter.java:90)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.BrowserExceptionFilter.doFilter(BrowserExceptionFilter.java:31)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.RestExceptionFilter.doFilter(RestExceptionFilter.java:30)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.NoResultExceptionFilter.doFilter(NoResultExceptionFilter.java:24)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.DasExceptionFilter.doFilter(DasExceptionFilter.java:29)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at datameer.dap.conductor.webapp.filter.RequestContextFilter.doFilter(RequestContextFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at com.jamonapi.http.JAMonJettyHandler.handle(JAMonJettyHandler.java:40)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.io.IOException: com.jcraft.jsch.JSchException: session is down
at datameer.dap.hadoop.filesystem.SftpFileSystem.runInSftpChannel(SftpFileSystem.java:246)
at datameer.dap.hadoop.filesystem.SftpFileSystem.getFileStatus(SftpFileSystem.java:189)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:768)
at datameer.dap.sdk.cluster.filesystem.HadoopClusterFileSystem.exists(HadoopClusterFileSystem.java:114)
at datameer.dap.sdk.datastore.FileDataStoreModel.testConnect(FileDataStoreModel.java:50)
... 144 more
Caused by: com.jcraft.jsch.JSchException: session is down
at com.jcraft.jsch.Channel.connect(Channel.java:185)
at com.jcraft.jsch.Channel.connect(Channel.java:144)
at datameer.dap.hadoop.filesystem.SftpFileSystem.runInSftpChannel(SftpFileSystem.java:238)
... 148 more

This error was observed in an environment where the configured user configured for the SFTP connection is not authorized to remotely login to the target SFTP (rlogin=false). The configured user account is only permitted to access data via the SFTP protocol.

The md5 sum of the <DatameerInstallationDirectory>/webapp/ROOT/WEB-INF/lib/aws-tasks-0.6.dev.jar file is: 7796c7f192d7a6a6151c3e2647295a31.

Cause

This is a known defect which was introduced in Datameer 4.4. 

Solution

To work-around this issue, the aws-tasks-0.6.dev.jar file may be replaced with an older version of the jar file. The md5 sum of an aws-tasks-0.6.dev.jar file which is not affected by this issue is e246980522366cea5edc87798c241c63. This version of the aws-tasks-0.6.dev.jar file was distributed with Datameer 4.1, 4.2 and 4.3.

Here are the steps to replace the aws-tasks-0.6.dev.jar file.
  1. Stop Datameer.
  2. Move the <DatameerInstallationDirectory>/webapp/ROOT/WEB-INF/lib/aws-tasks-0.6.dev.jar file to <DatameerInstallationDirectory>/aws-tasks-0.6.dev.jar.7796c7f192d7a6a6151c3e2647295a31
  3. Download a copy of the aws-tasks-0.6.dev.jar file with the md5 sum of e246980522366cea5edc87798c241c63.
  4. Copy the downloaded aws-tasks-0.6.dev.jar file to the <DatameerInstallationDirectory>/webapp/ROOT/WEB-INF/lib directory.
  5. Ensure that the aws-tasks-0.6.dev.jar file has file ownership and file permissions that match the other *.jar files in the <DatameerInstallationDirectory>/webapp/ROOT/WEB-INF/lib directory.
  6. Delete the <DatameerInstallationDirectory>/job-jar/aws-tasks-0.6.dev.jar and <DatameerInstallationDirectory>/job-jar/aws-tasks-0.6.dev.jar.md5 files.
  7. Copy the downloaded aws-tasks-0.6.dev.jar file to the <DatameerInstallationDirectory>/job-jar directory.
  8. Ensure that the aws-tasks-0.6.dev.jar file has file ownership and file permissions that match the other *.jar files in the <DatameerInstallationDirectory>/job-jar directory.
  9. Start Datameer.
  10. If Datameer is connected to a Hadoop cluster, remove the following file from HDFS: hdfs://<Server>/<DatameerPrivateFolder>/jobjars/<DatameerVersion>/jar/aws-tasks-0.6.dev.jar_7796c7f192d7a6a6151c3e2647295a31.jar (The new Job Jar will be copied automatically when a job is submitted.)

 

 

Related articles

Comments

0 comments

Please sign in to leave a comment.