Job Failure - java.io.FileNotFoundException: /<org>/dmr/file-stores/.../<input_file>

Problem

Import jobs fail with the following error:

ERROR [<timestamp>] [JobScheduler thread-1] (JobScheduler.java:809) - Job <jobID> failed with exception.
java.lang.RuntimeException: Cannot open maprfs:/<org>/dmr/file-stores/.../<input_file> at
datameer.dap.sdk.util.ExceptionUtil.convertToRuntimeException(ExceptionUtil.java:49) at
datameer.dap.common.filestore.DfsFileStore.open(DfsFileStore.java:140) at
datameer.plugins.cobol.CobolFileType.loadCopyBookHelper(CobolFileType.java:245) at
datameer.plugins.cobol.CobolFileType.createModel(CobolFileType.java:266) at
datameer.plugins.cobol.CobolFileType.createModel(CobolFileType.java:36) at
datameer.dap.sdk.importjob.FileImportJobModel.createImportFormat(FileImportJobModel.java:95) at
datameer.dap.sdk.importjob.FileImportJobModel.createImportFormat(FileImportJobModel.java:40) at
datameer.dap.sdk.importjob.ImportJobModel.createImportFormat(ImportJobModel.java:69) at
datameer.dap.conductor.job.ImportJobCreator.createDatameerJob(ImportJobCreator.java:59) at
datameer.dap.conductor.job.JobSchedulerJob$1$1.call(JobSchedulerJob.java:95) at
datameer.dap.conductor.job.JobSchedulerJob$1$1.call(JobSchedulerJob.java:87) at
datameer.dap.conductor.persistence.TransactionHandler.execute(TransactionHandler.java:118) at
datameer.dap.conductor.persistence.TransactionHandler.executeInTransaction(TransactionHandler.java:110) at
datameer.dap.conductor.job.JobSchedulerJob$1.call(JobSchedulerJob.java:87) at
datameer.dap.conductor.job.JobSchedulerJob$1.call(JobSchedulerJob.java:83) at
datameer.dap.common.security.DatameerSecurityService.runAsUser(DatameerSecurityService.java:96) at
datameer.dap.conductor.job.JobSchedulerJob.call(JobSchedulerJob.java:83) at
datameer.dap.conductor.job.JobSchedulerJob.call(JobSchedulerJob.java:40) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at
java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException:
/<org>/dmr/file-stores/.../<input_file> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:189) at
com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:810) at
org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800) at
datameer.dap.sdk.cluster.filesystem.HadoopFileSystem.open(HadoopFileSystem.java:106) at
datameer.dap.common.filestore.DfsFileStore.getRawInputStream(DfsFileStore.java:154) at
datameer.dap.common.filestore.GenericFileStore.getInputStream(GenericFileStore.java:66) at
datameer.dap.common.filestore.DfsFileStore.open(DfsFileStore.java:138) ... 20 more

Cause

The underlying connection is broken due a the missing file - in this casem it is the <input_file>. The directory <org>/dmr has 777 permissions and some scripts accidentally removed the underlying files.

Solution

The core Datameer directory shouldn't be given 770 permissions. Se the directory to 770 to avoid problems like the above. Once the missing file is restored, the connection and the import job run successfully.