We experienced a couple of jobs that failed today. Looking at the log, it gave us the message of
Too Many Open Files.
java.io.IOException: Cannot run program "tail": java.io.IOException: error=24, Too many open files ... [anonymous] ERROR (DasExceptionFilter.java:43) - Unhandled exception handling /admin/application-log ... java.lang.RuntimeException: exception on executing command '[tail, -n, 1000, /opt/datameer/<path>/logs/conductor.log]'
If you are interested in knowing what files are currently open you can use the following command.
lsof -p <pid of jvm>
You can also see what the
ulimit for open file is using:
This error is coming from the operating system because of not having enough file descriptors.You might need to increase the max number of open files - global level or per-user limits (or both).
To raise the global limit for file descriptors you need to edit, depending on your operating system, the
To do this on a user level you need to edit the
Usually we recommend to set the values mentioned within in given knowledge base articles.