Scheduled Jobs are not triggered by Datameer Scheduler

Problem

Scheduled Datameer artifacts are not being triggered by the scheduler.

Cause

One possible reason of this issue is a non-correct cron pattern for a scheduled execution that is saved in the Datameer database.

Job trigger settings for every artifact are stored in the Datameer application database in the table dap_job_configuration.

• Column pull_type keeps trigger type - manual (0), data-driven (1) or scheduled(2).
• Column schedule keeps corresponding cron pattern for jobs that have been scheduled to a certain time. 

By default, pull_type is set to manual and schedule value is NULL. In case you would like to schedule an artifact's execution, change the trigger type to scheduled and set up the appropriate cron pattern. When the cron pattern is invalid or doesn't set, the UI throws an error and asks for valid input.

The scheduler process reads data from the database at the application's startup only. All other changes made to jobs triggering settings in the Datameer UI or via REST API calls are handled directly by the scheduler demon.

In case for some reason (e.g. incorrect UPDATE statement executed against Datameer database) the pull_type for an artifact is set to scheduled (2), but cron pattern remains NULL or is invalid, at the next restart Datameer will report the below error and none of scheduled jobs will be triggered, even those which have the correct settings.

[anonymous] FATAL [2017-10-11 15:58:54.646] [pool-2-thread-2] (ContextListener.java:81) - uncaught exception from thread pool-2-thread-2
it.sauronsoftware.cron4j.InvalidPatternException: invalid pattern "2 y * * *". Error parsing hours field: invalid field "y", invalid element "y", invalid range, invalid value "y", invalid integer value.

or

[anonymous] FATAL [2017-10-10 17:18:32.952] [pool-2-thread-2] (ContextListener.java:81) - uncaught exception from thread pool-2-thread-2
it.sauronsoftware.cron4j.InvalidPatternException: invalid pattern: "NULL"

Solution

In order to fix this issue, it is required to review the artifacts configuration and ensure that scheduled jobs don't have invalid or NULL values configured.

The below MySQL query returns a list of scheduled jobs that have NULL for cron pattern.

SELECT pull_type,schedule,id FROM dap_job_configuration WHERE pull_type = 2 AND schedule = 'NULL';

In case this query returns any records, it's needed to change pull_type for these jobs to manual or set a valid cron pattern.

To let the scheduler demon pick up recent changes, please restart Datameer and ensure that no InvalidPatternException appears in conductor log.