Current version: 0.0.0 (Not released)
scw extends scheduled commands to
add concurrency locking, prerequisites, dependency checks, conflict
avoidance, randomised startup delays, flexible logging, and monitoring
metrics.
There are no complex dependencies - all you need to run
scw is a scheduler such as
cron.
Purpose
Many scheduled commands either dump their output into an email, or write logs which maybe have a start and end timestamp, but nothing in between. If the operator doesn’t get the email or check the logs, they won’t know about errors.
For example an old script might run 3 backups and produce a log like this:
Starting at Sat 9 Nov 10:42:22 GMT 2024 Backup job 1 OK tar: /root: Cannot open: Permission denied Backup job 2 failed! Backup job 3 OK Finished at Sat 9 Nov 13:42:43 GMT 2024
Searching the log for “failed” won’t say which day contained the failure, if the log spans several days - and there’s no way to tell specifically when backup job 2 failed, making fault analysis difficult.
Running the same script under scw* produces logs like this:
2024-11-09T10:42:22 [s] (begin) 2024-11-09T10:42:22 [-] Starting at Sat 9 Nov 10:42:22 GMT 2024 2024-11-09T11:20:01 [-] Backup job 1 OK 2024-11-09T11:30:47 [E] tar: /root: Cannot open: Permission denied 2024-11-09T11:50:55 [-] Backup job 2 failed! 2024-11-09T13:42:42 [-] Backup job 3 OK 2024-11-09T13:42:43 [-] Finished at Sat 9 Nov 13:42:43 GMT 2024 2024-11-09T13:42:43 [s] (end) exit status 1, elapsed time 10821s
The output is timestamped, the exit status of the script is recorded, and an error marker file is created for monitoring systems to detect. This is all done without having to change the original script.
Features
Each distinct scheduled command is referred to as an “item” and can have its own configuration for each of these features:
Concurrency locking prevents an item from being run more than once at the same time, for example if it’s scheduled to run every 2 minutes and occasionally takes longer than that to complete.
Prerequisites allow an item to be prevented from running if some prior condition is not met, for example checking whether this is the active node of a failover cluster, or whether some crucial underpinning service is running.
Dependency checks allow an item to run only if some other item has succeeded before it - with the possibility of waiting a short while for the dependency to finish rather than giving up straight away. Useful when linked items need to be scheduled separately at particular times but the later one can only run if the earlier one has succeeded. For example, a data load batch may have to run at some time in the early morning, and a subsequent data processing batch, which for business reasons has to run after a particular time of day, can only run if the early morning data load succeeded.
Conflict avoidance prevents an item from running if some other item is still running - with the possibility of waiting a short while for the conflicting item to finish rather than giving up straight away.
Randomised startup delays can be configured for items, which is useful for avoiding resource overconsumption when the same item is scheduled to run on multiple systems.
Logging can be to any combination of files, syslog, email, or HTTP, with or without timestamps. The standard output and standard error of items can be combined or separated, and a special status stream is also made available so that significant events (such as starting each step of a multi-step process) can be recorded separately.
Metrics files are generated for each item. Any monitoring agent can read these files and raise alerts based on their contents. An item list file (a JSON array of item descriptions) is automatically generated so that a system such as Zabbix can find and monitor all items without needing an operator to make adjustments when items are added or removed.
In its simplest form, scw can be invoked from existing scheduler entries like this:
# Original entry 0 * * * * /some/command --option ARGUMENT # Replacement 0 * * * * scw run mycommand -- /some/command --option ARGUMENT
To make use of the full range of features, scheduled command
definitions can instead be placed into the scw item
definition directory, and “scw update” can then generate
the appropriate crontab.
This software is distributed under the terms of the GNU General Public License version 3 or later.
- Manual for SCW
- Git repository (at Codeberg)
- Issue tracker (at Codeberg)
This package has not yet been released, as it is not yet in a usable state. See the Git repository for the current source code.
Comments, bug reports, and patches can be sent using the Issue tracker, or through the Contact Form.