Swarm Cronjob

Swarm-cronjob #

Swarm-cronjob brings a lot of convenience to the table. “Jobs” are configured via labels. It works well in a distributed setup and even implements basic locking mechanisms so the tasks don’t build up when one is running already.

Setup #

Deploy the following to your endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

version: "3.8"

services:
  swarm-cronjob:
    image: crazymax/swarm-cronjob
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    environment:
      - "TZ=Europe/Berlin"
      - "LOG_LEVEL=info"
      - "LOG_JSON=false"
    deploy:
      placement:
        constraints:
          - node.role == manager

A few things to note:

  1. swarm-cronjob will use docker APIs to look for new containers (more on this below). While convenient for this use-case, access to the Docker APIs has security implications.
  2. Take note of the LOG_LEVEL in case jobs don’t work as expected.
  3. Don’t run :latest :-)
To save some time, we’ve assembled the stack and a Makefile (for make deploy) into a repository on Github: hostwithquantum/example-swarm-cronjob

After a successful deployment of the stack, swarm-cronjob will wait for new services and directions. No further restarts are required.

Example #

Add the following to your docker-compose.yml:

10
11
12
13
14
15
16
17
18
19
20
21
22
23

...
  backup:
    image: minio/mc
    command: sh -c "mc cp -r /backup minio-alias/bucket"
    volumes:
      - backup:/backup:ro
    deploy:
      labels:
        - "swarm.cronjob.enable=true"
        - "swarm.cronjob.schedule=* */30 * * *"
        - "swarm.cronjob.skip-running=true"
      replicas: 0
      restart_policy:
        condition: none

The example runs the backup service/cronjob every 30 minutes.

For brevity, we omit the configuration of the mc client, but see the official documentation for help.

Tips #

The format used in cron expression in swarm-cronjob is not not the usual crond-format you may be familiar with. Instead of minutes (on the left), this expression starts with seconds. See the documentation for details.

When multiple jobs are deployed, execution may need to be spread out:

  • each job requires certain resources to start and run
  • executing many of jobs in parallel can impact other services