Swarm Cronjob

See what we're up to: Runway is here! 🚀

Swarm-cronjob #

Swarm-cronjob brings a lot of convenience to the table. “Jobs” are configured via labels. It works well in a distributed setup and even implements basic locking mechanisms so the tasks don’t build up when one is running already.

Setup #

Deploy the following to your endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

version: "3.8"

services:
  swarm-cronjob:
    image: crazymax/swarm-cronjob
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    environment:
      - "TZ=Europe/Berlin"
      - "LOG_LEVEL=info"
      - "LOG_JSON=false"
    deploy:
      placement:
        constraints:
          - node.role == manager

A few things to note:

  1. swarm-cronjob will use docker APIs to look for new containers (more on this below). While convenient for this use-case, access to the Docker APIs has security implications.
  2. Take note of the LOG_LEVEL in case jobs don’t work as expected.
  3. Don’t run :latest :-)
To save some time, we've assembled the stack and a `Makefile` (for `make deploy`) into a repository on Github: [hostwithquantum/example-swarm-cronjob](https://github.com/hostwithquantum/example-swarm-cronjob)

After a successful deployment of the stack, swarm-cronjob will wait for new services and directions. No further restarts are required.

Example #

Add the following to your docker-compose.yml:

10
11
12
13
14
15
16
17
18
19
20
21
22
23

...
  backup:
    image: minio/mc
    command: sh -c "mc cp -r /backup minio-alias/bucket"
    volumes:
      - backup:/backup:ro
    deploy:
      labels:
        - "swarm.cronjob.enable=true"
        - "swarm.cronjob.schedule=* */30 * * *"
        - "swarm.cronjob.skip-running=true"
      replicas: 0
      restart_policy:
        condition: none

The example runs the backup service/cronjob every 30 minutes.

For brevity, we omit the configuration of the `mc` client, but see the [official documentation](https://docs.min.io/docs/minio-client-complete-guide) for help.

Tips #

The format used in cron expression in swarm-cronjob is not not the usual `crond`-format you may be familiar with. Instead of minutes (on the left), this expression starts with seconds. See [the documentation](https://pkg.go.dev/github.com/robfig/cron?utm_source=godoc#hdr-CRON_Expression_Format) for details.

When multiple jobs are deployed, execution may need to be spread out:

  • each job requires certain resources to start and run
  • executing many of jobs in parallel can impact other services