Watchtower Docker Compose with Cool Notifications [2024]

Docker is great, and paired with compose is even better. This should be a surprise for you if you have been following our guides, starting from our famous Docker Media Server guide.

But, what about the update process? I mean:

• Check when a new app image is released
• Pull the image
• Shutdown the container
• Restart the container
• Repeat for all the containers

If our docker-compose stack consists of only a few containers OR if we need to use a specific docker image version, this shouldn't sound like a big difficult task to accomplish OR could seem useless as we don't need to automatically update Docker container images.

But, if we don't belong to these two categories of users, this Watchtower Docker Compose guide is what we're looking for, and we could quietly go ahead and read!

What is this Watchtower you speak of?

When I started playing around with docker, I used to use Ouroboros to automatically update my app images.

The features were the same as Watchtower, more or less, and it did its job fine, but it was no longer developed and it wasn't possible to receive Telegram notifications.

So, I decided to switch to something else: Watchtower.

It checks periodically if new app images, of our containerized app, have been released, manages all the processes of pulling them down, and also the shutting down / restarting of containers that have been updated.

Basically, we can choose which containers to update and which not, and be notified when updates take places, with different methods, but there are other interesting options that we're going to see later.

Which Containers to Auto-Update?

This is a matter of personal preference. But I will share my opinion.

I strongly advice to automatically update NOT CRITICAL CONTAINERS ONLY. For example:

• Reverse proxy --> CRITICAL
• Smarthome management --> CRITICAL
• VPN --> COULD BE CRITICAL
• PLEX --> NOT CRITICAL

A good practice is to manually update the app images once the compatibility with docker compose configuration file has been tested upon old and new images.

For example, environment variables could be deleted or changed in the new image, and old compose configuration wouldn't work anymore).

Two real-life examples:

• There was a major update to the smart home management system image. The automatic update of this app image led to losing control over some of the smart devices linked with it, due to massive changes in its code.
• When Traefik switched from version 2.2 to 2.3, those who had auto container updated enabled woke up with no access to their services because of the breaking changes.

So, always manually double-check compatibility between images.

You have been warned!

Be the 1 in 200,000. Help us sustain what we do.

114 / 150 by Dec 31, 2024

You will gain benefits such as discord roles, exclusive content, ad-free browsing, raffles, and more.

Become a Sponsor (starting from just $1.67/month)

Why did we start Memberships?

Watchtower Docker Compose Configuration

With all the basic info taken care of, let's see how to automate Docker container updates with Watchtower.

Requirements

As always, I assume that all the environments (docker, compose, folder structure, ...) have already been set following Anand's guides, especially for file and folder structures and permissions.

I guess you already have a working docker-compose stack. Otherwise, you wouldn't be reading this. But WHO KNOWS?!, so here are the guides anyway:

• Install Docker on Ubuntu
• Install Docker on Ubuntu Server
• Install Docker Compose on Ubuntu
• Set Docker Environment (MOST IMPORTANT)

Basic Watchtower Docker compose

By default, Watchtower monitors all the containers which are started by the same Docker daemon. Let's start with the Docker Compose for this configuration and customize it as we go.

This automates the full update process (control, pull, shutdown, restart) for all containers:

services:
  watchtower:
    container_name: watchtower
    image: containrrr/watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TZ=$TZ
      - WATCHTOWER_CLEANUP=true                                         
      - WATCHTOWER_INCLUDE_RESTARTING=true
      - WATCHTOWER_POLL_INTERVAL=36000                  
      #- WATCHTOWER_SCHEDULE=0 0 9 * * *
      #- WATCHTOWER_DEBUG=true   

If you have been following the Anand's Ultimate Docker Server series, then you would create a file called watchtower.yml in your compose folder, add the above docker compose for Watchtower, and activate it by calling the file under include block in the master Docker compose file.

Otherwise, you may just add it straight to the Docker compose file in your situation.

Environment Variables Explanation

Updating Specific Containers

If we want to add more control and choose which containers to update, then there are a few options.

First way is using the WATCHTOWER_LABEL_ENABLE=true and this requires adding 2 customizations shown below:

...
    environment:            
      - WATCHTOWER_LABEL_ENABLE=true                 
...
    labels:
      - "com.centurylinklabs.watchtower.enable=true"

We add WATCHTOWER_LABEL_ENABLE to the Watchtower's environment variable and com.centurylinklabs.watchtower.enable=true Docker label.

Then, for every container we want to auto update we need to add following Docker label to their respective Docker compose:

    labels:
      - "com.centurylinklabs.watchtower.enable=true"

Monitor Some Containers and Update Some Others

If we want to monitor some containers without updating, the Docker label we need to use is com.centurylinklabs.watchtower.monitor-only=true. :

This label only needs to be added in the compose configuration of the containers we want to keep monitored and not updated:

...
    labels:
      - "com.centurylinklabs.watchtower.monitor-only=true"

We will still receive a notification, if configured, every time a new app image is found.

All the other containers without the monitor-only label will be updated.

Alternative Configuration (No Labels)

If we're lazy, or for some reason we don't want to set a new label for every container, we could use a different approach (environment variables).

Add the following environment variable to Watchtower Docker compose:

...                     
      - WATCHTOWER_DISABLE_CONTAINERS=container1,container2
...     

With this configuration all the containers are monitored and updated, except for those listed under WATCHTOWER_DISABLE_CONTAINERS.

Alternative Configuration (No Labels for Update + Labels for Monitoring)

If you are going to mix and match labels and no-labels methods, then we need a way to set priority.

We have to add a new environment variable to the Docker Compose for Watchtower to make this possible:

...
      - WATCHTOWER_LABEL_TAKE_PRECEDENCE=true                   
      - WATCHTOWER_DISABLE_CONTAINERS=container1,container2               
...

With Watchtower, arguments (environment variables in our case) will take precedence over labels.

So, in our previous alternative configuration, the new environment variable WATCHTOWER_DISABLE_CONTAINERS, will exclude all the named containers from being monitored AND updated.

In addition, it also prevents us from monitoring, without updating, the remaining containers using the label we've seen before, because the environment variable has the precedence over it and it will be ignored.

WATCHTOWER_LABEL_TAKE_PRECEDENCE is what we need to add this granularity.

So now with the following settings we can further customize Watchtower's functionality.:

• WATCHTOWER_DISABLE_CONTAINERS to specify containers to exclude
• WATCHTOWER_LABEL_TAKE_PRECEDENCE set to true
• com.centurylinklabs.watchtower.monitor-only=true label for all containers we want to monitor-only

Setting Up Watchtower Telegram Notifications

Ok, Watchtower is now configured and we want to know when and what it's doing!
We're going to do this using Telegram notifications and Bots.

Telegram Bot

As all of you may know, Telegram is a messaging app released in 2013, that over the years has always stood out for its cool features over its competitors: channels, synced chats from multiple devices, bots, desktop client for Windows, Linux, MacOs just to name some.

In a few words, bots give us the opportunity to code our own applications and run them inside Telegram App, opening up endless new possibilities.

Actually, we're not going to code anything, we'll only use the Bot as some kind of monitor for notifications.

Creating a New Bot

For this very delicate task, we need to ask to a man bot of honor: the good old Botfather.

We could reach it from:

• Browser: https://t.me/botfather
• App: @botfather

After starting the bot with the /start message, all options, for bots, settings, web apps and games, are displayed:

We can also create a new bot by tapping on Menu and then /newbot:

Choosing a name for the newborn bot:

Choosing a username for the bot.

After that the creation process is done, the URL to reach the bot and its token needed to use HTTP API are displayed:

Note down the TOKEN. It will be used in the following steps.

Difference Between Bot's Name and Username

• The bot's name is the name displayed on the top of the chat. Many bots can have the same name.
• The username is the bot's unique identifier, must be different for every bot and must end in 'bot'.

Our bot is basically ready to work and we could customize it, adding a profile picture for example, to add a personal touch to it:

Chat ID

If not already done, we need to start a new chat with our bot.

This could be done by tapping directly on the t.me address, found in the Bothfather's chat:

Or, we could search for it, as shown couple of paragraphs above.

Press START and send a message to the bot:

Well done! That's the chat where we're going to receive all the Watchtower's notifications.

We know it, but we need to let Watchtower know too. This is done using the Chat ID (unique identifier for every chat).

To get the chat ID, visit the following link. Replace [api_token] with the TOKEN we noted previously, right after creating the bot.

https://api.telegram.org/bot[API_TOKEN]/getUpdates

For example, if our Bot API TOKEN is 0000000000:AAAaBBbbCCccCdDdDFFfFggGgGPpPPzZzz, this will be the full URL:

https://api.telegram.org/bot0000000000:AAAaBBbbCCccCdDdDFFfFggGgGPpPPzZzz/getUpdates

Paste the full URL in the address bar of your browser and note down the chat-id, as shown below:

Keep it close because we will need it in the next step to configure the notification library.

Shoutrrr Notification Library

We're going to use Shoutrrr for notifications.

Basically, this is the notification library I discussed previously. It allows us to send notifications via different services, thanks to a notification service url schema that simplifies drastically the task.

The schema we're looking for, is the one for Telegram Notifications:

The full list of services, and the respective service urls schemes can be found here.

Telegram URL Schema Explained

It's quite simple, but let's break it down anyway:

The full URL should look like:

Watchtower Docker Compose and Telegram Notifications Configuration

Hiding Telegram Notifications URL

To keep our full Telegram notifications URL safe from prying eyes, we can choose between two approaches: .env file or Docker secrets.

• .env file
• Secrets

Aligned with the approach in the Ultimate Docker Server series, we are going to provide the notification URL as a Docker Secret using the 4-step process described in our Docker guide.

First, create the secret file using:

sudo nano /home/user/docker/secrets/watcher_notifications

Add the following line to it (obviously, replace the Chat ID and the Token as described above):

telegram://0000000000:AAAaBBbbCCccCdDdDFFfFggGgGPpPPzZzz@telegram?chats=0000000

Change the permissions of the file:

sudo chown root:root /home/user/docker/secrets/watcher_notifications 
sudo chmod 600 /home/user/docker/secrets/watcher_notifications

Add the Docker secret to the Docker compose file as shown below:

secrets:
   watchtower_notifications
     file: $SECRETSDIR/watchtower_notifications

The environment variable $SECRETSDIR must be defined in .env file pointing to the path of the secrets folder:/home/user/docker/secrets.

Docker Compose Watchtower Secrets

In steps 3 and 4 of the process of adding Docker secrets, we will add the docker secret to Watchtower Docker Compose and then call it into the environmental variable as shown below:

...
    environment:
...
      - WATCHTOWER_NOTIFICATIONS=shoutrrr
      - WATCHTOWER_NOTIFICATION_URL=/run/secrets/watchtower_notifications
      # - WATCHTOWER_NOTIFICATION_TEMPLATE="{{range .}}{{.Message}}{{println}}{{end}}" 
    secrets:
      - watchtower_notifications
...

That's it! Save the Watchtower docker compose configuration file and start the new container. The Docker Compose path shown below is aligned with our Docker and Traefik guides, and Auto-Traefik. Be sure to use the actual path to your Docker compose file.

sudo docker compose -f /home/user/docker/docker-compose-HOSTNAME.yml up -d watchtower

Our first notification from Watchtower should arrive as soon as the docker is up and running, reporting our configuration options and the next scheduled run:

We're ready to receive all the notifications regarding our docker container base image updates on Telegram !!!

Troubleshooting

During the first few days of use, I encountered 2 problems with my Watchtower configuration:

Containers Marked for Updates Not Updated

WATCHTOWER_LABEL_ENABLE label was set to true. But some containers with the label com.centurylinklabs.watchtower.enable enabled were not being updated.

After a closer look, it was evident that all the containers that were listed before Watchtower in the Docker Compose file weren't being updated.

I solved this by placing the Watchtower Docker Compose as the first service, immediately afterwards services:, in the Docker Compose file.

I don't know if this is necessary, but it appeared to solve the problem for me.

Auto-Updating Watchtower Container

If you plan to update Watchtower itself, with automatic restart enabled, then be sure to not set a static IP to it, otherwise the restarting process will fail.

Before shutting itself down, Watchtower creates a second docker container using the same configuration options as the first isntance (static IP included) but using the new image.

This results in IP conflict and so the new container creation will fail.

BONUS TRACK - Additional Notifications Methods

We've just seen Watchtower with Telegram notifications, but if you prefer to be notified with another service, like MS Teams, IFTTT, Discord, etc..., then check this documentation.

Discord notification is shown below as an example.

Watchtower with Discord notifications

Let's take a look at how to receive Discord notifications:

FAQ

Final Thoughts

Watchtower is a great right arm to keep containers app images automatically updated. It supports many different services for notifications on what is going on, which makes it very versatile.

Another popular alterative called DIUN (Docker Image Update Notifier) is getting popular and a lot of community support.

But for now, all the features that Watchtower has to offer have served my needs long enough for me to trust it. In addition, Watchtower has many more features that weren't covered in this guide.

So, go ahead install Watchtower using Docker Compose and check it out for yourself.