alpine-buildmaster

MultiArch Alpine Linux + S6 + Python3 + Buildbot (Master)

This image serves as the base container for running a Buildbot master instance to build applications, run periodic tasks, and other automations. Checkout their docs for configurations or task definitions. Also, checkout the alpine-buildworker image for running a standalone worker as a service.

Based on Alpine Linux (v3.22) from the python3 image with the buildbot (Source) package(s) installed in it.

Get the Image¶

Docker Hub

Pull the image from Docker Hub.


docker pull woahbase/alpine-buildmaster

Image Tags

The image is tagged respectively for the following architectures,

latest tag is annotated as multiarch so pulling without specifying any architecture tags should fetch the correct image for your architecture. Same goes for any of the version tags.

non-x86_64 images used to contain the embedded qemu-user-static binary which has been redundant for a while, and is being deprecated starting with our Alpine Linux v3.22 base-image release, see qemu-user-static or the more recent binfmt instead for running multi-arch containers.

Run¶

Running the container starts the service.


docker run --rm \
  --name docker_buildmaster \
  -c 256 -m 256m \
  -p 8010:8010 \
  -p 9989:9989 \
  -p 9990:9990 \
  -p 9991:9991 \
  -v $PWD/buildmaster`#
`:/home/alpine/buildbot \
woahbase/alpine-buildmaster

Multi-Arch Support

If you want to run images built for other architectures on the same host (e.g. a x86_64 machine), you will need to have the specific binary format support configured on your host machine before running the image (otherwise you get an exec format error). Here's how,

binfmtqemu-user-static

For recent images, we can use tonistiigi's binfmt image to register binary execution support for the target architecture, like the following,


docker run --rm --privileged tonistiigi/binfmt --install <architecture>

Architecture is that of the image we're trying to run, can be arm64 for aarch64, arm for both armv7l and armhf, or amd64 for x86_64. See binfmt.

Previously, multiarch, had made it easy for us by packing qemu into an image, so we could just run


docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

However, that image (see qemu-user-static) seems to have fallen behind in updates, and with newer images the binfmt method is preferable.

Now images built for other architectures will also be executable. This is optional though, without the above, you can still run the image that is specifically made to support your host architecture.

Configuration¶

We can customize the runtime behaviour of the container with the following environment variables.

ENV Vars	Default	Description
BUILDBOT_HOME	/home/alpine/buildbot	(Preset) Default root directory for buildbot configurations.
BUILDBOT_PROJECTNAME	buildbot	Project name that is prepended to the master name, e.g. default is `buildbot`-master.
BUILDBOT_SETUP_ARGS	--force --log-count=2 --log-size=5000 --relocatable	These arguments are passed to setup the master.
BUILDBOT_SKIP_SETUP	unset	Skips master setup tasks, useful when your already have your configurations setup, or would like to do it manually.
BUILDBOT_MASTERNAME	${BUILDBOT_PROJECTNAME}-master	Name of the service, defaults to `projectname-rolename`.
BUILDBOT_MASTERCFG	${BUILDBOT_MASTERNAME}/master.cfg.sample	Path to custom configuration file that is copied into place as `master.cfg` before starting service.
BUILDBOT_USE_CUSTOM_TACFILE	unset	Whether to use custom tacfile provided in the image that logs to `stdout` by default, set to a non-empty string (e.g `1`) to enable, or use the one generated by package.
BUILDBOT_CUSTOM_TACFILE	/defaults/master.tac	Customizable path to tacfile provided in the image.
BUILDBOT_BASEDIR	unset	Used in the custom tacfile to determine where builder files are stored, defaults to "." when unset (current directory where `buildbot.tac` exists), or any other directory (must exist).
BUILDBOT_LOGDEST	stdout	Used in the custom tacfile to determine where logs are sent, can be either of `stdout` (default), `syslog`, or `file`.
BUILDBOT_LOGROTATE_LENGTH	5000	Used in the custom tacfile to determine maximum lines-in-logfile before it is rotated.
BUILDBOT_LOGROTATE_MAXFILES	2	Used in the custom tacfile to determine maximum rotated logfiles that are kept in storage.
BUILDBOT_CONFIG_URL	unset	If set, tries to fetch configuration from remote location. Can be a `tar.gz` file, or a git repository, or a file link to `master.cfg` itself.
BUILDBOT_CONFIG_DIR	${BUILDBOT_HOME}/config	Configurations fetched from remote location unpacked in this directory.
BUILDBOT_CONFIG_TMP	${BUILDBOT_HOME}/.tmp	Configurations fetched from remote location are temporarily stored in this directory. (Useful if `/tmp` is non-writable or limited space)
BUILDBOT_CONFIG_CFGFILE	master.cfg	Customizable path to master.cfg as relative to downloaded configurations. Default expects at the root of directory, if it exists, it is copied into place as `master.cfg` before starting service.
BUILDBOT_CONFIG_TACFILE	buildbot.tac	Customizable path to buildbot.tac as relative to downloaded configurations. Default expects at the root of directory, if it exists, it is copied into place as `buildbot.tac` before starting service.
BUILDBOT_SKIP_CHECKCONFIG	unset	If `true`. skips checking configurations before starting service.
BUILDBOT_UPGRADE_MASTER	unset	If `true`, upgrades master database before starting service.
BUILDBOT_CLEANUP_DB	unset	If `true`, runs cleanup tasks on master database before starting service.
BUILDBOT_WORKERNAME	${BUILDBOT_PROJECTNAME}-worker	Name of the default worker updated in `master.cfg`. (Only when using the sample configurations)
BUILDBOT_WORKERPASS	insecurebydefault	Password of the default worker updated in `master.cfg`. (Only when using the sample configurations)
BUILDBOT_SKIP_CUSTOMIZE	unset	Skip post-setup customization tasks.
BUILDBOT_SKIP_PERMFIX	unset	If set to a non-empty-string value (e.g. `1`), skip ensuring files in `${BUILDBOT_HOME}` are owned by `${S6_USER}`, enabled by default.
BUILDBOT_PIDFILE	twistd.pid	If PID file exists from last run, it is removed before starting the service.
BUILDBOT_TACFILE	buildbot.tac	(Preset) The `tacfile` to use for starting `buildbot` service.
BUILDBOT_ARGS	--nodaemon --no_save	Customizable arguments passed to master service. (Runs as a `twisted` application instead of calling `buildbot` executable)
S6_PIP_PACKAGES	empty string	Space-separated list of packages to install globally with `pip`.
S6_PIP_REQUIREMENTS	empty string	Path to `requirements.txt` to install globally with `pip`.
S6_PIP_USER_PACKAGES	empty string	Space-separated list of packages to install with `pip` for `${S6_USER}`. These are installed in `~/.local/`.
S6_PIP_USER_REQUIREMENTS	empty string	Path to `requirements.txt` to install with `pip` for `${S6_USER}`.
S6_NEEDED_PACKAGES	empty string	Space-separated list of extra APK packages to install on start. E.g. `"curl git tzdata"`
PUID	1000	Id of `${S6_USER}`.
PGID	1000	Group id of `${S6_USER}`.
S6_USER	alpine	(Preset) Default non-root user for services to drop privileges to.
S6_USERHOME	/home/alpine	(Preset) HOME directory for `${S6_USER}`.

Did you know?

You can check your own UID/GID by running the command id in a terminal.

Also,

The env variable ${BUILDBOT_ROLE} determines if you are running a master or worker. This also determines what image you'll be running when used with the makefile. It is baked into the image so does not need to be changed unless you know what you're doing.
Setup tasks are only run when the buildbot.tac file does not exist or ${BUILDBOT_SKIP_SETUP} is not set. Same goes for arguments / environment variables specific to setup, they are not needed anymore after setup is complete.
However, if ${BUILDBOT_MASTERCFG} is defined, it will always be copied into project before starting the service. This is useful for large projects that maintain the configurations separately from master configurations.
Includes a placeholder script for further customizations before starting processes. Override the shellscript located at /etc/s6-overlay/s6-rc.d/p22-buildbot-customize/run with your custom pre-tasks as needed.
The service does not run buildbot for master, instead calls twistd directly, pass ${BUILDBOT_ARGS} accordingly.
For the master node, mount the configurations at the ${BUILDBOT_HOME} directory inside the container, by default it is /home/alpine/buildbot.

Stop the container with a timeout, (defaults to 2 seconds)


docker stop -t 2 docker_buildmaster

Restart the container with


docker restart docker_buildmaster

Removes the container, (always better to stop it first and -f only when needed most)


docker rm -f docker_buildmaster

Shell access¶

Get a shell inside a already running container,


docker exec -it docker_buildmaster /bin/bash

Optionally, login as a non-root user, (default is alpine)


docker exec -u alpine -it docker_buildmaster /bin/bash

Or set user/group id e.g 1000/1000,


docker exec -u 1000:1000 -it docker_buildmaster /bin/bash

Logs¶

To check logs of a running container in real time


docker logs -f docker_buildmaster

As-A-Service¶

Run the container as a service with the following as reference (and modify it as needed).

composenomad

With docker-compose (alpine-buildmaster.yml)


---
services:
  buildmaster:
    container_name: buildmaster
    # depends_on:
    #   mysql:
    #     condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: '2.00'
          memory: 1024M
      restart_policy:
        condition: on-failure
        delay: 10s
        max_attempts: 5
        window: 120s
    environment:
      BUILDBOT_PROJECTNAME: ${BUILDMASTER_PROJECTNAME:-buildbot}
      # BUILDBOT_MASTERCFG: /custom/master.cfg
      # BUILDBOT_USE_CUSTOM_TACFILE: 1
      # BUILDBOT_SKIP_SETUP: 1
      # BUILDBOT_SKIP_PERMFIX: 1

      # # configurations from remote source
      # BUILDBOT_CONFIG_URL=https://github.com/buildbot/buildbot/releases/download/v3.11.9/buildbot-3.11.9.tar.gz
      # BUILDBOT_CONFIG_CFGFILE=buildbot/scripts/sample.cfg
      # # or
      # BUILDBOT_CONFIG_URL=https://github.com/buildbot/buildbot.git
      # BUILDBOT_CONFIG_CFGFILE=master/docker-example/master.cfg
      # BUILDBOT_CONFIG_TACFILE=master/contrib/docker/master/buildbot.tac
      # # or
      # BUILDBOT_CONFIG_URL=https://raw.githubusercontent.com/buildbot/buildbot/refs/heads/master/master/docker-example/master.cfg

      BUILDBOT_SKIP_CHECKCONFIG: ${BUILDMASTER_SKIP_CHECKCONFIG:-false}
      BUILDBOT_UPGRADE_MASTER: ${BUILDMASTER_UPGRADE_MASTER:-false}
      BUILDBOT_CLEANUP_DB: ${BUILDMASTER_CLEANUP_DB:-false}

      PUID: ${PUID}
      PGID: ${PGID}
      # TZ: ${TZ}
    # healthcheck:
    #   interval: 2m
    #   retries: 5
    #   start_period: 5m
    #   test:
    #     - CMD-SHELL
    #     - >
    #       wget --quiet --tries=1 --no-check-certificate --spider http://localhost:8010/ || exit 1
    #   timeout: 10s
    hostname: buildmaster
    image: woahbase/alpine-buildmaster:${BUILDMASTER_TAG:-latest}
    network_mode: bridge
    ports:
      - protocol: tcp
        host_ip: 0.0.0.0
        published: ${BUILDMASTER_WEB_PORT:-8010}
        target: 8010
      - protocol: tcp
        host_ip: 0.0.0.0
        published: ${BUILDMASTER_PB_PORT:-9989}
        target: 9989
      - protocol: tcp
        host_ip: 0.0.0.0
        published: ${BUILDMASTER_CLI_PORT:-9990}
        target: 9990
      - protocol: tcp
        host_ip: 0.0.0.0
        published: ${BUILDMASTER_METRICS_PORT:-9991}
        target: 9991
    volumes:
      - type: bind
        source: ${BUILDMASTER_DIR:?err}
        target: /home/alpine/buildbot
        bind:
          create_host_path: false
      # # pass your own master.cfg
      # # gets copied to proper place before service start
      # - type: bind
      #   source: ${BUILDMASTER_DIR:?err}/custom_config/master.cfg
      #   target: /custom/master.cfg
      #   bind:
      #     create_host_path: false
      - type: bind
        source: /etc/localtime
        target: /etc/localtime
        read_only: true
        bind:
          create_host_path: false

With HashiCorp Nomad (alpine-buildmaster.hcl)


variables {
  dc   = "dc1" # to load the dc-local config file
  pgid = 1000  # gid for docker
  puid = 1000  # uid for docker
  version = "3.11.9"
}
# locals { var = yamldecode(file("${var.dc}.vars.yml")) } # load dc-local config file

job "buildmaster" {
  datacenters = [var.dc]
  # namespace   = local.var.namespace
  priority    = 70
  # region      = local.var.region
  type        = "service"

  constraint { distinct_hosts = true }

  group "docker" {
    count = 1

    restart {
      attempts = 2
      interval = "2m"
      delay    = "15s"
      mode     = "fail"
    }
    update {
      max_parallel     = 1
      min_healthy_time = "10s"
      healthy_deadline = "3m"
      auto_revert      = false
    }

    service {
      name        = NOMAD_JOB_NAME
      port        = "http"
      tags        = ["ins${NOMAD_ALLOC_INDEX}", attr.unique.hostname, "urlprefix-/buildbot"]
      canary_tags = ["canary${NOMAD_ALLOC_INDEX}", "urlprefix-/c/buildbot"]
      check {
        name      = "${NOMAD_JOB_NAME}@${attr.unique.hostname}:${NOMAD_HOST_PORT_http}"
        type      = "http"
        path      = "/"
        interval  = "60s"
        timeout   = "10s"
      }
      check_restart {
        limit     = 3
        grace     = "10s"
      }
    }
    service {
      name        = "${NOMAD_JOB_NAME}-pb"
      port        = "pb"
      tags        = ["ins${NOMAD_ALLOC_INDEX}", attr.unique.hostname, "host=build", "proto=tcp"]
      canary_tags = ["canary${NOMAD_ALLOC_INDEX}"]
      check {
        name      = "${NOMAD_JOB_NAME}@${attr.unique.hostname}:${NOMAD_HOST_PORT_pb}"
        type      = "tcp"
        interval  = "60s"
        timeout   = "10s"
      }
      check_restart {
        limit     = 3
        grace     = "10s"
      }
    }

    ephemeral_disk { size = 128 } # MB
    network {
      # dns { servers = local.var.dns_servers }
      port "http"    { static = 8010 }
      port "pb"      { static = 9989 }
      port "cli"     { static = 9990 }
      port "metrics" { static = 9991 }
    }
    volume "nomad-buildmaster-project" {
      type      = "host"
      read_only = false
      source    = "nomad-buildmaster-project"
    }

    task "buildmaster" {
      driver = "docker"

      config {
        healthchecks { disable = true }
        hostname     = NOMAD_JOB_NAME
        image        = "woahbase/alpine-buildmaster:${var.version}"
        network_mode = "bridge"
        ports        = ["http", "pb", "cli", "metrics"]

        logging {
          type = "journald"
          config {
            mode = "non-blocking"
            tag = NOMAD_JOB_NAME
          }
        }

        # mount {
        #   source   = "local/buildbot.tac"
        #   target   = "/home/alpine/buildbot/buildbot-master/buildbot.tac"
        #   type     = "bind"
        #   readonly = false
        # }

        # mount {
        #   source   = "local/master.cfg"
        #   target   = "/custom/master.cfg"
        #   type     = "bind"
        #   readonly = false
        # }

        mount {
          type     = "bind"
          target   = "/etc/localtime"
          source   = "/etc/localtime"
          readonly = true
        }
      }

      volume_mount {
        # ensure policies allow vault-generated-token to read-write to the volume
        volume      = "nomad-buildmaster-project"
        destination = "/home/alpine/buildbot"
        read_only   = false
      }

      env {
        BUILDBOT_PROJECTNAME = "buildbot"
        # BUILDBOT_MASTERCFG = "/custom/master.cfg"
        # BUILDBOT_USE_CUSTOM_TACFILE = "1"
        # BUILDBOT_SKIP_SETUP = "1"

        # # configurations from remote source
        # BUILDBOT_CONFIG_URL = "https://github.com/buildbot/buildbot/releases/download/v3.11.9/buildbot-3.11.9.tar.gz"
        # BUILDBOT_CONFIG_CFGFILE = "buildbot/scripts/sample.cfg"
        # # or
        # BUILDBOT_CONFIG_URL = "https://github.com/buildbot/buildbot.git"
        # BUILDBOT_CONFIG_CFGFILE = "master/docker-example/master.cfg"
        # BUILDBOT_CONFIG_TACFILE = "master/contrib/docker/master/buildbot.tac"
        # # or
        # BUILDBOT_CONFIG_URL = "https://raw.githubusercontent.com/buildbot/buildbot/refs/heads/master/master/docker-example/master.cfg"

        BUILDBOT_SKIP_CHECKCONFIG = "true"
        BUILDBOT_UPGRADE_MASTER = "false"
        BUILDBOT_CLEANUP_DB = "false"

        PGID = var.pgid
        PUID = var.puid
        # TZ   = local.var.tz
      }

      resources {
        cpu    = 1000 # MHz
        memory = 1024 # MB
      }

      # template {
      #   destination = "local/buildbot.tac"
      #   data        = <<-EOC
      #     {{ key "nomad/${var.dc}/buildbot/master/buildbot.tac" }}
      #   EOC
      #   change_mode = "restart"
      #   perms       = "644"
      #   error_on_missing_key = true
      # }

      # template {
      #   destination = "local/master.cfg"
      #   data        = <<-EOC
      #     {{ key "nomad/${var.dc}/buildbot/master/master.cfg" }}
      #   EOC
      #   change_mode = "restart"
      #   perms       = "644"
      #   error_on_missing_key = true
      # }
    }

    task "await-service-mysql" {
      driver = "raw_exec"
      # user   = local.var.exec_user

      config {
        command = "bash"
        args    = [ "-c", <<-EOS
          echo -n Waiting for $SVC_MYSQL.; \
          until \
            drill $SVC_MYSQL 2>/dev/null | grep -q 'rcode: NOERROR'; \
            do echo -n ' .'; sleep $WAIT_SEC; done; \
          echo 'Available.'; \
        EOS
        ]
      }

      env {
        SVC_MYSQL = "mysql.service.${var.dc}.consul"
        WAIT_SEC  = 10
      }

      lifecycle {
        hook    = "prestart"
        sidecar = false
      }
    }
  }
}

Reverse Proxy¶

To proxy it through a web server, see below

NGINX

This snippet can be used to reverse-proxy the service using NGINX.


upstream proxy_buildmaster {
    server your.host.local:<buildmaster-port> fail_timeout=5;
}

## the following goes inside a server block

location /buildbot/ {
    proxy_pass http://proxy_buildbot/;
}

location /buildbot/sse/ {
    # proxy buffering will prevent sse to work
    proxy_buffering off;
    proxy_pass http://proxy_buildbot/sse/;
}

# required for websocket
location /buildbot/ws {
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    # raise the proxy timeout for the websocket
    proxy_read_timeout 6000s;

    proxy_pass http://proxy_buildbot/ws;
}

Build Your Own¶

Feel free to clone (or fork) the repository and customize it for your own usage, build the image for yourself on your own systems, and optionally, push it to your own public (or private) repository.

Here's how...

Setting up¶

Before we clone the /repository, we must have Git, GNU make, and Docker (optionally, with buildx plugin for multi-platform images) setup on the machine. Also, for multi-platform annotations, we might require enabling experimental features of Docker.

Now, to get the code,

Github

Clone the repository with,


git clone https://github.com/woahbase/alpine-buildbot
cd alpine-buildbot

To get a list of all available targets, run


make help

Always Check Before You Make!

Did you know, we could check what any make target is going to execute before we actually run them, with


make -n <targetname> <optional args>

Build and Test¶

To create the image for your architecture, run the build and test target with


make build test ROLE=master

For building an image that targets another architecture, it is required to specify the ARCH parameter when building. e.g.

aarch64armhfarmv7li386ppc64leriscv64s390xx86_64


make build test ARCH=aarch64 ROLE=master


make build test ARCH=armhf ROLE=master


make build test ARCH=armv7l ROLE=master


make build test ARCH=i386 ROLE=master


make build test ARCH=ppc64le ROLE=master


make build test ARCH=riscv64 ROLE=master


make build test ARCH=s390x ROLE=master


make build test ARCH=x86_64 ROLE=master

Build Parameters

All images have a few common build parameters that can be customized at build time, like

ARCH

The target architecture to build for. Defaults to host architecture, auto-detected at build-time if not specified. Also determines if binfmt support is required before build or run and runs the regbinfmt (or inbinfmt for recent images) target automatically. Possible values could be aarch64, armhf, armv7l, or x86_64.

BUILDDATE

The date of the build. Can be used to create separate tags for images. (format: yyyymmdd)

DOCKERFILE

The dockerfile to use for build. Defaults to the file Dockerfile, but if per-arch dockerfiles exist, (e.g. for x86_64 the filename would be Dockerfile_x86_64) that is used instead.

TESTCMD

The command to run for testing the image after build. Runs in a bash shell.

VERSION

The version of the app/tool, may need to be preset before starting the build (e.g. for binaries from github releases), or extracted from the image after build (e.g. for APK or pip packages).

REGISTRY

The registry to push to, defaults to the Docker Hub Registry (docker.io) or any custom registry that is set via docker configurations. Does not need to be changed for local or test builds, but to override, either pass it by setting an environment variable, or with every make command.

ORGNAME

The organization (or user) name under which the image repositories exist, defaults to woahbase. Does not need to be changed for local or test builds, but to override, either pass it by setting an environment variable, or with every make command.

The image may also require custom parameters (like binary architecture). Before you build, check the makefile for a complete list of parameters to see what may (or may not) need to be set.

BuildX and Self-signed certificates

If you're using a private registry (a-la docker distribution server) with self-signed certificates, that fail to validate when pulling/pushing images. You will need to configure buildx to allow insecure access to the registry. This is configured via the config.toml file. A sample is provided in the repository, make sure to replace YOUR.PRIVATE.REGISTRY with your own (include port if needed).

Make to Run¶

Running the image creates a container and either starts a service (for service images) or provides a shell (can be either a root-shell or usershell) to execute commands in, depending on the image. We can run the image with


make run ROLE=master

But if we just need a root-shell in the container without any fance pre-tasks (e.g. for debug or to test something bespoke), we can run bash in the container with --entrypoint /bin/bash. This is wrapped in the makefile as


make shell ROLE=master

Nothing vs All vs Run vs Shell

By default, if make is run without any arguments, it calls the target all. In our case this is usually mapped to the target run (which in turn may be mapped to shell).

There may be more such targets defined as per the usage of the image. Check the makefile for more information.

Push the Image¶

If the build and test steps finish without any error, and we want to use the image on other machines, it is the next step push the image we built to a container image repository (like /hub), for that, run the push target with


make push ROLE=master

If the built image targets another architecture then it is required to specify the ARCH parameter when pushing. e.g.

aarch64armhfarmv7li386ppc64leriscv64s390xx86_64


make push ARCH=aarch64 ROLE=master


make push ARCH=armhf ROLE=master


make push ARCH=armv7l ROLE=master


make push ARCH=i386 ROLE=master


make push ARCH=ppc64le ROLE=master


make push ARCH=riscv64 ROLE=master


make push ARCH=s390x ROLE=master


make push ARCH=x86_64 ROLE=master

Pushing Multiple Tags

With a single make push, we are actually pushing 3 tags of the same image, e.g. for x86_64 architecture, they're namely

alpine-buildmaster:x86_64

The actual image that is built.

alpine-buildmaster:x86_64_${version}

It is expected that the application is versioned when built or packaged, it can be specified in the tag, this makes pulling an image by tag possible. Usually this is obtained from the parameter VERSION, which by default, is set by calling a function to extract the version string from the package installed in the container, or from github releases. Can be skipped with the parameter SKIP_VERSIONTAG to a non-empty string value like 1.

alpine-buildmaster:x86_64_${version}_${builddate}

When building multiple versions of the same image (e.g. for providing fixes or revisions), this ensures that a more recent push does not fully replace a previously pushed image. This way, although the architecture and version tags are replaced, it is possible to roll back to the previously built image by build date (format yyyymmdd). This value is obtained from the BUILDDATE parameter, and if not essential, can be skipped by setting the parameter SKIP_BUILDDATETAG to a non-empty string value like 1.

Pushing To A Private Registry

If you want to push the image to a custom registry that is not pre-configured on your system, you can set the REGISTRY variable either on the build environment, or as a makefile parameter, and that will be used instead of the default Docker Hub repository. Make sure to have push access set up before you actually push, and include port if needed. E.g.


export REGISTRY=your.private.registry:5000
make build test push

or


make build test push REGISTRY=your.private.registry:5000

Annotate Manifest(s)¶

For single architecture images, the above should suffice, the built image can be used in the host machine, and on other machines that have the same architecture too, i.e. after a push.

But for use-cases that need to support multiple architectures, there's a couple more things that need to be done. We need to create (or amend if already created beforehand) a manifest for the image(s) that we built, then annotate it to map the images to their respective architectures. And for our three tags created above we need to do it thrice.

Did you know?

We can inspect the manifest of any image by running


docker manifest inspect <imagename>:<optional tag, default is latest>

Tag Latest¶

Assuming we built the images for all supported architectures, to facilitate pulling the correct image for the architecture, we can create/amend the latest manifest and annotate it to map the tags :aarch64, :armhf, :armv7l, :i386, :ppc64le, :riscv64, :s390x, :x86_64 to the tag :latest by running


make annotate_latest ROLE=master

How it works

First we create or amend the manifest with the tag latest

createamend


docker manifest create \
woahbase/alpine-buildmaster:latest \
woahbase/alpine-buildmaster:aarch64 \
woahbase/alpine-buildmaster:armhf \
woahbase/alpine-buildmaster:armv7l \
woahbase/alpine-buildmaster:i386 \
woahbase/alpine-buildmaster:ppc64le \
woahbase/alpine-buildmaster:riscv64 \
woahbase/alpine-buildmaster:s390x \
woahbase/alpine-buildmaster:x86_64 \
;


docker manifest create --amend \
woahbase/alpine-buildmaster:latest \
woahbase/alpine-buildmaster:aarch64 \
woahbase/alpine-buildmaster:armhf \
woahbase/alpine-buildmaster:armv7l \
woahbase/alpine-buildmaster:i386 \
woahbase/alpine-buildmaster:ppc64le \
woahbase/alpine-buildmaster:riscv64 \
woahbase/alpine-buildmaster:s390x \
woahbase/alpine-buildmaster:x86_64 \
;

Then annotate the image for each architecture in the manifest with

aarch64armhfarmv7li386ppc64leriscv64s390xx86_64


docker manifest annotate --os linux --arch arm64 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:aarch64;


docker manifest annotate --os linux --arch arm --variant v6 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:armhf;


docker manifest annotate --os linux --arch arm --variant v7 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:armv7l;


docker manifest annotate --os linux --arch 386 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:i386;


docker manifest annotate --os linux --arch ppc64le \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:ppc64le;


docker manifest annotate --os linux --arch riscv64 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:riscv64;


docker manifest annotate --os linux --arch s390x \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:s390x;


docker manifest annotate --os linux --arch amd64 \
    woahbase/alpine-buildmaster:latest \
    woahbase/alpine-buildmaster:x86_64;

And finally, push it to the repository using


docker manifest push -p woahbase/alpine-buildmaster:latest

Tag Version¶

Next, to facilitate pulling images by version, we create/amend the image-version manifest and annotate it to map the tags :aarch64_${version}, :armhf_${version}, :armv7l_${version}, :i386_${version}, :ppc64le_${version}, :riscv64_${version}, :s390x_${version}, :x86_64_${version} to the tag :${version} by running


make annotate_version ROLE=master

How it works

First we create or amend the manifest with the tag ${version}

createamend


docker manifest create \
woahbase/alpine-buildmaster:${version} \
woahbase/alpine-buildmaster:aarch64_${version} \
woahbase/alpine-buildmaster:armhf_${version} \
woahbase/alpine-buildmaster:armv7l_${version} \
woahbase/alpine-buildmaster:i386_${version} \
woahbase/alpine-buildmaster:ppc64le_${version} \
woahbase/alpine-buildmaster:riscv64_${version} \
woahbase/alpine-buildmaster:s390x_${version} \
woahbase/alpine-buildmaster:x86_64_${version} \
;


docker manifest create --amend \
woahbase/alpine-buildmaster:${version} \
woahbase/alpine-buildmaster:aarch64_${version} \
woahbase/alpine-buildmaster:armhf_${version} \
woahbase/alpine-buildmaster:armv7l_${version} \
woahbase/alpine-buildmaster:i386_${version} \
woahbase/alpine-buildmaster:ppc64le_${version} \
woahbase/alpine-buildmaster:riscv64_${version} \
woahbase/alpine-buildmaster:s390x_${version} \
woahbase/alpine-buildmaster:x86_64_${version} \
;

Then annotate the image for each architecture in the manifest with

aarch64armhfarmv7li386ppc64leriscv64s390xx86_64


docker manifest annotate --os linux --arch arm64 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:aarch64_${version};


docker manifest annotate --os linux --arch arm --variant v6 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:armhf_${version};


docker manifest annotate --os linux --arch arm --variant v7 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:armv7l_${version};


docker manifest annotate --os linux --arch 386 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:i386_${version};


docker manifest annotate --os linux --arch ppc64le \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:ppc64le_${version};


docker manifest annotate --os linux --arch riscv64 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:riscv64_${version};


docker manifest annotate --os linux --arch s390x \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:s390x_${version};


docker manifest annotate --os linux --arch amd64 \
    woahbase/alpine-buildmaster:${version} \
    woahbase/alpine-buildmaster:x86_64_${version};

And finally, push it to the repository using


docker manifest push -p woahbase/alpine-buildmaster:${version}

Tag Build-Date¶

Then, (optionally) we create/amend the ${version}_${builddate} manifest and annotate it to map the tags :aarch64_${version}_${builddate}, :armhf_${version}_${builddate}, :armv7l_${version}_${builddate}, :i386_${version}_${builddate}, :ppc64le_${version}_${builddate}, :riscv64_${version}_${builddate}, :s390x_${version}_${builddate}, :x86_64_${version}_${builddate} to the tag :${version}_${builddate} by running


make annotate_date ROLE=master

How it works

First we create or amend the manifest with the tag ${version}_${builddate}

createamend


docker manifest create \
woahbase/alpine-buildmaster:${version}_${builddate} \
woahbase/alpine-buildmaster:aarch64_${version}_${builddate} \
woahbase/alpine-buildmaster:armhf_${version}_${builddate} \
woahbase/alpine-buildmaster:armv7l_${version}_${builddate} \
woahbase/alpine-buildmaster:i386_${version}_${builddate} \
woahbase/alpine-buildmaster:ppc64le_${version}_${builddate} \
woahbase/alpine-buildmaster:riscv64_${version}_${builddate} \
woahbase/alpine-buildmaster:s390x_${version}_${builddate} \
woahbase/alpine-buildmaster:x86_64_${version}_${builddate} \
;


docker manifest create --amend \
woahbase/alpine-buildmaster:${version}_${builddate} \
woahbase/alpine-buildmaster:aarch64_${version}_${builddate} \
woahbase/alpine-buildmaster:armhf_${version}_${builddate} \
woahbase/alpine-buildmaster:armv7l_${version}_${builddate} \
woahbase/alpine-buildmaster:i386_${version}_${builddate} \
woahbase/alpine-buildmaster:ppc64le_${version}_${builddate} \
woahbase/alpine-buildmaster:riscv64_${version}_${builddate} \
woahbase/alpine-buildmaster:s390x_${version}_${builddate} \
woahbase/alpine-buildmaster:x86_64_${version}_${builddate} \
;

Then annotate the image for each architecture in the manifest with

aarch64armhfarmv7li386ppc64leriscv64s390xx86_64


docker manifest annotate --os linux --arch arm64 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:aarch64_${version}_${builddate};


docker manifest annotate --os linux --arch arm --variant v6 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:armhf_${version}_${builddate};


docker manifest annotate --os linux --arch arm --variant v7 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:armv7l_${version}_${builddate};


docker manifest annotate --os linux --arch 386 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:i386_${version}_${builddate};


docker manifest annotate --os linux --arch ppc64le \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:ppc64le_${version}_${builddate};


docker manifest annotate --os linux --arch riscv64 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:riscv64_${version}_${builddate};


docker manifest annotate --os linux --arch s390x \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:s390x_${version}_${builddate};


docker manifest annotate --os linux --arch amd64 \
    woahbase/alpine-buildmaster:${version}_${builddate} \
    woahbase/alpine-buildmaster:x86_64_${version}_${builddate};

And finally, push it to the repository using


docker manifest push -p woahbase/alpine-buildmaster:${version}_${builddate}

That's all folks! Happy containerizing!

Maintenance¶

Sources at Github. Built and tested at home using Buildbot. Images at Docker Hub.

Maintained (or sometimes a lack thereof?) by WOAHBase.

Helped you save a bit of your valuable time and effort that is best spent doing stuff you actually enjoy? If you have some pocket-change to spare, kindly consider helping out.