Hugo CI/CD Pipeline

Table of Contents

TL;DR
#

If you just want to use it, follow the README.md instructions in the repository.

Docker compose for an automated hugo static website generator setup.

What for?
#

Creating a website and adding new content to it should be an easy and straightforward process. So just automate it, right?
I searched the web for already available pipeline setups but couldn’t find anything that fit my needs. This was the perfect opportunity to learn about Docker, webhooks, and of course Hugo.

Let’s make it happen
#

As a hobbyist in programming and other technical fields, I like to keep everything under version control. If something goes wrong or doesn’t work the way I want it to, I can simply revert to an earlier version. Also, documenting with Markdown files is an easy way to get “pretty” formatting.

The classic approach is to use something like WordPress, but that uses a lot of unnecessary resources for a static website. So I discovered Hugo, which is a static site generator from Markdown, exactly what I was looking for! Text-based files are great for version control with Git.

The plan
#

flowchart TB
    A["Git Push"] --> X["Remote\nGit Server"]
    X --> B["POST Webhook"]
    C <-. secret .-> B
    D <-. Deploy Key .-> X
    B -- on push--> C["Webhook"]
    subgraph " "
        C --> D["Git fetch/pull"]
        D --> E["Build hugo\nstatic files"]
    end
    E --> F["Serve static files"]

It’s coming to life
#

I want to work with Docker because I think it is the easiest way to set up the whole pipeline. I’m using docker-compose because I want the deployer (webhook, Hugo build, and Git fetch/pull) and the web server to run in separate containers.

Project structure
hugo-auto-deployer
├── docker-compose.yml
├── deployer
│   ├── Dockerfile
│   ├── deploy.sh
│   ├── entrypoint.sh
│   └── hooks.json.tpl
├── .env
└── nginx
    └── nginx.conf

docker-compose.yml
 1services:
 2
ssh-init:
  image: alpine:latest
  container_name: hugo-ssh-init
  volumes:
    - ssh-keys:/root/.ssh
  command: >
    sh -c "
      apk add --no-cache openssh-keygen > /dev/null 2>&1 &&
      mkdir -p /root/.ssh &&
      chmod 700 /root/.ssh &&
      if [ ! -f /root/.ssh/id_ed25519 ]; then
        ssh-keygen -t ed25519 -C 'hugo-deployer' -f /root/.ssh/id_ed25519 -N '' &&
        chmod 600 /root/.ssh/id_ed25519 &&
        chmod 644 /root/.ssh/id_ed25519.pub &&
        echo '' &&
        echo '════════════════════════════════════════════════════════' &&
        echo ' Generated SSH deploy key.' &&
        echo ' Please add following public key:' &&
        echo '════════════════════════════════════════════════════════' &&
        cat /root/.ssh/id_ed25519.pub &&
        echo '════════════════════════════════════════════════════════'
      else
        echo 'SSH Key already present. Skip generating. Public Key:' &&
        cat /root/.ssh/id_ed25519.pub
      fi
    "
  restart: "no"
30
deployer:
  build:
    context: ./deployer
    args:
      HUGO_VERSION: ${HUGO_VERSION:-0.159.1}
  container_name: hugo-deployer
  restart: unless-stopped
  depends_on:
    ssh-init:
      condition: service_completed_successfully
  environment:
    GIT_REPO_URL:      ${GIT_REPO_URL}
    GIT_BRANCH:        ${GIT_BRANCH:-main}
    WEBHOOK_SECRET:    ${WEBHOOK_SECRET}
    HUGO_EXTRA_FLAGS:  ${HUGO_EXTRA_FLAGS:-}
    TZ:                ${TZ:-Europe/Vienna}
47
  volumes:
    - ssh-keys:/root/.ssh
    - hugo-public:/public
51
  ports:
    - "${WEBHOOK_PORT:-9000}:9000"
  networks:
    - internal
56
nginx:
  image: nginx:alpine
  container_name: hugo-nginx
  restart: unless-stopped
  volumes:
    - hugo-public:/usr/share/nginx/html:ro
    - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:rw
  ports:
    - "${HTTP_PORT:-80}:80"
  networks:
    - internal
  depends_on:
    - deployer
  healthcheck:
    test: ["CMD-SHELL", "wget -qSO/dev/null http://127.0.0.1/ 2>&1 | grep -q 'HTTP/' || exit 1"]
    interval: 30s
    timeout: 5s
    retries: 3
    start_period: 15s
76
77volumes:
hugo-public:
  driver: local
ssh-keys:
  driver: local
82
83networks:
internal:
  driver: bridge

I created three services ( = containers): ssh-init, deployerand nginx.
The ssh-container is only used once at startup and gets shutdown afterwards. It is needed to configure the ssh-key for fetching/pulling the git repo. The container generates a (public) ssh-key that can be copied from the log file.
Brains of the whole pipeline is the deployer-container, which is a custom container. Its functions and operations are going to be discussed later. The ssh-container and deployer share a common volume to exchange the ssh-keys.
And at last there is the webserver running on nginx. It shares a volume with the deployer-container in which the web-page files are contained.

Dockerfile
 1FROM alpine:latest
 2
 3ARG HUGO_VERSION=0.159.1
 4
 5RUN apk add --no-cache \
 6    git \
 7    openssh-client \
 8    ca-certificates \
 9    wget \
10    tar \
11    gettext \
12    su-exec \
13    tzdata \
14    webhook \
15    gcompat \
16    libstdc++
17
18ENV TZ=Europe/Vienna
19
20RUN ARCH=$(uname -m) && \
21    case "$ARCH" in \
22      x86_64)  HUGO_ARCH="amd64" ;; \
23      aarch64) HUGO_ARCH="arm64" ;; \
24      armv7l)  HUGO_ARCH="arm"   ;; \
25      *) echo "Unsupported arch: $ARCH" && exit 1 ;; \
26    esac && \
27    URL="https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_linux-${HUGO_ARCH}.tar.gz" && \
28    echo "Downloading Hugo ${HUGO_VERSION} from: ${URL}" && \
29    wget -O /tmp/hugo.tar.gz "${URL}" \
30      || { echo "ERROR: Hugo-Download"; exit 1; } && \
31    mkdir -p /tmp/hugo-extract && \
32    tar -xzf /tmp/hugo.tar.gz -C /tmp/hugo-extract && \
33    ls -la /tmp/hugo-extract/ && \
34    HUGO_BIN=$(find /tmp/hugo-extract -type f -name 'hugo' | head -1) && \
35    [ -n "$HUGO_BIN" ] || { echo "ERROR: hugo-Binary not in archive"; exit 1; } && \
36    mv "$HUGO_BIN" /usr/local/bin/hugo && \
37    chmod +x /usr/local/bin/hugo && \
38    rm -rf /tmp/hugo.tar.gz /tmp/hugo-extract && \
39    hugo version
40
41RUN webhook --version && hugo version
42
43WORKDIR /app
44
45COPY hooks.json.tpl  /app/hooks.json.tpl
46COPY deploy.sh       /app/deploy.sh
47COPY entrypoint.sh   /app/entrypoint.sh
48
49RUN chmod +x /app/deploy.sh /app/entrypoint.sh
50
51RUN mkdir -p /public /app/repo
52
53EXPOSE 9000
54
55ENTRYPOINT ["/app/entrypoint.sh"]

I built on top of the commonly used image alpine, a very lightweight linux distro. First we install all needed packages. Next we pull the prebuilt binary for hugo. After that a quick check if webhook and hugo a correctly installed. Then the pre-written scripts and template get copied.

entrypoint.sh
 1#!/bin/sh
 2set -eu
 3
 4LOG_PREFIX="[entrypoint]"
 5log() { echo "${LOG_PREFIX} $*"; }
 6
 7: "${GIT_REPO_URL:?  ERROR: GIT_REPO_URL not set (SSH-Format: git@host:user/repo.git)}"
 8: "${WEBHOOK_SECRET:? ERROR: WEBHOOK_SECRET not set}"
 9: "${GIT_BRANCH:=main}"
10
11if [ ! -f /root/.ssh/id_ed25519 ]; then
12  echo ""
13  echo "  ╔═════════════════════════════╗"
14  echo "  ║  ERROR: SSH-Key not found!  ║"
15  echo "  ╚═════════════════════════════╝"
16  echo ""
17  exit 1
18fi
19
20chmod 600 /root/.ssh/id_ed25519
21log "SSH-Key found: /root/.ssh/id_ed25519"
22 
23log "Generating hooks.json from template..."
24export GIT_BRANCH WEBHOOK_SECRET
25envsubst '${WEBHOOK_SECRET} ${GIT_BRANCH}' \
26  < /app/hooks.json.tpl \
27  > /app/hooks.json
28
29log "hooks.json done."
30
31log "Initating first deploy..."
32/app/deploy.sh || {
33  log "WARN: Error during first deploy. Webhook-listener will still start."
34}
35
36log "Webhook-Endpoint: POST /hooks/hugo-deploy"
37
38exec webhook \
39  -hooks /app/hooks.json \
40  -port 9000 \
41  -ip 0.0.0.0 \
42  -verbose \
43  -hotreload

A simple entrypoint script that checks for the ssh-key and sets the rigths. It also starts the webhook.

deploy.sh
 1#!/bin/sh
 2set -euo pipefail
 3
 4REPO_DIR="/app/repo"
 5PUBLIC_DIR="/public"
 6SSH_KEY="/root/.ssh/id_ed25519"
 7LOG_PREFIX="[hugo-deployer]"
 8
 9log()  { echo "${LOG_PREFIX} $(date '+%Y-%m-%d %H:%M:%S') INFO  $*"; }
10warn() { echo "${LOG_PREFIX} $(date '+%Y-%m-%d %H:%M:%S') WARN  $*" >&2; }
11err()  { echo "${LOG_PREFIX} $(date '+%Y-%m-%d %H:%M:%S') ERROR $*" >&2; exit 1; }
12
13: "${GIT_REPO_URL:?GIT_REPO_URL must be set (SSH-Format: git@host:user/repo.git)}"
14: "${GIT_BRANCH:=main}"
15
16[ -f "$SSH_KEY" ] || err "SSH-Key not found: $SSH_KEY."
17
18chmod 600 "$SSH_KEY"
19
20if [ -f /root/.ssh/known_hosts ]; then
21  log "known_hosts found"
22  export GIT_SSH_COMMAND="ssh -i ${SSH_KEY} -o StrictHostKeyChecking=yes -o BatchMode=yes"
23else
24  warn "No known_hosts"
25  export GIT_SSH_COMMAND="ssh -i ${SSH_KEY} -o StrictHostKeyChecking=accept-new -o BatchMode=yes"
26fi
27
28log "GIT_SSH_COMMAND: $GIT_SSH_COMMAND"
29
30if [ -d "${REPO_DIR}/.git" ]; then
31  log "Repository already cloned... fetching instead"
32  cd "$REPO_DIR"
33
34  git fetch origin "${GIT_BRANCH}"
35  git checkout "${GIT_BRANCH}"
36  git reset --hard "origin/${GIT_BRANCH}"
37  git submodule update --init --recursive
38
39  log "Sync done. Current commit: $(git log -1 --oneline)"
40else
41  log "Cloning ${GIT_REPO_URL} (Branch: ${GIT_BRANCH})..."
42  git clone \
43    --branch "${GIT_BRANCH}" \
44    --depth 1 \
45    --recurse-submodules \
46    "${GIT_REPO_URL}" \
47    "${REPO_DIR}"
48
49  log "Clone done. Current commit: $(git -C "${REPO_DIR}" log -1 --oneline)"
50fi
51
52cd "$REPO_DIR"
53
54log "Starting hugo-build..."
55
56mkdir -p data
57
58HASH=$(git rev-parse HEAD)
59AUTHOR=$(git log -1 --format="%an")
60DATE=$(git log -1 --format="%ad" --date=format:"%d.%m.%Y %H:%M:%S")
61SUBJECT=$(git log -1 --format="%s")
62
63cat <<EOF > data/build_info.json
64{
65  "hash": "$HASH",
66  "author": "$AUTHOR",
67  "date": "$DATE",
68  "subject": "$SUBJECT"
69}
70EOF
71
72log "Build info generated: ${HASH}"
73
74
75# shellcheck disable=SC2086
76hugo \
77  --destination "${PUBLIC_DIR}" \
78  --cleanDestinationDir \
79  ${HUGO_EXTRA_FLAGS:-}
80
81log "Hugo-Build done."
82log "Stored in ${PUBLIC_DIR}:"
83find "$PUBLIC_DIR" -maxdepth 2 -type f | head -30

The deploy script executes git fetch/pull, generate build info (page) and build hugo files.

hooks.json.tpl
 1[
{
  "id": "hugo-deploy",
  "execute-command": "/app/deploy.sh",
  "command-working-directory": "/app",
  "response-message": "Deploy triggered",
  "response-headers": [
    { "name": "Content-Type", "value": "text/plain; charset=utf-8" }
  ],
10
  "pass-environment-to-command": [
    { "source": "entire-payload",  "envname": "WEBHOOK_PAYLOAD"  },
    { "source": "header",          "name": "X-GitHub-Event",    "envname": "GIT_EVENT"      },
    { "source": "header",          "name": "X-Gitlab-Event",    "envname": "GITLAB_EVENT"   }
  ],
16
  "trigger-rule": {
    "and": [
      {
        "match": {
          "type": "payload-hmac-sha256",
          "secret": "${WEBHOOK_SECRET}",
          "parameter": {
            "source": "header",
            "name": "X-Hub-Signature-256"
          }
        }
      },
      {
        "or": [
          {
            "match": {
              "type": "value",
              "value": "refs/heads/${GIT_BRANCH}",
              "parameter": { "source": "payload", "name": "ref" }
            }
          },
          {
            "match": {
              "type": "value",
              "value": "${GIT_BRANCH}",
              "parameter": { "source": "payload", "name": "ref" }
            }
          }
        ]
      }
    ]
  }
},
50
{
  "id": "hugo-deploy-gitlab",
  "execute-command": "/app/deploy.sh",
  "command-working-directory": "/app",
  "response-message": "GitLab Deploy triggered.",
  "response-headers": [
    { "name": "Content-Type", "value": "text/plain; charset=utf-8" }
  ],
59
  "pass-environment-to-command": [
    { "source": "entire-payload", "envname": "WEBHOOK_PAYLOAD" },
    { "source": "header",         "name": "X-Gitlab-Event",   "envname": "GITLAB_EVENT" }
  ],
64
  "trigger-rule": {
    "and": [
      {
        "match": {
          "type": "value",
          "value": "${WEBHOOK_SECRET}",
          "parameter": { "source": "header", "name": "X-Gitlab-Token" }
        }
      },
      {
        "match": {
          "type": "value",
          "value": "refs/heads/${GIT_BRANCH}",
          "parameter": { "source": "payload", "name": "ref" }
        }
      }
    ]
  }
}
84]

Here is an example for your .env file:

.env
 1# git config
 2GIT_REPO_URL=git@git.example.com:user/repo.git
 3GIT_BRANCH=main
 4
 5# generate with: openssl rand -hex 32
 6WEBHOOK_SECRET=paste_your_32_char_long_secret_here
 7WEBHOOK_PORT=9000
 8
 9# hugo config (versions: https://github.com/gohugoio/hugo/releases)
10HUGO_EXTRA_FLAGS=--minify
11HUGO_VERSION=0.159.1
12
13# web server port
14HTTP_PORT=8091
15
16# timezone
17TZ=Your/Timezone

Finishing up
#

After your setup is finished, it’s about time to copy over the public ssh-key and webhook secret to your git server. Make sure your webhook is reachable from outside (the internet…) and maybe put a proxy server between for security reasons. Setting up the deploy key basically works the same on all platforms (github, gitlab, gitea, forgejo):

Open the repo settings
Open the tab “Deploy Keys”
Add deploy key: paste the public ssh-key here
Done

The webhook service can be added in a similar manner:

Open the repo settings
Open the tab “Webhooks”
Add Webhook: paste https://your.domain/hooks/hugo-deploy
Check option “on push”
Done

Author

Erik Tóth

TL;DR#

What for?#

Let’s make it happen#

The plan#

It’s coming to life#

Finishing up#