estradiol.cloud/content/posts/hugo-on-k8s-nginx/index.md

+++
title = 'Hugo on Kubernetes & NGINX'
date = 2024-02-28T15:35:46-08:00
draft = true
series = ['wtf']
categories = ['Tutorial']
tags = ['meta', 'k8s', 'flux', 'hugo']
toc = true
+++

i decided to make a website. a static one. this one. with [Hugo][hugo]. the
main reason i have for needing a website is as a vanity project, because i need
some stuff to host in a [Kubernetes][k8s] cluster i'm running. the k8s cluster
is also a vanity project.

because i don't like software, i wanted a way to deploy my site that doesn't
involve much of it. this post is about that.

## Getting Started

i built my site by following the straight-forward
_[Getting Started][hugo-started]_ guide in the Hugo documentation.

i did `hugo new site estradiol.cloud`. and then `cd estradiol.cloud; git init`.
and then i picked a ridiculous theme
["inspired by terminal ricing aesthetics"][risotto], installing it like `git
submodule add https://github.com/joeroe/risotto.git themes/risotto; echo "theme
= 'risotto'" >> hugo.toml`.[^1]

[^1]: i appreciate the culinary branding.

at this point, my website is basically finished (i also changed the title in
`hugo.toml`). i probably won't be putting anything on it, so there's no point
fiddling with other details.

about deployment, the guide's _[Basic Usage][hugo-deploy]_ page has this to
offer:

> Most of our users deploy their sites using a CI/CD workflow, where a push{{< sup "1" >}}
> to their GitHub or GitLab repository triggers a build and deployment. Popular
> providers include AWS Amplify, CloudCannon, Cloudflare Pages, GitHub Pages,
> GitLab Pages, and Netlify.
>
> 1. The Git repository contains the entire project directory, typically excluding the
> public directory because the site is built _after_ the push.

importantly, you can't make a post about deploying this way. _everyone_ deploys
this way. if _i_ deploy this way, this site will have no content.

this approach also involves a build system somewhere that can run Hugo to
build and push the compiled code and assets onto my cluster. i definitely
already need Hugo installed on my workstation if i'm going to post
anything.[^2] so now i'm running Hugo in two places. there's surely
going to be other complex nonsense like webhooks involved.

[^2]: unlikely.

![diagram: deploy w/ GitHub Pages & Actions](images/hugo-github-pages.svg)

----

and hang on. let's look at this again:

> 1. The Git repository contains the entire project directory, typically excluding the
> public directory because the site is built _after_ the push.

you're telling me i'm going to build a nice static site and not check the
_actual content_ into version control? couldn't be me.

## Getting Static

suppose i instead check my content into git exactly as i intend to serve it?
then i could shell into my server box, pull the site, and _nifty-galifty!_ isn't
this the way it has [always been done][worm-love]?

my problem is that i don't have a server box. i have a _container orchestration
system_. there are several upsides to this[^3] but it means that _somehow_ my
generated content needs to end up in a container. because [Pods][k8s-pods] are
ephemeral and i'd like to run my site with horizontal scalability[^4], i don't
want my container to need to retain runtime state across restarts or replicas.

[^3]: few of which could be considered relevant for my project.
[^4]: i absolutely will not need this

i _could_ run a little pipeline that builds a container wrapping my content and
pushes it to a registry somewhere my deployments can pull it. all ready to go.
but now i've got _software_ again: build stages and webhooks and, to make matters
worse, now i'm hosting and versioning container images.

![diagram: deploy w/ container build](images/hugo-container-build.svg)

i don't want any of this. i just want to put some HTML and static assets behind a
web server.

---

instead, i'd like to deploy a popular container image from a public registry
and deliver my content to it continuously.

a minimal setup to achieve this might look like:

  - a `Pod` with:
    - an `nginx` container to serve the content;
    - a `git-pull` sidecar that loops, pulling the git content;
    - an `initContainer` to do the initial checkout;
    - an `emptyDir` volume to share between the containers.
  - a `ConfigMap` to store the nginx config.

![diagram: minimal pod/configmap setup](images/hugo-minimal-pod-setup.svg)

when a new pod comes up, the `initContainer` mounts the
[`emptyDir`][k8s-emptydir] and clones the repository into `/www`. i use
`git sparse-checkout` to avoid pulling repository contents i don't want
to serve out:

```bash
# git-clone command
git clone https://code.estradiol.cloud/tamsin/estradiol.cloud.git --no-checkout --branch trunk /tmp/www;
cd /tmp/www;
git sparse-checkout init --cone;
git sparse-checkout set public;
git checkout;
shopt -s dotglob
mv /tmp/www/* /www
```

for the sidecar, i script up my `git pull` loop:

```bash
# git-pull command
while true; do
  cd /www && git -c safe.directory=/www pull origin trunk
  sleep 60
done
```

and i create a [ConfigMap][k8s-configmap] with a server block to configure
`nginx` to use Hugo's `public/` as root:


```txt
# ConfigMap; data: default.conf
server {
  listen 80;
  location / {
    root   /www/public;
    index  index.html;
  }
}
```

the rest of this is pretty much boilerplate:

{{< code-details summary="`kubectl apply -f estradiol-cloud.yaml`" lang="yaml" details=`
# estradiol-cloud.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/instance: estradiol-cloud
    app.kubernetes.io/name: nginx
  name: nginx-server-block
data:
  default.conf: |-
    server {
      listen 80;
      location / {
        root   /www/public;
        index  index.html;
      }
    }
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app.kubernetes.io/instance: estradiol-cloud
    app.kubernetes.io/name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.25.4
    ports:
    - containerPort: 80
    volumeMounts:
    - mountPath: /www
      name: www
    - mountPath: /etc/nginx/conf.d
      name: nginx-server-block
  - name: git-pull
    image: bitnami/git
    command:
    - /bin/bash
    - -ec
    - |
      while true; do
        cd /www && git -c safe.directory=/www pull origin trunk
        sleep 60
      done
    volumeMounts:
    - mountPath: /www
      name: www
  initContainers:
  - name: git-clone
    image: bitnami/git
    command:
    - /bin/bash
    - -c
    - |
      shopt -s dotglob
      git clone https://code.estradiol.cloud/tamsin/estradiol.cloud.git --no-checkout --branch trunk /tmp/www;
      cd /tmp/www;
      git sparse-checkout init --cone;
      git sparse-checkout set public;
      git checkout;
      mv /tmp/www/* /www
    volumeMounts:
    - mountPath: /www
      name: www
  volumes:
  - name: www
    emptyDir: {}
  - name: nginx-server-block
    configMap:
      name: nginx-server-block
` >}}

---

my Hugo workflow now looks like:

  1. make changes to source;
  1. run `hugo --gc --minify`;[^7]
  1. commit & push.

my `git pull` control loop takes things over from here and i'm on easy street.

[^7]: i added `disableHTML = true` and `disableXML = true` to `[minify]`
configuration in `hugo.toml` to keep HTML and RSS diffs readable.


## Getting Web

this is going great! my Pod is running. it's serving out my code. i get
continuous deployment™ for the low price of 11 lines `bash`. i mean...
no one can actually browse to my website[^8] but that will be an easy fix,
right?

[^8]: i can check that its working, at least, with a [port-forward][k8s-port].

first, i need a [`Service`][k8s-svc]. this gives me a proxy with service
discovery. TK: what is this really?

{{< code-details summary="kubectl apply -f service.yaml" lang="yaml" details=`
# service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: estradiol-cloud
    app.kubernetes.io/name: nginx
  name: nginx
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/instance: estradiol-cloud
    app.kubernetes.io/name: nginx
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
` >}}


and i need an [`Ingress`][k8s-ingress] to handle traffic inbound to the cluster
and direct it to the `Service`:

{{< code-details summary="kubectl apply -f ingress.yaml" lang="yaml" details=`
# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app.kubernetes.io/instance: estradiol-cloud
    app.kubernetes.io/name: nginx
  name: nginx
spec:
  rules:
  - host: estradiol.cloud
    http:
      paths:
      - backend:
          service:
            name: nginx
            port:
              name: http
        path: /
        pathType: Prefix
` >}}

TK: wtf? Ingress controller

---

as this has come together, i've gotten increasinly anxious about how much
YAML i've had to write. this is a problem because YAML is software and, as
we've established, i'm hoping not to have much of that. it's also annoying
that most of this YAML really is boilerplate.

conveniently, [Bitnami][bitnami] maintains a [Helm][helm] Chart that hides all
the boilerplate and does exactly what we've just been doing manually.[^9]

[^9]: what incredible luck! (obviously, until now i've been working backward from this chart)

TK: install helm
TK: pull bitnami chart
TK: helm values

{{< code-details summary="`helm upgrade --install --create-namespace --namespace estradiol-cloud -f values.yaml`" lang="yaml" details=`
# values.yaml
cloneStaticSiteFromGit:
  enabled: true
  repository: "https://code.estradiol.cloud/tamsin/estradiol.cloud.git"
  branch: trunk
  gitClone:
    command:
    - /bin/bash
    - -ec
    - |
      [[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
      git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
     [[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
ingress:
  enabled: true
  hostname: estradiol.cloud
  ingressClassName: nginx
  tls: true
  annotations: {
    cert-manager.io/cluster-issuer: letsencrypt-prod
  }
serverBlock: |-
  server {
    listen 8080;
    root /app/public;
    index index.html;
  }
service:
  type: ClusterIP
`>}}

![diagram: helm setup](images/hugo-helm-setup.svg)

## Getting Flux'd

by this point i'm pretty `git push`-pilled and i'm thinking i don't much like
having this `helm` client software installed on my laptop. plus, i still have
some YAML and it's not really great that i'm storing it in flat files and
pushing it to my cluster manually. i love automation. i might love automation
more than i disdain software. i feel prepared to get some software if it
will get this yaml out of my shell history and into a git repo.


TK: Flux

![diagram: flux git push/deploy sequence](images/flux-seq.svg)

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: bitnami
  namespace: default
spec:
  url: https://charts.bitnami.com/bitnami
```

{{< code-details summary="`release.yaml`" lang="yaml" details=`
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: web
  namespace: estradiol-cloud
spec:
  interval: 5m
  chart:
    spec:
      chart: nginx
      version: '15.12.2'
      sourceRef:
        kind: HelmRepository
        name: bitnami
        namespace: default
      interval: 1m
  values:
    cloneStaticSiteFromGit:
      enabled: true
      repository: "https://code.estradiol.cloud/tamsin/estradiol.cloud.git"
      branch: trunk
      gitClone:
        command:
          - /bin/bash
          - -ec
          - |
            [[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
            git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
            [[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
    ingress:
      enabled: true
      hostname: estradiol.cloud
      ingressClassName: nginx
      tls: true
      annotations: {
        cert-manager.io/cluster-issuer: letsencrypt-prod
      }
    serverBlock: |-
      server {
        listen 8080;
        root /app/public;
        index index.html;
      }
    service:
      type: ClusterIP
`>}}

## A Note About Software

at this point i'm forced to admit there's still a lot of software involved in this.
setting aside the stuff that provisions and scales my cluster nodes, i have:

  - `nginx` (running from a stock image);
  - `git` & `bash` (running from a stock image);
  - a remote git server (i'm running `gitea`[^8], but github dot com is fine here);
  - Kubernetes (oops!);
    - `fluxcd`, especially `kustomize-controller` and `helm-controller`;
    - `nginx-ingress` controller;
  - the `bitnami/nginx` Helm chart;

[^8]: because i'm running `gitea` in my cluster and i want to avoid a circular
dependency for my `flux` source repository, i also depend on GitLab dot com.

i get to maintain my two `bash` scripts for `git-clone` and `git-pull`, my
NGINX config, and a couple of blobs of YAML.

at least there are no webhooks.

---

_fin_


[bitnami]:      https://bitnami.com/
[cert-mgr]:     https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
[helm]:         https://helm.sh
[hugo]:         https://gohugo.io
[hugo-deploy]:  https://gohugo.io/getting-started/usage/#deploy-your-site
[hugo-started]: https://gohugo.io/getting-started
[k8s]:          https://kubernetes.io
[k8s-emptydir]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
[k8s-init]:     https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
[k8s-ingress]:  https://kubernetes.io/docs/concepts/services-networking/ingress/
[k8s-pods]:     https://kubernetes.io/docs/concepts/workloads/pods/
[k8s-port]:     https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
[k8s-pv]:       https://kubernetes.io/docs/concepts/storage/persistent-volumes/
[k8s-svc]:      https://kubernetes.io/docs/concepts/services-networking/service/
[risotto]:      https://github.com/joeroe/risotto
[worm-love]:    https://www.mikecurato.com/worm-loves-worm