469 lines
14 KiB
Markdown
469 lines
14 KiB
Markdown
+++
|
|
title = 'Hugo on Kubernetes & NGINX'
|
|
date = 2024-02-28T15:35:46-08:00
|
|
draft = true
|
|
series = ['wtf']
|
|
categories = ['Tutorial']
|
|
tags = ['meta', 'k8s', 'flux', 'hugo']
|
|
toc = true
|
|
+++
|
|
|
|
i decided to make a website. a static one. this one. with [Hugo][hugo]. the
|
|
main reason i have for needing a website is as a vanity project, because i need
|
|
some stuff to host in a [Kubernetes][k8s] cluster i'm running. the k8s cluster
|
|
is also a vanity project.
|
|
|
|
because i don't like software, i wanted a way to deploy my site that doesn't
|
|
involve much of it. this post is about that.
|
|
|
|
## Getting Started
|
|
|
|
i built my site by following the straight-forward
|
|
_[Getting Started][hugo-started]_ guide in the Hugo documentation.
|
|
|
|
i did `hugo new site estradiol.cloud`. and then `cd estradiol.cloud; git init`.
|
|
and then i picked a ridiculous theme
|
|
["inspired by terminal ricing aesthetics"][risotto], installing it like `git
|
|
submodule add https://github.com/joeroe/risotto.git themes/risotto; echo "theme
|
|
= 'risotto'" >> hugo.toml`.[^1]
|
|
|
|
[^1]: i appreciate the culinary branding.
|
|
|
|
at this point, my website is basically finished (i also changed the title in
|
|
`hugo.toml`). i probably won't be putting anything on it, so there's no point
|
|
fiddling with other details.
|
|
|
|
about deployment, the guide's _[Basic Usage][hugo-deploy]_ page has this to
|
|
offer:
|
|
|
|
> Most of our users deploy their sites using a CI/CD workflow, where a push{{< sup "1" >}}
|
|
> to their GitHub or GitLab repository triggers a build and deployment. Popular
|
|
> providers include AWS Amplify, CloudCannon, Cloudflare Pages, GitHub Pages,
|
|
> GitLab Pages, and Netlify.
|
|
>
|
|
> 1. The Git repository contains the entire project directory, typically excluding the
|
|
> public directory because the site is built _after_ the push.
|
|
|
|
importantly, you can't make a post about deploying this way. _everyone_ deploys
|
|
this way. if _i_ deploy this way, this site will have no content.
|
|
|
|
this approach also involves a build system somewhere that can run Hugo to
|
|
build and push the compiled code and assets onto my cluster. i definitely
|
|
already need Hugo installed on my workstation if i'm going to post
|
|
anything.[^2] so now i'm running Hugo in two places. there's surely
|
|
going to be other complex nonsense like webhooks involved.
|
|
|
|
[^2]: unlikely.
|
|
|
|
![diagram: deploy w/ GitHub Pages & Actions](images/hugo-github-pages.svg)
|
|
|
|
----
|
|
|
|
and hang on. let's look at this again:
|
|
|
|
> 1. The Git repository contains the entire project directory, typically excluding the
|
|
> public directory because the site is built _after_ the push.
|
|
|
|
you're telling me i'm going to build a nice static site and not check the
|
|
_actual content_ into version control? couldn't be me.
|
|
|
|
## Getting Static
|
|
|
|
suppose i instead check my content into git exactly as i intend to serve it?
|
|
then i could shell into my server box, pull the site, and _nifty-galifty!_ isn't
|
|
this the way it has [always been done][worm-love]?
|
|
|
|
my problem is that i don't have a server box. i have a _container orchestration
|
|
system_. there are several upsides to this[^3] but it means that _somehow_ my
|
|
generated content needs to end up in a container. because [Pods][k8s-pods] are
|
|
ephemeral and i'd like to run my site with horizontal scalability[^4], i don't
|
|
want my container to need to retain runtime state across restarts or replicas.
|
|
|
|
[^3]: few of which could be considered relevant for my project.
|
|
[^4]: i absolutely will not need this
|
|
|
|
i _could_ run a little pipeline that builds a container wrapping my content and
|
|
pushes it to a registry somewhere my deployments can pull it. all ready to go.
|
|
but now i've got _software_ again: build stages and webhooks and, to make matters
|
|
worse, now i'm hosting and versioning container images.
|
|
|
|
![diagram: deploy w/ container build](images/hugo-container-build.svg)
|
|
|
|
i don't want any of this. i just want to put some HTML and static assets behind a
|
|
web server.
|
|
|
|
---
|
|
|
|
instead, i'd like to deploy a popular container image from a public registry
|
|
and deliver my content to it continuously.
|
|
|
|
a minimal setup to achieve this might look like:
|
|
|
|
- a `Pod` with:
|
|
- an `nginx` container to serve the content;
|
|
- a `git-pull` sidecar that loops, pulling the git content;
|
|
- an `initContainer` to do the initial checkout;
|
|
- an `emptyDir` volume to share between the containers.
|
|
- a `ConfigMap` to store the nginx config.
|
|
|
|
![diagram: minimal pod/configmap setup](images/hugo-minimal-pod-setup.svg)
|
|
|
|
when a new pod comes up, the `initContainer` mounts the
|
|
[`emptyDir`][k8s-emptydir] and clones the repository into `/www`. i use
|
|
`git sparse-checkout` to avoid pulling repository contents i don't want
|
|
to serve out:
|
|
|
|
```bash
|
|
# git-clone command
|
|
git clone https://code.estradiol.cloud/tamsin/estradiol.cloud.git --no-checkout --branch trunk /tmp/www;
|
|
cd /tmp/www;
|
|
git sparse-checkout init --cone;
|
|
git sparse-checkout set public;
|
|
git checkout;
|
|
shopt -s dotglob
|
|
mv /tmp/www/* /www
|
|
```
|
|
|
|
for the sidecar, i script up my `git pull` loop:
|
|
|
|
```bash
|
|
# git-pull command
|
|
while true; do
|
|
cd /www && git -c safe.directory=/www pull origin trunk
|
|
sleep 60
|
|
done
|
|
```
|
|
|
|
and i create a [ConfigMap][k8s-configmap] with a server block to configure
|
|
`nginx` to use Hugo's `public/` as root:
|
|
|
|
|
|
```txt
|
|
# ConfigMap; data: default.conf
|
|
server {
|
|
listen 80;
|
|
location / {
|
|
root /www/public;
|
|
index index.html;
|
|
}
|
|
}
|
|
```
|
|
|
|
the rest of this is pretty much boilerplate:
|
|
|
|
{{< code-details summary="`kubectl apply -f estradiol-cloud.yaml`" lang="yaml" details=`
|
|
# estradiol-cloud.yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/instance: estradiol-cloud
|
|
app.kubernetes.io/name: nginx
|
|
name: nginx-server-block
|
|
data:
|
|
default.conf: |-
|
|
server {
|
|
listen 80;
|
|
location / {
|
|
root /www/public;
|
|
index index.html;
|
|
}
|
|
}
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: nginx
|
|
labels:
|
|
app.kubernetes.io/instance: estradiol-cloud
|
|
app.kubernetes.io/name: nginx
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.25.4
|
|
ports:
|
|
- containerPort: 80
|
|
volumeMounts:
|
|
- mountPath: /www
|
|
name: www
|
|
- mountPath: /etc/nginx/conf.d
|
|
name: nginx-server-block
|
|
- name: git-pull
|
|
image: bitnami/git
|
|
command:
|
|
- /bin/bash
|
|
- -ec
|
|
- |
|
|
while true; do
|
|
cd /www && git -c safe.directory=/www pull origin trunk
|
|
sleep 60
|
|
done
|
|
volumeMounts:
|
|
- mountPath: /www
|
|
name: www
|
|
initContainers:
|
|
- name: git-clone
|
|
image: bitnami/git
|
|
command:
|
|
- /bin/bash
|
|
- -c
|
|
- |
|
|
shopt -s dotglob
|
|
git clone https://code.estradiol.cloud/tamsin/estradiol.cloud.git --no-checkout --branch trunk /tmp/www;
|
|
cd /tmp/www;
|
|
git sparse-checkout init --cone;
|
|
git sparse-checkout set public;
|
|
git checkout;
|
|
mv /tmp/www/* /www
|
|
volumeMounts:
|
|
- mountPath: /www
|
|
name: www
|
|
volumes:
|
|
- name: www
|
|
emptyDir: {}
|
|
- name: nginx-server-block
|
|
configMap:
|
|
name: nginx-server-block
|
|
` >}}
|
|
|
|
---
|
|
|
|
my Hugo workflow now looks like:
|
|
|
|
1. make changes to source;
|
|
1. run `hugo --gc --minify`;[^7]
|
|
1. commit & push.
|
|
|
|
my `git pull` control loop takes things over from here and i'm on easy street.
|
|
|
|
[^7]: i added `disableHTML = true` and `disableXML = true` to `[minify]`
|
|
configuration in `hugo.toml` to keep HTML and RSS diffs readable.
|
|
|
|
|
|
## Getting Web
|
|
|
|
this is going great! my Pod is running. it's serving out my code. i get
|
|
continuous deployment™ for the low price of 11 lines `bash`. i mean...
|
|
no one can actually browse to my website[^8] but that will be an easy fix,
|
|
right?
|
|
|
|
[^8]: i can check that its working, at least, with a [port-forward][k8s-port].
|
|
|
|
first, i need a [`Service`][k8s-svc]. this gives me a proxy with service
|
|
discovery. TK: what is this really?
|
|
|
|
{{< code-details summary="kubectl apply -f service.yaml" lang="yaml" details=`
|
|
# service.yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/instance: estradiol-cloud
|
|
app.kubernetes.io/name: nginx
|
|
name: nginx
|
|
spec:
|
|
type: ClusterIP
|
|
selector:
|
|
app.kubernetes.io/instance: estradiol-cloud
|
|
app.kubernetes.io/name: nginx
|
|
ports:
|
|
- name: http
|
|
port: 80
|
|
protocol: TCP
|
|
targetPort: http
|
|
` >}}
|
|
|
|
|
|
and i need an [`Ingress`][k8s-ingress] to handle traffic inbound to the cluster
|
|
and direct it to the `Service`:
|
|
|
|
{{< code-details summary="kubectl apply -f ingress.yaml" lang="yaml" details=`
|
|
# ingress.yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/instance: estradiol-cloud
|
|
app.kubernetes.io/name: nginx
|
|
name: nginx
|
|
spec:
|
|
rules:
|
|
- host: estradiol.cloud
|
|
http:
|
|
paths:
|
|
- backend:
|
|
service:
|
|
name: nginx
|
|
port:
|
|
name: http
|
|
path: /
|
|
pathType: Prefix
|
|
` >}}
|
|
|
|
TK: wtf? Ingress controller
|
|
|
|
---
|
|
|
|
as this has come together, i've gotten increasinly anxious about how much
|
|
YAML i've had to write. this is a problem because YAML is software and, as
|
|
we've established, i'm hoping not to have much of that. it's also annoying
|
|
that most of this YAML really is boilerplate.
|
|
|
|
conveniently, [Bitnami][bitnami] maintains a [Helm][helm] Chart that hides all
|
|
the boilerplate and does exactly what we've just been doing manually.[^9]
|
|
|
|
[^9]: what incredible luck! (obviously, until now i've been working backward from this chart)
|
|
|
|
TK: install helm
|
|
TK: pull bitnami chart
|
|
TK: helm values
|
|
|
|
{{< code-details summary="`helm upgrade --install --create-namespace --namespace estradiol-cloud -f values.yaml`" lang="yaml" details=`
|
|
# values.yaml
|
|
cloneStaticSiteFromGit:
|
|
enabled: true
|
|
repository: "https://code.estradiol.cloud/tamsin/estradiol.cloud.git"
|
|
branch: trunk
|
|
gitClone:
|
|
command:
|
|
- /bin/bash
|
|
- -ec
|
|
- |
|
|
[[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
|
|
git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
|
|
[[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
|
|
ingress:
|
|
enabled: true
|
|
hostname: estradiol.cloud
|
|
ingressClassName: nginx
|
|
tls: true
|
|
annotations: {
|
|
cert-manager.io/cluster-issuer: letsencrypt-prod
|
|
}
|
|
serverBlock: |-
|
|
server {
|
|
listen 8080;
|
|
root /app/public;
|
|
index index.html;
|
|
}
|
|
service:
|
|
type: ClusterIP
|
|
`>}}
|
|
|
|
![diagram: helm setup](images/hugo-helm-setup.svg)
|
|
|
|
## Getting Flux'd
|
|
|
|
by this point i'm pretty `git push`-pilled and i'm thinking i don't much like
|
|
having this `helm` client software installed on my laptop. plus, i still have
|
|
some YAML and it's not really great that i'm storing it in flat files and
|
|
pushing it to my cluster manually. i love automation. i might love automation
|
|
more than i disdain software. i feel prepared to get some software if it
|
|
will get this yaml out of my shell history and into a git repo.
|
|
|
|
|
|
TK: Flux
|
|
|
|
![diagram: flux git push/deploy sequence](images/flux-seq.svg)
|
|
|
|
```yaml
|
|
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
|
kind: HelmRepository
|
|
metadata:
|
|
name: bitnami
|
|
namespace: default
|
|
spec:
|
|
url: https://charts.bitnami.com/bitnami
|
|
```
|
|
|
|
{{< code-details summary="`release.yaml`" lang="yaml" details=`
|
|
apiVersion: helm.toolkit.fluxcd.io/v2beta1
|
|
kind: HelmRelease
|
|
metadata:
|
|
name: web
|
|
namespace: estradiol-cloud
|
|
spec:
|
|
interval: 5m
|
|
chart:
|
|
spec:
|
|
chart: nginx
|
|
version: '15.12.2'
|
|
sourceRef:
|
|
kind: HelmRepository
|
|
name: bitnami
|
|
namespace: default
|
|
interval: 1m
|
|
values:
|
|
cloneStaticSiteFromGit:
|
|
enabled: true
|
|
repository: "https://code.estradiol.cloud/tamsin/estradiol.cloud.git"
|
|
branch: trunk
|
|
gitClone:
|
|
command:
|
|
- /bin/bash
|
|
- -ec
|
|
- |
|
|
[[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
|
|
git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
|
|
[[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
|
|
ingress:
|
|
enabled: true
|
|
hostname: estradiol.cloud
|
|
ingressClassName: nginx
|
|
tls: true
|
|
annotations: {
|
|
cert-manager.io/cluster-issuer: letsencrypt-prod
|
|
}
|
|
serverBlock: |-
|
|
server {
|
|
listen 8080;
|
|
root /app/public;
|
|
index index.html;
|
|
}
|
|
service:
|
|
type: ClusterIP
|
|
`>}}
|
|
|
|
## A Note About Software
|
|
|
|
at this point i'm forced to admit there's still a lot of software involved in this.
|
|
setting aside the stuff that provisions and scales my cluster nodes, i have:
|
|
|
|
- `nginx` (running from a stock image);
|
|
- `git` & `bash` (running from a stock image);
|
|
- a remote git server (i'm running `gitea`[^8], but github dot com is fine here);
|
|
- Kubernetes (oops!);
|
|
- `fluxcd`, especially `kustomize-controller` and `helm-controller`;
|
|
- `nginx-ingress` controller;
|
|
- the `bitnami/nginx` Helm chart;
|
|
|
|
[^8]: because i'm running `gitea` in my cluster and i want to avoid a circular
|
|
dependency for my `flux` source repository, i also depend on GitLab dot com.
|
|
|
|
i get to maintain my two `bash` scripts for `git-clone` and `git-pull`, my
|
|
NGINX config, and a couple of blobs of YAML.
|
|
|
|
at least there are no webhooks.
|
|
|
|
---
|
|
|
|
_fin_
|
|
|
|
|
|
[bitnami]: https://bitnami.com/
|
|
[cert-mgr]: https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
|
|
[helm]: https://helm.sh
|
|
[hugo]: https://gohugo.io
|
|
[hugo-deploy]: https://gohugo.io/getting-started/usage/#deploy-your-site
|
|
[hugo-started]: https://gohugo.io/getting-started
|
|
[k8s]: https://kubernetes.io
|
|
[k8s-emptydir]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
|
|
[k8s-init]: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
|
|
[k8s-ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
|
|
[k8s-pods]: https://kubernetes.io/docs/concepts/workloads/pods/
|
|
[k8s-port]: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
|
|
[k8s-pv]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
|
|
[k8s-svc]: https://kubernetes.io/docs/concepts/services-networking/service/
|
|
[risotto]: https://github.com/joeroe/risotto
|
|
[worm-love]: https://www.mikecurato.com/worm-loves-worm
|