tamsin/estradiol.cloud
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

finish up hugo-on-k8s-nginx post

Browse Source
This commit is contained in:
tamsin woo 2024-03-12 16:27:06 -07:00
parent d2d6dd1a56
commit 579ca6ffbf
10 changed files with 832 additions and 697 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
.d2/hugo-helm-setup.d2
View File

@ -6,23 +6,30 @@ title: |md
}
my laptop -> repo: git push
my laptop -> kubernetes: `kubectl apply`
my laptop -> kubernetes: `helm upgrade`
kubernetes {
icon: https://icons.terrastruct.com/azure%2F_Companies%2FKubernetes.svg
ingress -> service
service -> pod(s)
service -> deployment.pod(s)
pod(s) {
git-pull -> emptyDir: mounts
nginx -> emptyDir: mounts
emptyDir { icon: https://icons.terrastruct.com/tech%2Fdiskette.svg }
deployment {
pod(s) {
git-pull -> emptyDir: mounts
init {
style.stroke-dash: 3
}
init -> emptyDir: mounts
nginx -> emptyDir: mounts
emptyDir { icon: https://icons.terrastruct.com/tech%2Fdiskette.svg }
}
}
pod(s).nginx -> configMap: mounts
deployment.pod(s).nginx -> configMap: mounts
}
kubernetes.pod(s).git-pull -> repo: pull
kubernetes.deployment.pod(s).git-pull -> repo: pull
web {
icon: https://icons.terrastruct.com/essentials%2F140-internet.svg

5
.d2/hugo-minimal-pod-setup.d2
View File

@ -13,7 +13,12 @@ kubernetes {
pod {
git-pull -> emptyDir: mounts
init {
style.stroke-dash: 3
}
init -> emptyDir: mounts
nginx -> emptyDir: mounts
emptyDir { icon: https://icons.terrastruct.com/tech%2Fdiskette.svg }
}
pod.nginx -> configMap: mounts

601
content/posts/hugo-on-k8s-nginx/images/hugo-helm-setup.svg
View File

File diff suppressed because one or more lines are too long
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 151 KiB

After

Width:  |  Height:  |  Size: 155 KiB

584
content/posts/hugo-on-k8s-nginx/images/hugo-minimal-pod-setup.svg
View File

File diff suppressed because one or more lines are too long
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 136 KiB

After

Width:  |  Height:  |  Size: 139 KiB

225
content/posts/hugo-on-k8s-nginx/index.md
View File

@ -1,6 +1,6 @@
+++
title = 'Hugo on Kubernetes & NGINX'
date = 2024-02-28T15:35:46-08:00
date = 2024-03-13T08:25:00-07:00
draft = true
series = ['wtf']
categories = ['Tutorial']
@ -9,9 +9,9 @@ toc = true
+++
i decided to make a website. a static one. this one. with [Hugo][hugo]. the
main reason i have for needing a website is as a vanity project, because i need
some stuff to host in a [Kubernetes][k8s] cluster i'm running. the k8s cluster
is also a vanity project.
main reason i have for needing a website is as a vanity project, so i have some
stuff to host in a [Kubernetes][k8s] cluster i'm running. the k8s cluster is
also a vanity project.
because i don't like software, i wanted a way to deploy my site that doesn't
involve much of it. this post is about that.
@ -33,8 +33,8 @@ at this point, my website is basically finished (i also changed the title in
`hugo.toml`). i probably won't be putting anything on it, so there's no point
fiddling with other details.
about deployment, the guide's _[Basic Usage][hugo-deploy]_ page has this to
offer:
about deployment, the Hugo guide's _[Basic Usage][hugo-deploy]_ page has this
to offer:
> Most of our users deploy their sites using a CI/CD workflow, where a push{{< sup "1" >}}
> to their GitHub or GitLab repository triggers a build and deployment. Popular
@ -48,10 +48,10 @@ importantly, you can't make a post about deploying this way. _everyone_ deploys
this way. if _i_ deploy this way, this site will have no content.
this approach also involves a build system somewhere that can run Hugo to
build and push the compiled code and assets onto my cluster. i definitely
already need Hugo installed on my workstation if i'm going to post
anything.[^2] so now i'm running Hugo in two places. there's surely
going to be other complex nonsense like webhooks involved.
compile the code and assets and push them onto my host. i definitely
already need Hugo installed on my laptop if i'm going to post anything.[^2]
so now i'm running Hugo in two places. there's surely going to be other
complex nonsense like webhooks involved.
[^2]: unlikely.
@ -69,7 +69,7 @@ _actual content_ into version control? couldn't be me.
## Getting Static
suppose i instead check my content into git exactly as i intend to serve it?
suppose i instead check my content in exactly as i intend to serve it?
then i could shell into my server box, pull the site, and _nifty-galifty!_ isn't
this the way it has [always been done][worm-love]?
@ -77,15 +77,16 @@ my problem is that i don't have a server box. i have a _container orchestration
system_. there are several upsides to this[^3] but it means that _somehow_ my
generated content needs to end up in a container. because [Pods][k8s-pods] are
ephemeral and i'd like to run my site with horizontal scalability[^4], i don't
want my container to need to retain runtime state across restarts or replicas.
want my container to retain runtime state across restarts or replicas.
[^3]: few of which could be considered relevant for my project.
[^4]: i absolutely will not need this
i _could_ run a little pipeline that builds a container wrapping my content and
pushes it to a registry somewhere my deployments can pull it. all ready to go.
but now i've got _software_ again: build stages and webhooks and, to make matters
worse, now i'm hosting and versioning container images.
i _could_ run a little pipeline that builds a container image wrapping my
content and pushes it to a registry. when i deploy, the cluster pulls the
image, content and all. all ready to go. but now i've got _software_ again:
build stages and webhooks and, to make matters worse, now i'm hosting
and versioning container images.
![diagram: deploy w/ container build](images/hugo-container-build.svg)
@ -101,17 +102,17 @@ a minimal setup to achieve this might look like:
- a `Pod` with:
- an `nginx` container to serve the content;
- a `git-pull` sidecar that loops, pulling the git content;
- a `git-pull` sidecar that loops, pulling the content;
- an `initContainer` to do the initial checkout;
- an `emptyDir` volume to share between the containers.
- a `ConfigMap` to store the nginx config.
![diagram: minimal pod/configmap setup](images/hugo-minimal-pod-setup.svg)
when a new pod comes up, the `initContainer` mounts the
[`emptyDir`][k8s-emptydir] and clones the repository into `/www`. i use
`git sparse-checkout` to avoid pulling repository contents i don't want
to serve out:
when a new `Pod` comes up, the `initContainer` mounts the
[`emptyDir`][k8s-emptydir] at `/www` and clones the repository into it. i use
`git sparse-checkout` to avoid pulling repository contents i don't want to serve
out:
```bash
# git-clone command
@ -124,7 +125,7 @@ shopt -s dotglob
mv /tmp/www/* /www
```
for the sidecar, i script up my `git pull` loop:
for the sidecar, i script up a `git pull` loop:
```bash
# git-pull command
@ -151,7 +152,7 @@ server {
the rest of this is pretty much boilerplate:
{{< code-details summary="`kubectl apply -f estradiol-cloud.yaml`" lang="yaml" details=`
{{< code-details summary="`kubectl apply -f https://estradiol.cloud/posts/hugo-on-k8s/site.yaml`" lang="yaml" details=`
# estradiol-cloud.yaml
apiVersion: v1
kind: ConfigMap
@ -232,7 +233,7 @@ my Hugo workflow now looks like:
1. make changes to source;
1. run `hugo --gc --minify`;[^7]
1. commit & push.
1. `git` commit & push.
my `git pull` control loop takes things over from here and i'm on easy street.
@ -242,17 +243,19 @@ configuration in `hugo.toml` to keep HTML and RSS diffs readable.
## Getting Web
this is going great! my Pod is running. it's serving out my code. i get
continuous deployment™ for the low price of 11 lines `bash`. i mean...
this is going great! my `Pod` is running. it's serving out my code. i get
Continuous Deployment™ for the low price of 11 lines `bash`. i mean...
no one can actually browse to my website[^8] but that will be an easy fix,
right?
right? yes. networking is always the easy part.
[^8]: i can check that its working, at least, with a [port-forward][k8s-port].
first, i need a [`Service`][k8s-svc]. this gives me a proxy with service
discovery. TK: what is this really?
first, i need a [`Service`][k8s-svc]. this gives me a proxy to my several
replicas[^11] and in-cluster service discovery.
{{< code-details summary="kubectl apply -f service.yaml" lang="yaml" details=`
[^11]: lmao
{{< code-details summary="`kubectl apply -f https://estradiol.cloud/posts/hugo-on-k8s-nginx/service.yaml`" lang="yaml" details=`
# service.yaml
apiVersion: v1
kind: Service
@ -273,11 +276,10 @@ spec:
targetPort: http
` >}}
and i need an [`Ingress`][k8s-ingress] to handle traffic inbound to the cluster
next, i need an [`Ingress`][k8s-ingress] to handle traffic inbound to the cluster
and direct it to the `Service`:
{{< code-details summary="kubectl apply -f ingress.yaml" lang="yaml" details=`
{{< code-details summary="`kubectl apply -f https://estradiol.cloud/posts/hugo-on-k8s-nginx/ingress.yaml`" lang="yaml" details=`
# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
@ -300,25 +302,37 @@ spec:
pathType: Prefix
` >}}
TK: wtf? Ingress controller
this part expresses a routing rule: traffic reaching the cluster via
`estradiol.cloud` should go to my `Service`, and then to one of its backend `Pod`s.
to actually apply this rule, i need an ingress controller. mine is
[ingress-nginx][nginx-ingress].
when i deployed controller in my cluster, it created _some more_ `nginx` `Pod`s
in my cluster. these update their configuration dynamically based on the rules
in my `Ingress` resource(s). the controller also also creates a `Service` of
type `LoadBalancer`, which [magically][do-lb] creates a load balancer appliance
in my cloud provider. off-screen, i can point DNS to *that* appliance to finish
the setup.
[![diagram: kubernetes ingress](https://kubernetes.io/docs/images/ingress.svg)][k8s-ingress-wtf]
you can tell it's working by looking at your browser bar.
---
as this has come together, i've gotten increasinly anxious about how much
as this has come together, i've gotten increasingly anxious about how much
YAML i've had to write. this is a problem because YAML is software and, as
we've established, i'm hoping not to have much of that. it's also annoying
that most of this YAML really is boilerplate.
established, i'm hoping not to have much of that. it's also annoying that most
of this YAML really is just boilerplate.
conveniently, [Bitnami][bitnami] maintains a [Helm][helm] Chart that hides all
the boilerplate and does exactly what we've just been doing manually.[^9]
conveniently, [Bitnami][bitnami] maintains a [Helm][helm] Chart that templates
out all the boilerplate and does exactly what i've just been doing.[^9] i can
replace all my YAML with a call out to this chart and a few lines of
configuration, assuming i have [helm client installed][helm-install]:
[^9]: what incredible luck! (obviously, until now i've been working backward from this chart)
TK: install helm
TK: pull bitnami chart
TK: helm values
{{< code-details summary="`helm upgrade --install --create-namespace --namespace estradiol-cloud -f values.yaml`" lang="yaml" details=`
{{< code-details summary="`helm upgrade --install --create-namespace --namespace estradiol-cloud -f https://estradiol.cloud/posts/hugo-on-k8s-nginx/values.yaml oci://registry-1.docker.io/bitnamicharts/nginx`" lang="yaml" details=`
# values.yaml
cloneStaticSiteFromGit:
enabled: true
@ -331,7 +345,7 @@ cloneStaticSiteFromGit:
- |
[[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
[[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
[[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
ingress:
enabled: true
hostname: estradiol.cloud
@ -352,21 +366,53 @@ service:
![diagram: helm setup](images/hugo-helm-setup.svg)
configuration for the `git-clone` script and our custom server block are added
via `values.yaml`. the `git-pull` loop configured by the chart works as-is.
by using the chart, we get a few other nicities. for instance,
my `Pod`s are now managed by a [`Deployment`][k8s-deployment].[^44] this will make my
grand scale-out plans a breeze.
[^44]: i also snuck a TLS certificate configuration via Let's Encrypt with
[`cert-manager`][cert-mgr] into this iteration. if you're following along at home and don't have `cert-manager` installed, this should still work fine (but with no HTTPs).
## Getting Flux'd
by this point i'm pretty `git push`-pilled and i'm thinking i don't much like
having this `helm` client software installed on my laptop. plus, i still have
some YAML and it's not really great that i'm storing it in flat files and
pushing it to my cluster manually. i love automation. i might love automation
more than i disdain software. i feel prepared to get some software if it
will get this yaml out of my shell history and into a git repo.
by now, i'm riding high. my whole setup is my static site code and <30 lines of
YAML.
i *do* have a bunch of stuff deployed into my cluster, and none of this is very
reproducible without all of that. my workflow has also expanded to:
TK: Flux
1. for routine site deploys:
1. make changes to source;
1. run `hugo --gc --minify`;[^7]
1. `git` commit & push.
1. to update `nginx`, the chart version, or change config:
1. make changes to `values.yaml`
1. `helm upgrade`
![diagram: flux git push/deploy sequence](images/flux-seq.svg)
i could do without the extra `helm` clientdependency on my laptop. i'm also
pretty `git push`-pilled, and i really want the solution to all my problems
to take the now familiar shape: put a control loop in my cluster and push
to a `git` repository.
enter [`flux`][fluxcd].
with `flux`, i decide on a repository (and maybe a path within it) to act as
a source for my Kubernetes YAML. i go through a short [bootstrap][fluxcd-boot]
process which installs the `flux` controllers and add them to repository. to
make a change to a resource in my cluster, i edit the YAML and push to the
repository. `flux` listens and applies the changes.
`flux` supports Helm deploys, so i can get that `helm` client off my laptop.
i can also use it to manage my ingress controller, `cert-manager`, `flux`
itself and whatever other infrastructural junk i may end up needing.
to move my web stack into `flux`, i create a `HelmRepository` resource for
the `bitnami` Helm charts:
```yaml
# bitnami-helm.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
@ -376,6 +422,9 @@ spec:
url: https://charts.bitnami.com/bitnami
```
and add a `HelmRelease` pointing to the repository/chart version and containing
my `values.yaml`:
{{< code-details summary="`release.yaml`" lang="yaml" details=`
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
@ -424,25 +473,39 @@ spec:
type: ClusterIP
`>}}
when i push these to my `flux` [source repository][sublingual-ec], the Helm
release rolls out.
![diagram: flux git push/deploy sequence](images/flux-seq.svg)
## A Note About Software
at this point i'm forced to admit there's still a lot of software involved in this.
setting aside the stuff that provisions and scales my cluster nodes, i have:
in the end, i'm forced to admit there's still a lot of software involved in all
of this. setting aside the stuff that provisions and scales my cluster nodes,
and the _magic_ `LoadBalancer`, i have:
- `nginx` (running from a stock image);
- `git` & `bash` (running from a stock image);
- a remote git server (i'm running `gitea`[^8], but github dot com is fine here);
- a remote git server (i'm running `gitea`[^99], but github dot com is fine here);
- Kubernetes (oops!);
- `fluxcd`, especially `kustomize-controller` and `helm-controller`;
- `nginx-ingress` controller;
- `flux`, especially `kustomize-controller` and `helm-controller`;
- `ingress-nginx` controller;
- `cert-manager` and Let's Encrypt;
- the `bitnami/nginx` Helm chart;
[^8]: because i'm running `gitea` in my cluster and i want to avoid a circular
[^99]: because i'm running `gitea` in my cluster and i want to avoid a circular
dependency for my `flux` source repository, i also depend on GitLab dot com.
i get to maintain my two `bash` scripts for `git-clone` and `git-pull`, my
the bulk of this i'll be able to reuse for the other things i deploy on the
cluster[^80]. and it replaces SASS black-boxes like "AWS Amplify, CloudCannon,
Cloudflare Pages, GitHub Pages, GitLab Pages, and Netlify" in the recommended
Hugo deployment.
to actually deploy my site, i get to maintain a `bash` scripts for `git-clone`, my
NGINX config, and a couple of blobs of YAML.
[^80]: i won't.
at least there are no webhooks.
---
@ -450,19 +513,29 @@ at least there are no webhooks.
_fin_
[bitnami]: https://bitnami.com/
[cert-mgr]: https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
[helm]: https://helm.sh
[hugo]: https://gohugo.io
[hugo-deploy]: https://gohugo.io/getting-started/usage/#deploy-your-site
[hugo-started]: https://gohugo.io/getting-started
[k8s]: https://kubernetes.io
[k8s-emptydir]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
[k8s-init]: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
[k8s-ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[k8s-pods]: https://kubernetes.io/docs/concepts/workloads/pods/
[k8s-port]: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
[k8s-pv]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
[k8s-svc]: https://kubernetes.io/docs/concepts/services-networking/service/
[risotto]: https://github.com/joeroe/risotto
[worm-love]: https://www.mikecurato.com/worm-loves-worm
[bitnami]: https://bitnami.com/
[cert-mgr]: https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
[do-lb]: https://docs.digitalocean.com/products/kubernetes/how-to/add-load-balancers/
[fluxcd]: https://fluxcd.io/
[fluxcd-boot]: https://fluxcd.io/flux/installation/bootstrap/
[helm]: https://helm.sh
[helm-install]: https://helm.sh/docs/intro/install/
[hugo]: https://gohugo.io
[hugo-deploy]: https://gohugo.io/getting-started/usage/#deploy-your-site
[hugo-started]: https://gohugo.io/getting-started
[k8s]: https://kubernetes.io
[k8s-configmap]: https://kubernetes.io/docs/concepts/configuration/configmap/
[k8s-deployment]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
[k8s-emptydir]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
[k8s-init]: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
[k8s-ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[k8s-ingress-wtf]: https://kubernetes.io/docs/concepts/services-networking/ingress/#what-is-ingress
[k8s-pods]: https://kubernetes.io/docs/concepts/workloads/pods/
[k8s-port]: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
[k8s-pv]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
[k8s-svc]: https://kubernetes.io/docs/concepts/services-networking/service/
[nginx-ingress]: https://kubernetes.github.io/ingress-nginx/
[risotto]: https://github.com/joeroe/risotto
[sublingual-ec]: https://gitlab.com/no_reply/sublingual/-/tree/trunk/estradiol.cloud
[worm-love]: https://www.mikecurato.com/worm-loves-worm

19
content/posts/hugo-on-k8s-nginx/ingress.yaml Normal file
View File

@ -0,0 +1,19 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
labels:
app.kubernetes.io/instance: estradiol-cloud
app.kubernetes.io/name: nginx
name: nginx
spec:
rules:
- host: estradiol.cloud
http:
paths:
- backend:
service:
name: nginx
port:
name: http
path: /
pathType: Prefix

17
content/posts/hugo-on-k8s-nginx/service.yaml Normal file
View File

@ -0,0 +1,17 @@
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: estradiol-cloud
app.kubernetes.io/name: nginx
name: nginx
spec:
type: ClusterIP
selector:
app.kubernetes.io/instance: estradiol-cloud
app.kubernetes.io/name: nginx
ports:
- name: http
port: 80
protocol: TCP
targetPort: http

19
content/posts/hugo-on-k8s-nginx/site.yaml
View File

@ -28,7 +28,6 @@ spec:
image: nginx:1.25.4
ports:
- containerPort: 80
name: http
volumeMounts:
- mountPath: /www
name: www
@ -70,21 +69,3 @@ spec:
- name: nginx-server-block
configMap:
name: nginx-server-block
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: estradiol-cloud
app.kubernetes.io/name: nginx
name: nginx
spec:
type: ClusterIP
selector:
app.kubernetes.io/instance: estradiol-cloud
app.kubernetes.io/name: nginx
ports:
- name: http
port: 80
protocol: TCP
targetPort: http

28
content/posts/hugo-on-k8s-nginx/values.yaml Normal file
View File

@ -0,0 +1,28 @@
cloneStaticSiteFromGit:
enabled: true
repository: "https://code.estradiol.cloud/tamsin/estradiol.cloud.git"
branch: trunk
gitClone:
command:
- /bin/bash
- -ec
- |
[[ -f "/opt/bitnami/scripts/git/entrypoint.sh" ]] && source "/opt/bitnami/scripts/git/entrypoint.sh"
git clone {{ .Values.cloneStaticSiteFromGit.repository }} --no-checkout --branch {{ .Values.cloneStaticSiteFromGit.branch }} /tmp/app
[[ "$?" -eq 0 ]] && cd /tmp/app && git sparse-checkout init --cone && git sparse-checkout set public && git checkout && shopt -s dotglob && rm -rf /app/* && mv /tmp/app/* /app/
ingress:
enabled: true
hostname: estradiol.cloud
ingressClassName: nginx
tls: true
annotations: {
cert-manager.io/cluster-issuer: letsencrypt-prod
}
serverBlock: |-
server {
listen 8080;
root /app/public;
index index.html;
}
service:
type: ClusterIP

8
public/sitemap.xml
View File

@ -3,7 +3,10 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://estradiol.cloud/</loc>
<lastmod>2024-03-04T14:56:38-08:00</lastmod>
<lastmod>2024-03-13T08:25:00-07:00</lastmod>
</url><url>
<loc>https://estradiol.cloud/posts/</loc>
<lastmod>2024-03-13T08:25:00-07:00</lastmod>
</url><url>
<loc>https://estradiol.cloud/categories/</loc>
<lastmod>2024-03-04T14:56:38-08:00</lastmod>
@ -19,9 +22,6 @@
</url><url>
<loc>https://estradiol.cloud/posts/miniflux-rss/</loc>
<lastmod>2024-03-04T14:56:38-08:00</lastmod>
</url><url>
<loc>https://estradiol.cloud/posts/</loc>
<lastmod>2024-03-04T14:56:38-08:00</lastmod>
</url><url>
<loc>https://estradiol.cloud/tags/rss/</loc>
<lastmod>2024-03-04T14:56:38-08:00</lastmod>