Development Inside Kubernetes Part 4: Private Container Registry
In Development Inside Kubernetes Part 3: Let's Encrypt Dashboard that was just the warmup. I am now going to use the exact same process to setup and deploy my container registry. I will need a private/local container registry to use for my local development environment.
Recall that I opted for containerd
rather than moby/docker to manage my local container images. This means, I do not have builtin docker features. I have to push my container images somewhere when I build them. I am opting for a local private registry to do this. Docker has their own registry, but I may switch to another. For now, docker's registry is the easiest to setup. The path to least resistance.
Docker Secret
For HTTP auth, I need to create a set of credentials, then add it to a secret for kubernetes. To create a password, I can generate one from /dev/urandom
and use the docker registry
container to create the htpasswd
file.
export PASS=$(head -c 16 /dev/urandom | shasum | cut -d " " -f 1) # Generates random pw
export USER=admin
echo $USER > creds.txt
echo $PASS >> creds.txt
docker run --entrypoint htpasswd registry:2 -Bbn admin $PASS > ./htpasswd # Uses an older container image because newer images don't have htpasswd
unset $PASS $USER
There is a helm chart for docker-registry
that allows pretty much any case of private container registry installation. It even includes ingress which I am going to replace because I do not care for it.
helm repo add stable https://charts.helm.sh/stable # Helm 3 requires we add the repo for the stable repository
helm install stable/docker-registry \
--name registry \
--namespace default \ # kube-system is also a good place for this
--set persistence.enabled=true \ # This enables a PVC
--set secrets.htpasswd=$(cat ./htpasswd)
Traefik for Ingress
First, we need to setup ingress. It points to a secret with the same name as the registry deployment. In my case, I called it registry
❯ kubectl get secrets
NAME TYPE DATA AGE
registry-internal-docker-registry-secret Opaque 2 21m
sh.helm.release.v1.registry-internal.v1 helm.sh/release.v1 1 21m
I can see the secret name created from the helm deployment. I can pass that off to Traefik as a middleware for auth.
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: registry-auth
namespace: default
spec:
basicAuth:
secret: registry-internal-docker-registry-secret
The next piece requires me to setup the Traefik routes. Notice that for SSL, I am reusing my certificate from before local-acme-crt
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: registry
namespace: default
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host("registry.local.thelastpri.me")
services:
- kind: Service
port: 5000
name: registry-internal-docker-registry
namespace: default
middlewares:
- name: registry-auth
- kind: Rule
match: Host("registry.local.thelastpri.me") && PathPrefix("/v2")
services:
- kind: Service
port: 5000
name: registry-internal-docker-registry
namespace: default
middlewares:
- name: registry-auth
tls:
secretName: local-acme-crt
Next time, I am going to setup buildkit
and kaniko
to push out a build.