Setting up Zot on Kubernetes with S3 Storage and OIDC

calendar Jun 4, 2026 · 7 min read · Kubernetes Registry OCI Docker  ·
Share on: linkedin copy

I wanted a light weight container registry that could run on Kubernetes, store images in S3-compatible storage.

Zot ended up being a good fit for this setup. It supports S3 storage, has a built-in web UI, works with OpenID Connect, and can still use htpasswd for Docker CLI logins.

That last part matters because Docker does not handle browser-based OIDC login flows.

The web UI can use Authentik, while docker login can use a normal username and password from an htpasswd file.

This post walks through the setup I used with Kubernetes, Helm, S3 , and Authentik.

Requirements

You will need:

  • A Kubernetes cluster
  • Helm
  • S3-compatible storage, such as RustFS, MinIO, or AWS S3
  • A Bucket already created for Zot

Step 1: Set Up the Access Policy for Bucket

The policy needs to allow:

  • Reading, writing, and deleting objects
  • Multipart uploads (this is critical for Docker pushes)

Here's the policy I used:

 1{
 2  "Version": "2012-10-17",
 3  "Statement": [
 4    {
 5      "Sid": "AllowBucketAccess",
 6      "Effect": "Allow",
 7      "Action": [
 8        "s3:ListBucket",
 9        "s3:GetBucketLocation",
10        "s3:ListBucketMultipartUploads"
11      ],
12      "Resource": ["arn:aws:s3:::zot-registry"]
13    },
14    {
15      "Sid": "AllowObjectAccess",
16      "Effect": "Allow",
17      "Action": [
18        "s3:PutObject",
19        "s3:GetObject",
20        "s3:DeleteObject",
21        "s3:ListMultipartUploadParts",
22        "s3:AbortMultipartUpload"
23      ],
24      "Resource": ["arn:aws:s3:::zot-registry/*"]
25    }
26  ]
27}

Step 2: Configure Authentik

In Authentik, create a new application and provider.

Use:

1OAuth2/OpenID Connect

For the redirect URI, use your Zot public URL with the OIDC callback path:

1https://registry.homelab.pk/zot/auth/callback/oidc

Set this as a strict redirect URI.

From the Authentik provider page, copy the following values:

  • Client ID
  • Client Secret
  • Issuer URL

You will use these later in the Zot configuration.

Step 3: Create the required Kubernetes secrets

This setup uses three secrets:

  1. S3 access credentials
  2. OIDC client credentials
  3. An htpasswd file for Docker CLI login

For Docker CLI access, generate an htpasswd file locally:

1htpasswd -Bbn youruser 'yourpassword' > htpasswd

Zot expects the OIDC credentials as a small JSON file in this format:

1{
2  "clientid": "YOUR_CLIENT_ID",
3  "clientsecret": "YOUR_CLIENT_SECRET"
4}

I use External Secrets, so the secrets are created before Zot starts.

 1apiVersion: external-secrets.io/v1
 2kind: ExternalSecret
 3metadata:
 4  name: zot-oidc-creds
 5  namespace: zot
 6  annotations:
 7    argocd.argoproj.io/sync-wave: "-1"
 8    argocd.argoproj.io/hook: PreSync
 9spec:
10  secretStoreRef:
11    name: bitwarden-cluster-secretsmanager
12    kind: ClusterSecretStore
13  refreshInterval: "24h"
14  target:
15    name: zot-oidc-creds
16    template:
17      engineVersion: v2
18      data:
19        oidc-credentials.json: |
20          {
21            "clientid": "{{ .client_id }}",
22            "clientsecret": "{{ .client_secret }}"
23          }          
24  data:
25    - secretKey: client_id
26      remoteRef:
27        key: zot-oidc-client-id
28    - secretKey: client_secret
29      remoteRef:
30        key: zot-oidc-client-secret
31---
32apiVersion: external-secrets.io/v1
33kind: ExternalSecret
34metadata:
35  name: zot-htpasswd
36  namespace: zot
37  annotations:
38    argocd.argoproj.io/sync-wave: "-1"
39    argocd.argoproj.io/hook: PreSync
40spec:
41  secretStoreRef:
42    name: bitwarden-cluster-secretsmanager
43    kind: ClusterSecretStore
44  refreshInterval: "24h"
45  target:
46    name: zot-htpasswd
47    template:
48      engineVersion: v2
49      data:
50        htpasswd: '{{ .htpasswd }}'
51  data:
52    - secretKey: htpasswd
53      remoteRef:
54        key: zot-htpasswd
55---
56apiVersion: external-secrets.io/v1
57kind: ExternalSecret
58metadata:
59  name: zot-rustfs-creds
60  namespace: zot
61  annotations:
62    argocd.argoproj.io/sync-wave: "-1"
63    argocd.argoproj.io/hook: PreSync
64spec:
65  secretStoreRef:
66    name: bitwarden-cluster-secretsmanager
67    kind: ClusterSecretStore
68  refreshInterval: "24h"
69  target:
70    name: zot-rustfs-creds
71  data:
72    - secretKey: accesskey
73      remoteRef:
74        key: zot-rustfs-access
75    - secretKey: secretkey
76      remoteRef:
77        key: zot-rustfs-secret

Adjust the secretStoreRef and remoteRef values to match your own secret backend.

Step 4: Configure Zot with Helm

Here is the complete values.yaml used for the deployment.

 1replicaCount: 1
 2
 3image:
 4  repository: ghcr.io/project-zot/zot-linux-amd64
 5  pullPolicy: IfNotPresent
 6  tag: "v2.1.17"
 7
 8namespace: ""
 9
10serviceAccount:
11  create: true
12  name: ""
13
14service:
15  type: ClusterIP
16  port: 5000
17  nodePort: null
18  annotations: {}
19  clusterIP: null
20
21env:
22  - name: AWS_ACCESS_KEY_ID
23    valueFrom:
24      secretKeyRef:
25        name: zot-rustfs-creds
26        key: accesskey
27  - name: AWS_SECRET_ACCESS_KEY
28    valueFrom:
29      secretKeyRef:
30        name: zot-rustfs-creds
31        key: secretkey
32
33externalSecrets:
34  - secretName: "zot-oidc-creds"
35    mountPath: "/etc/zot/oidc"
36  - secretName: "zot-htpasswd"
37    mountPath: "/etc/zot/htpasswd"
38
39mountConfig: true
40
41configFiles:
42  config.json: |-
43    {
44      "storage": {
45        "rootDirectory": "/var/lib/registry",
46        "dedupe": false,
47        "storageDriver": {
48          "name": "s3",
49          "region": "us-east-1",
50          "regionendpoint": "http://192.168.0.20:9000",
51          "bucket": "zot-registry",
52          "secure": false,
53          "skipverify": false,
54          "forcepathstyle": true
55        }
56      },
57      "http": {
58        "address": "0.0.0.0",
59        "port": "5000",
60        "readTimeout": "60s",
61        "writeTimeout": "60s",
62        "externalUrl": "https://registry.homelab.pk",
63        "compat": ["docker2s2"],
64        "auth": {
65          "htpasswd": {
66            "path": "/etc/zot/htpasswd/htpasswd"
67          },
68          "openid": {
69            "providers": {
70              "oidc": {
71                "name": "Authentik",
72                "credentialsFile": "/etc/zot/oidc/oidc-credentials.json",
73                "issuer": "https://auth.homelab.pk/application/o/zot/",
74                "keypath": "",
75                "scopes": ["openid", "profile", "email", "groups"]
76              }
77            }
78          }
79        }
80      },
81      "extensions": {
82        "search": {
83          "enable": true
84        },
85        "ui": {
86          "enable": true
87        }
88      },
89      "log": {
90        "level": "info"
91      }
92    }    
93
94strategy:
95  type: RollingUpdate

Replace these values with your own:

1regionendpoint: "http://192.168.0.20:9000"
2bucket: "zot-registry"
3externalUrl: "https://registry.homelab.pk"
4issuer: "https://auth.homelab.pk/application/o/zot/"

The issuer value should match the issuer URL shown in your Authentik provider.

Notes on the Zot configuration

A few settings are worth calling out.

externalUrl

This must match the public URL you use for the registry.

For example:

1"externalUrl": "https://registry.homelab.pk"

If this does not match the URL used by Docker, pushes can fail with confusing manifest errors.

compat: ["docker2s2"]

Keep this enabled:

1"compat": ["docker2s2"]

Without it, Docker may upload the blobs but fail when pushing the final manifest. A common error is HTTP 415.

dedupe: false

Zot supports deduplication, but with S3 it requires Redis.

For a single-replica homelab deployment, disabling it keeps the setup simpler:

1"dedupe": false

rootDirectory

Even though image layers are stored in S3, Zot still needs a local root directory:

1"rootDirectory": "/var/lib/registry"

Zot uses this for local metadata and runtime data.

forcepathstyle: true

Use this for self-hosted S3-compatible storage such as RustFS or MinIO:

1"forcepathstyle": true

For AWS S3, you may not need it.

Secret mount paths

The mounted secret paths must match the paths in config.json.

In this example:

1externalSecrets:
2  - secretName: "zot-oidc-creds"
3    mountPath: "/etc/zot/oidc"
4  - secretName: "zot-htpasswd"
5    mountPath: "/etc/zot/htpasswd"

And in the Zot config:

1"credentialsFile": "/etc/zot/oidc/oidc-credentials.json"

1"path": "/etc/zot/htpasswd/htpasswd"

If the file names or paths do not match, authentication will fail.

Step 5: Deploy Zot

Deploy Zot with Helm:

1helm upgrade --install zot project-zot/zot \
2  --namespace zot \
3  --create-namespace \
4  -f values.yaml

Wait for the deployment to become ready:

1kubectl -n zot rollout status deployment/zot-zot

Step 6: Check the mounted files

Once the pod is running, check that the secrets were mounted correctly:

1kubectl exec -it -n zot deploy/zot-zot -- sh

Then inside the pod:

1ls -l /etc/zot/oidc
2ls -l /etc/zot/htpasswd

You should see:

1oidc-credentials.json
2htpasswd

If either file is missing, check your ExternalSecret configuration before troubleshooting Zot itself.

Step 7: Test the Authentik login

Open the registry URL in your browser:

1https://registry.homelab.pk

You should see a sign-in option for OIDC.

Click it, log in through Authentik, and you should be redirected back to Zot as an authenticated user.

If this fails, check:

  • The externalUrl in Zot
  • The redirect URI in Authentik
  • The Authentik issuer URL
  • The mounted OIDC credentials file
  • The Client ID and Client Secret

Step 8: Test Docker CLI access

Now test Docker login and push:

1docker login registry.homelab.pk

Use the username and password from your htpasswd file.

Then push a test image:

1docker pull alpine:3.21.3
2docker tag alpine:3.21.3 registry.homelab.pk/alpine:3.21.3
3docker push registry.homelab.pk/alpine:3.21.3

If the push succeeds, Zot is working with S3 storage and Docker CLI authentication.

Troubleshooting

NoSuchBucket

The S3 bucket does not exist.

Create the bucket first, then restart Zot.

Access Denied during large uploads

Your S3 policy may be missing multipart upload permissions.

Check that the policy allows actions such as:

1ListMultipartUploadParts
2AbortMultipartUpload

Manifest errors during Docker push

Check these two settings:

1"externalUrl": "https://registry.homelab.pk"

1"compat": ["docker2s2"]

A missing docker2s2 compatibility setting can cause the blobs to upload successfully, then fail when Docker pushes the manifest.

Web UI login works, but docker login fails

OIDC and Docker CLI authentication are separate in this setup.

If the web UI works but Docker login fails, check:

  • The htpasswd file
  • The mounted secret path
  • The password hash format
  • The path configured in Zot:
1"path": "/etc/zot/htpasswd/htpasswd"

Final notes

The main things to get right are the Authentik redirect URI, the Zot externalUrl, the S3 bucket and permissions, and the compat: ["docker2s2"] setting.

Once those are in place, Zot works well as a container registry with S3 storage, Authentik login for the web UI, and normal docker login support for CLI access.