Example from my test cluster. Let’s say my failing pod is called sentry-worker
for the sake of example:
ada@ada-kurl:~$ kubectl get deployments.apps sentry-worker -o yaml | grep -C3 imagePullSecrets
--
- mountPath: /var/lib/sentry/files
name: sentry-data
dnsPolicy: ClusterFirst
imagePullSecrets:
- name: annarchy-registry
restartPolicy: Always
schedulerName: default-scheduler
I know my imagePullSecret is called annarchy-registry
for my test app. I examine the secret content:
ada@ada-kurl:~$ kubectl get secret annarchy-registry
NAME TYPE DATA AGE
annarchy-registry kubernetes.io/dockerconfigjson 1 153d
ada@ada-kurl:~$ kubectl get secret annarchy-registry -o yaml
apiVersion: v1
data:
.dockerconfigjson: eyJhdXRocyI6eyJwcm94eS5yZXBsaWNhdGVkLmNvbSI6eyJhdXRoIjoiTWtjd05rcGxTbVpSUVRCTFYxcERaMlZwWVROSFZIaDFlV0phT2pKSE1EWktaVXBtVVVFd1MxZGFRMmRsYVdFelIxUjRkWGxpV2c9PSJ9LCJyZWdpc3RyeS5yZXBsaWNhdGVkLmNvbSI6eyJhdXRoIjoiTWtjd05rcGxTbVpSUVRCTFYxcERaMlZwWVROSFZIaDFlV0phT2pKSE1EWktaVXBtVVVFd1MxZGFRMmRsYVdFelIxUjRkWGxpV2c9PSJ9fX0=
kind: Secret
metadata:
annotations:
...
name: annarchy-registry
namespace: default
resourceVersion: "3275557"
uid: 6609712a-b9cd-4a2f-a5fb-6fd0d4742d52
type: kubernetes.io/dockerconfigjson
Notice the .dockerconfigjson
base64 string in this secret - after decoding, this is exactly what would be contained in a regular docker
client CLI after doing a docker login <registry.domain.com>
. So you can recreate the same credentials by taking this string and decoding it and using it to authorize to our registry.
Normally, the docker client credentials file is at ~/.docker/config.json
. Add the decoded string to the authorizations list, or backup and replace your existing config.json
file:
# note in my case my working directory is /home/ada/snap/docker/2915/.docker/config.json because this is installed with ubuntu snap
ada@ada-kurl:~/snap/docker/2915/.docker$ ls
config.json
ada@ada-kurl:~/snap/docker/2915/.docker$ cat config.json
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "YWRhbWFuY2luaTo+cDZRdS5pJmZvWyUtTi5QTUIneUR8O3pm"
}
}
# this `o go-template=` option just automates the process of running the .dockerconfigjson value through `base64 --decode`
ada@ada-kurl:~/snap/docker/2915/.docker$ kubectl get secret annarchy-registry -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'
.dockerconfigjson: {"auths":{"proxy.replicated.com":{"auth":"MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="},"registry.replicated.com":{"auth":"MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="}}}
ada@ada-kurl:~/snap/docker/2915/.docker$ cp config.json config.json.bak
ada@ada-kurl:~/snap/docker/2915/.docker$ echo '{"auths":{"proxy.replicated.com":{"auth":"MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="},"registry.replicated.com":{"auth":"MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="}}}' > config.json
Printing the file through jq
prints the data nicely, we can see there are 2 endpoints authorized; registry.
and proxy.
; both use the same credentials. If your registry secret only contains one or the other, you could rewrite the host portion to proxy.replicated.com
for the purpose of this test.
ada@ada-kurl:~/snap/docker/2915/.docker$ cat config.json | jq
{
"auths": {
"proxy.replicated.com": {
"auth": "MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="
},
"registry.replicated.com": {
"auth": "MkcwNkplSmZRQTBLV1pDZ2VpYTNHVHh1eWJaOjJHMDZKZUpmUUEwS1daQ2dlaWEzR1R4dXliWg=="
}
}
}
My test app doesn’t actually have any private images in it, but I can authorize against the proxy that I’ve configured with my vendor portal account and docker hub credentials:
ada@ada-kurl:~/snap/docker/2915/.docker$ docker pull proxy.replicated.com/proxy/annarchy/index.docker.io/library/ubuntu:latest
latest: Pulling from proxy/annarchy/index.docker.io/library/ubuntu
57c139bbda7e: Pull complete
Digest: sha256:e9569c25505f33ff72e88b2990887c9dcf230f23259da296eb814fc2b41af999
Status: Downloaded newer image for proxy.replicated.com/proxy/annarchy/index.docker.io/library/ubuntu:latest
proxy.replicated.com/proxy/annarchy/index.docker.io/library/ubuntu:latest