After multiple upgrades of a kURL cluster, you might get to a point where the partition used by kubernetes, usually root, starts filling up. After some troubleshooting, you might see loads of files in /etc/kubernetes/tmp
directory named either kubeadm-backup-etcd-<date>-<time>
or kubeadm-backup-manifests-<date>-<time>
. These are backups stored by kubeadm upgrade
command which the kURL installer uses internally.
kubeadm-backup-manifests-<date>-<time>
: Static pod manifests. These are fairly small in size (<1 KB)kubeadm-backup-etcd-<date>-<time>
: etcd database snapshots backed up just before runningkubeadm upgrade
. These can end up being large directories especially if the cluster has many resources. Each control plane node in the cluster will have these backups. These are the directories you most likely want to clean up.
NOTE: When deleting etcd
backups, ensure you leave at least one backup behind for recovery if needed
To view all the backups you have in a node, you would run
$ sudo ls -al /etc/kubernetes/tmp/
total 20
drwx------ 5 root root 4096 Sep 5 13:44 .
drwxr-xr-x 5 root root 4096 Sep 5 14:19 ..
drwx------ 3 root root 4096 Sep 5 13:42 kubeadm-backup-etcd-2023-09-05-13-42-31
drwx------ 2 root root 4096 Sep 5 13:44 kubeadm-backup-manifests-2023-09-05-13-42-31
drwx------ 2 root root 4096 Sep 5 13:44 kubeadm-kubelet-config388725400
References