When installing on an embedded multi-node cluster with the kURL weave add-on, there is a known issue where cluster networking does not work due to the following Weave error:
INFO: 2023/01/05 19:11:35.576705 ->[172.31.7.29:6783|da:05:9b:70:12:be(ip-172-31-7-29.ec2.internal)]: connection shutting down due to error: IP allocation was seeded by different peers (received: [da:05:9b:70:12:be(ip-172-31-7-29.ec2.internal)], ours: [86:64:6c:67:65:4f(ip-172-31-9-78.ec2.internal)])
INFO: 2023/01/05 19:11:35.576804 ->[172.31.7.29:6783|da:05:9b:70:12:be(ip-172-31-7-29.ec2.internal)]: connection deleted
Workaround
The official Weave docs provide the following workaround: Troubleshooting the IP Allocator
For kURL specifically, the following workaround will work:
# On each Kubernetes node run the following:
rm /var/lib/weave/weave-netdata.db
reboot
Going forward vendors and customers should plan to move away from Weave and adopt Flannel as their kURL CNI plugin.