# Troubleshooting Cluster Networking **URL:** https://community.replicated.com/t/troubleshooting-cluster-networking/568 **Category:** Supporting your customers **Created:** [August 30, 2021, 9:24pm UTC](https://community.replicated.com/t/troubleshooting-cluster-networking/568 "2021-08-30T21:24:53Z") **Posts on this page:** 1 **Page:** 1
### Author: ![areed](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.replicated.com/areed/32/12_2.png) [@areed](https://community.replicated.com/u/areed) #### Post date: [August 30, 2021, 9:24pm UTC](https://community.replicated.com/t/troubleshooting-cluster-networking/568/1 "2021-08-30T21:24:53Z")
One of the most common problems enterprises encounter when installing Kubernetes is configuring their systems to be compatible with pod networking. The following steps can help identify the source of networking problems. These steps can also help identify errors detected by the kURL installer’s cluster networking check that runs after the CNI plugin (weave/antrea) is installed. If a problem is detected it will print the error `There appears to be a problem with cluster networking` and may print log lines that include the message `Temporary failure in name resolution`: ```auto Post "http://kurlnet.default.svc.cluster.local:8080": dial tcp: lookup kurlnet.default.svc.cluster.local: Temporary failure in name resolution ``` Usually this is due to a general pod networking problem and is not specific to DNS lookup, but if the below steps do not identify the issue, review the coredns logs with the command `kubectl -n kube-system logs deployment/coredns`. ## Firewalls If no firewall is expected to be running use `systemctl status firewalld` or `systemctl status ufw` to ensure that neither is running. If the firewall is expected to be running, temporarily disable it to determine whether it is responsible for the networking failure. (Note that after stopping your firewall, it may be necessary to restart docker and weave to re-create missing iptables rules: `systemctl restart docker` and `kubectl -n kube-system rollout restart daemonset/weave-net`.) ## IPTables Check IPTables with the `-v` flag to get a count of packets handled by each rule in the first column. Pay close attention to any rules with target `DROP` that have a packet count above 0, as well as to the default policy of each chain. In the example output below, the `INPUT` chain has a default policy of `ACCEPT`, but the last rule is dropping all packets originating from an IP in the default range assigned to pods on kURL clusters. Also note that the FORWARD chain has a default policy of `DROP` but because the packet count for the default policy is 0, it’s not the cause of any networking problems encountered on the server. ```auto $ iptables -L -v Chain INPUT (policy ACCEPT 3193 packets, 533K bytes) pkts bytes target prot opt in out source destination 511K 1583M KUBE-FIREWALL all -- any any anywhere anywhere 653K 3852M sshguard all -- any any anywhere anywhere 0 0 DROP tcp -- any any anywhere localhost tcp dpt:6784 ADDRTYPE match src-type !LOCAL ! ctstate RELATED,ESTABLISHED /* Block non-local access to Weave Net control port */ 6351 450K WEAVE-NPC-EGRESS all -- weave any anywhere anywhere 480K 1573M WEAVE-IPSEC-IN all -- any any anywhere anywhere 202 18610 DROP all -- any any 10.32.0.0/22 anywhere Chain FORWARD (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 99 11996 WEAVE-NPC-EGRESS all -- weave any anywhere anywhere /* NOTE: this must go before '-j KUBE-FORWARD' */ 2 320 WEAVE-NPC all -- any weave anywhere anywhere /* NOTE: this must go before '-j KUBE-FORWARD' */ 0 0 NFLOG all -- any weave anywhere anywhere state NEW nflog-group 86 0 0 DROP all -- any weave anywhere anywhere 2 170 ACCEPT all -- weave !weave anywhere anywhere 0 0 ACCEPT all -- any weave anywhere anywhere ctstate RELATED,ESTABLISHED 0 0 KUBE-FORWARD all -- any any anywhere anywhere /* kubernetes forwarding rules */ 0 0 DOCKER-USER all -- any any anywhere anywhere 0 0 DOCKER-ISOLATION-STAGE-1 all -- any any anywhere anywhere 0 0 ACCEPT all -- any docker0 anywhere anywhere ctstate RELATED,ESTABLISHED 0 0 DOCKER all -- any docker0 anywhere anywhere 0 0 ACCEPT all -- docker0 !docker0 anywhere anywhere 0 0 ACCEPT all -- docker0 docker0 anywhere anywhere ``` ## Sysctl Settings When a server is able to make outbound requests to the Internet from the host, but pods cannot make the same request, it may be that ip forwarding between interfaces is disabled. Run `sysctl -a | grep forwarding` and ensure that the following settings are enabled: ```auto net.ipv4.conf.all.forwarding = 1 net.ipv4.conf.default.forwarding = 1 net.ipv4.conf.weave.forwarding = 1 ``` Also check that bridge interfaces (like weave) are integrated with IPtables: ```auto $ sysctl -a | grep nf-call-iptables net.bridge.bridge-nf-call-iptables = 1 ``` ## Routing Conflicts Ensure that IPs used by servers on the local network do not overlap with the IP range used for pods or services (10.32.0.0/20 and 10.96.0.0/22 by default).