remoteCollector - context deadline exceeded

Hi All,

I am getting most of the time context deadline exceeded error while using remoteCollectors.

I have a cluster with 4 worker nodes and below yaml been tried .

Command used : kubectl preflight test.yaml

even tried with --request-timeout 2m, no luck

Thanks,
Iyappan S

test.yaml

apiVersion: troubleshoot.sh/v1beta2
kind: HostPreflight
metadata:
name: nodes
spec:
remoteCollectors:
- cpu: {}
- kernelModules: {}
- diskUsage:
collectorName: ephemeral
path: /var/lib/kubelet
- ipv4Interfaces: {}
- memory: {}
analyzers:
- cpu:
outcomes:
- warn:
when: “< 6”
message: This server has less than 6 CPU cores, and we require 8, but recommend 12
- warn:
when: “< 8”
message: This server has at least 4 CPU cores, but we recommend 12 or more
- pass:
message: This server has sufficient CPU cores
- kernelModules:
outcomes:
- warn:
when: “target_core_mod != loaded,loadable”
message: The ‘target_core_mod’ kernel module is not loaded or loadable
- warn:
when: “target_core_file != loaded,loadable”
message: The ‘target_core_file’ kernel module is not loaded or loadable
- warn:
when: “tcm_loop != loaded,loadable”
message: The ‘tcm_loop’ kernel module is not loaded or loadable
- warn:
when: “nvme != loaded”
message: The system is not using NVME storage, which will provide better performance
- pass:
when: “target_core_mod,target_core_file,tcm_loop == loaded,loadable”
message: The ‘target_core_mod’, target_core_file’, and ‘tcm_loop’ kernel modules are loaded or loadable
- diskUsage:
collectorName: ephemeral
outcomes:
- warn:
when: “total < 20Gi”
message: /var/lib/kubelet has less than 20Gi of total space
- warn:
when: “available < 10Gi”
message: /var/lib/kubelet has less than 10Gi of disk space available
- warn:
when: “used/total > 70%”
message: /var/lib/kubelet is more than 70% full
- pass:
message: /var/lib/kubelet has sufficient disk space available
- ipv4Interfaces:
outcomes:
- warn:
when: “count == 0”
message: No IPv4 interfaces detected
- warn:
when: “count >= 2”
message: Multiple IPv4 interfaces detected
- pass:
when: “count == 1”
message: IPv4 interface detected
- memory:
outcomes:
- warn:
when: “< 8Gi”
message: At least 8Gi of memory is required
- warn:
when: “< 12Gi”
message: At least 12Gi of memory is recommended
- pass:
message: The system has as sufficient memory

Hi @Iyappan_Sankaran What version of preflight (troubleshoot) are you using?
Also are these preflight checks supposed to be part of your kURL installer?

Hi jdewinne,

Version - $ kubectl preflight version
Replicated Preflight 0.56.0

We are not using with kurl, running as a standalone and deriving the result like

kubectl preflight k8scluster.yaml --interactive=false

The filesystemperformance always failing while collecting data. User has the permission to write.

Is the preflight internally uses any container for collecting the data?

Thanks,
Iyappan S

Thanks. I don’t fully understand your use case about running a preflight that checks the host from within the k8s cluster? How would you know which node it will run on?

Are you looking for host collectors and analyzers?

@lyappan_Sankaran the remote host collector feature is one we haven’t developed much to this point, and the future of it is unknown, so not many around who can debug it unfortunately. Would you mind opening an issue for this on Issues · replicatedhq/troubleshoot · GitHub for now? I’ll provide an update for you by end of day tomorrow what are plans are here. If the outcome is to keep the feature around, we can treat this as a bug as well as start getting some proper docs in place to assist with its usage.

1 Like

@lyappan_Sankaran Sorry for the late update here. We haven’t quite decided if the remote host collector feature is going to stick around or not. In the meantime, best course of action will be for you to open an issue on the Troubleshoot Github repo.