Kots admin console graphs

Packaging an application
1

I tested customising the graphs on the kots admin console this morning, documented here:

When changing the kots-app.yaml to add a graphs section the CPU and Memory Usage charts were blank and viewing the logs from the kotsadm pod there were lots if messages:

{"level":"error","ts":"2022-09-08T10:10:11Z","msg":"failed to prometheus query range: Unexpected status code 400"}

Narrowing things down it looks like there is an issue with the usage of fmt.sprintf in the query syntax.

by temporary hard coding the namespace I was able to recreate the original charts, note the two commented out queries in CPU Usage will give the 400 return status/error code:

   graphs:
     # CPU, Memory and Disk Usage charts
     - title: CPU Usage
       #query: fmt.Sprintf(`sum(rate(container_cpu_usage_seconds_total{namespace="%s",container!="POD",pod!=""}[5m])) by (pod)`, util.PodNamespace)'
       #query: 'fmt.Sprintf(`sum(rate(container_cpu_usage_seconds_total{namespace="%s",container!="POD",pod!=""}[5m])) by (pod)`, "default")'
       query: 'sum(rate(container_cpu_usage_seconds_total{namespace="default",container!="POD",pod!=""}[5m])) by (pod)'
 
       legend: '{{ pod }}'
     - title: Memory Usage
       queries:
         - query: 'sum(container_memory_usage_bytes{namespace="default",container!="POD",pod!=""}) by (pod)'
           legend: '{{ pod }}'
       yAxisFormat: bytes
     - title: Disk Usage
       queries:
         - query: 'sum((node_filesystem_size_bytes{job="node-exporter",fstype!="",instance!=""} - node_filesystem_avail_bytes{job="node-exporter", fstype!=""})) by (instance)'
           legend: 'Used: {{ instance }}'
         - query: 'sum((node_filesystem_avail_bytes{job="node-exporter",fstype!="",instance!=""})) by (instance)'
           legend: 'Available: {{ instance }}'
       yAxisFormat: bytes

Anyone get the sprintf working, what am I missing?, ty

2

I don’t think you can make use of fmt in the go templating part. (I would be super happy if I’m wrong on this one).
Haven’t tried this one, but maybe this works? (I’m not sure even if go templating is supporting in graph queries)

'sum(rate(container_cpu_usage_seconds_total{namespace="{{repl Namespace}}",container!="POD",pod!=""}[5m])) by (pod)'
3

Thanks @jdewinne, I tested the {{repl Namespace}} suggestion this morning and it doesn’t give the 400 return status, which is good, but the chart is still blank.

4

Did it resolve the {{repl Namespace}} or did it just keep it as is and not replace it?

5

It ends up being null at the prometheus end. I tried enabling DEBUG on kotsadm as per @adamancini 's post but no new messages related to the graphs unfortunately. Might be because I’ve not specified a non-default ns for the app, will try that next

6

Just to update this thread, the {{repl Namespace}} is still coming through blank when kotsadm and the app are deployed in a non-default namespace.
Tried a few other variants like {{repl ConfigOption “app_ns”}} but that gets a 400 from prometheus so guess it’s passing through un-substituted. Odd as it is used elsewhere in the same kots Application yaml file

7

Interesting. I think there was an issue recently solved that made more things templateable than before in the kots.io/v1beta1 Application kind. I wonder if the graphs item didn’t the cut. I’m assuming you’re using a pretty recent version of KOTS.

8

Yes, kotsadm/kotsadm:v1.84.0