5.1 Tasks: kube-prometheus metrics

Task 5.1.1: Memory usage of Prometheus

Task description:

  • Display the memory usage of both Prometheus pods
  • Use a filter to just display metrics from your prometheus containers

Use the [Thanos Querier web UI](https://<user>-thanos-query.training.cluster.acend.ch to execute the queries.

Hints
container_memory_working_set_bytes{pod=~"prometheus-prometheus-0", container="prometheus",namespace=~"<user>.*"}

Task 5.1.2: Kubernetes pod count

Task description:

  • Display how many pods are currently running on your Kubernetes platform
Hints

There are different ways to archive this. You can for example query all running containers and group them by pod and namespace.

count(sum(kube_pod_container_status_running == 1) by (pod,namespace,cluster))

You may also sum() all running pods on your Kubernetes nodes

sum(kubelet_running_pods)

Task 5.1.3: Display metrics in Kubernetes Grafana

Let’s try to inspect some cluster metrics in Grafana. Of course we could build our dashboard from scratch again and feel free to try, but in most cases there will always be someone who has already built a dashboard for your purpose. As we already have seen grafana has a searchable dashboard database to get prebuilt dashboards from. Currently we are interested in resource usage of our kubernetes namespace.

Task description:

  • Open and copy the id or json of the Kubernetes / Views / Namespaces dashboard
  • Navigate to your Kubernetes Grafana
  • Create a new dashboard in your Grafana instance and choose ‘Import’
  • Paste the Json or Id from the dashboard and choose ’thanos-querier’ as your datasource

Inspect the dashboard for your namespace <user>, you will find metrics and visualizations for most important metrics. You can inspect and edit panels for your purpose.

Task 5.1.4: (Optional) Kubernetes Dashboards

There are a whole bunch of Kubernetes Dashbaords available, which you can find here: https://github.com/dotdc/grafana-dashboards-kubernetes/tree/master/dashboards

If you have enough time or are interested in more Kubernetes Dashboards go a head and import those as well.