5.1 Tasks: kube-prometheus metrics
Task 5.1.1: Memory usage of Prometheus
Task description:
- Display the memory usage of both Prometheus pods
- Use a filter to just display metrics from your
prometheuscontainers
Use the [Thanos Querier web UI](https://<user>-thanos-query.training.cluster.acend.ch to execute the queries.
Note
Search for a metric withmemory_working_set in its nameHints
container_memory_working_set_bytes{pod=~"prometheus-prometheus-0", container="prometheus",namespace=~"<user>.*"}
Task 5.1.2: Kubernetes pod count
Task description:
- Display how many pods are currently running on your Kubernetes platform
Hints
There are different ways to archive this. You can for example query all running containers and group them by pod and namespace.
count(sum(kube_pod_container_status_running == 1) by (pod,namespace,cluster))
You may also sum() all running pods on your Kubernetes nodes
sum(kubelet_running_pods)
Task 5.1.3: Display metrics in Kubernetes Grafana
Let’s try to inspect some cluster metrics in Grafana. Of course we could build our dashboard from scratch again and feel free to try, but in most cases there will always be someone who has already built a dashboard for your purpose. As we already have seen grafana has a searchable dashboard database to get prebuilt dashboards from. Currently we are interested in resource usage of our kubernetes namespace.
Task description:
- Open and copy the id or json of the Kubernetes / Views / Namespaces dashboard
- Navigate to your Kubernetes Grafana
- Create a new dashboard in your Grafana instance and choose ‘Import’
- Paste the Json or Id from the dashboard and choose ’thanos-querier’ as your datasource
Inspect the dashboard for your namespace <user>, you will find metrics and visualizations for most important metrics. You can inspect and edit panels for your purpose.
Task 5.1.4: (Optional) Kubernetes Dashboards
There are a whole bunch of Kubernetes Dashbaords available, which you can find here: https://github.com/dotdc/grafana-dashboards-kubernetes/tree/master/dashboards
If you have enough time or are interested in more Kubernetes Dashboards go a head and import those as well.