Why monitor CoreDNS?

CoreDNS is DNS server that can serve as the Kubernetes cluster DNS (Kubernetes 1.23+ uses CoreDNS by default). CoreDNS is a critical component of a Kubernetes cluster that can be difficult to debug during an error scenario. Surfacing monitoring to this component can help teams respond faster to these unexpected scenarios.

CoreDNS quickstart highlights

The New Relic quickstart uses dashboards to proactively monitor your CoreDNS servers, like:

• load shared amongst running instances
• request & response stats (rate, payload size, etc.)
• cache hit ratio
• response codes and panics, and more.

Monitoring CoreDNS

This quickstart utilizes New Relic's ability to ingest Prometheus data (either from our OpenMetrics integration or via Prometheus remote write). See instructions for sending Prometheus data to New Relic here.

How do I enable monitoring in CoreDNS

CoreDNS provides a plugin to surface Prometheus metrics on localhost:9153/metrics. Before attempting to modify your CoreDNS configuration, you should be able to kubectl port-forward pod/<pod_name> -n kube-system 9153 against a CoreDNS server in your cluster to verify it returns metrics.

For instructions on modifying your cluster's CoreDNS configuration see https://coredns.io/2018/01/29/deploying-kubernetes-with-coredns-using-kubeadm/

How do I configure my integration to scrape CoreDNS severs?

The approach to getting your Prometheus metrics into New Relic differs depending on which integration you use.

In addition to setting up your remote_write configuration as described here, you will need to add the following scrape configuration to your prometheus.yml config file:

(This helpful Prometheus job defnition sourced from sysdig blog)

Then, after reloading Prometheus config, you can check that your CoreDNS pods are appearing under your Prometheus targets.

POMI will scrape any resource that contains the label or annotation prometheus.io/scrape (which is configurable value here https://github.com/newrelic/helm-charts/blob/ecef47fc938b7ddca8a50e63cb290924f654c56a/charts/nri-prometheus/values.yaml#L117). This will make targets visible to POMI but to confiugre the relabel_configs as above, the defaults would have to be updated in the Helm chart here: https://github.com/newrelic/helm-charts/blob/ecef47fc938b7ddca8a50e63cb290924f654c56a/charts/nri-prometheus/values.yaml#L67-L244