Infra Notes

PodMonitor exists, but Prometheus still does not scrape the target

A quick checklist for debugging PodMonitor issues when the object exists but the target does not appear in Prometheus.

KubernetesKubernetesPrometheusPodMonitorObservabilityDebugging

Problem

PodMonitor is created, but Prometheus does not scrape the target.

Why it happens

Common causes include label selector mismatch, namespace selector mismatch, wrong port name, missing pod labels, or Prometheus not selecting that PodMonitor.

Fix / Approach

Check labels, namespace selectors, pod labels, endpoint port name, Prometheus configuration, and target discovery.

Command / Config

Useful checks:

bash

kubectl get podmonitor -A
kubectl get pods --show-labels
kubectl describe podmonitor <name>
kubectl get svc -A
kubectl port-forward svc/prometheus 9090:9090

When to use

  • When the metrics endpoint works but the target is missing in Prometheus

When not to use

  • When the application itself does not expose metrics