CloudWatch now sees every GPU in your EKS fleet and prices the waste

November 2026. AWS introduces CloudWatch accelerated compute insights at KubeCon North America 2026. November 12, 2026. The capability is available in every Region where CloudWatch Omni already is. Why it matters: GPUs have become the most expensive resource in modern computing, and most organizations still run them blind — unable to say who is using which card, or what the waste is costing them.

The most expensive blind spot in the budget

A single training cluster can run for days and cost millions of dollars. To earn that capital back, teams pool accelerators across shared clusters. But in those shared environments, platform teams cannot answer three basic questions: which team is using which accelerator, is an inference workload generating tokens efficiently, and what is wasting money.

The symptom is well known. When a workload degrades — a failing accelerator, a thermally throttled node — teams only find out after a run crashes or the bill lands. The device-level signals needed to isolate the cause (request traces, kernel logs, memory metrics from the card itself) are neither collected nor connected. With no way to point at one accelerator, the default response is to cycle the whole instance: seven healthy GPUs are sacrificed to deal with one faulty card, followed by a wait for scarce capacity.

That is exactly the gap the new capability targets. Its pitch fits in one sentence, quoted by AWS in the announcement: “GPUs have become the most expensive resource in modern computing, and yet most organizations operate them with little visibility.”

What accelerated compute insights does

Accelerated compute insights delivers a single operational view of every accelerator in the fleet. Concretely, it shows who is using the GPU and AWS AI Accelerator capacity across Amazon EKS, detects failures and bottlenecks as they happen, estimates the financial impact of each problem, and links those findings to the actions that resolve them.

The novelty is in bringing together three sources that conventional monitoring tools never cross: accelerator telemetry, workload boundaries, and AWS pricing data. That join is what turns a raw metric into a budget question.

Every accelerator in the cluster is enriched with Kubernetes metadata: pod, namespace, node, node group, and custom labels (team, application). The result is per-team and per-workload utilization across the entire fleet — finally answering the “who is using what” question.

One design detail deserves attention: automatic detection of MIG (Multi-Instance GPU) partitioned GPUs. The capability also recognizes other partitioning schemes, including Kubernetes DRA (Dynamic Resource Allocation) and time-slicing, by reading device labels. No configuration is required.

Three accelerator families, one view

On EKS, the capability covers NVIDIA GPUs, AWS Trainium, and AWS Inferentia. Metrics from NVIDIA and Inferentia cards are normalized into a common schema — through a layer called the Unified Accelerator Health Layer — and presented in a single Accelerator Health view. A team running a mixed fleet therefore manages every accelerator type in one place, without a dashboard per vendor.

The pricing join produces three categories of recoverable waste:

• Training efficiency gaps: low tensor-core (MFU) utilization relative to the workload’s own historical baseline, a sign the job is not getting full value from the hardware;
• Inference below capacity: endpoints serving fewer requests than the hardware supports, correlated with device utilization;
• Idle resources: capacity that is allocated but not consumed.

For each finding, the capability identifies the workload or node responsible. One important caveat, stated by AWS: this is an operational tool, not a billing tool. The figures are estimates meant to guide decisions, and the authoritative source remains AWS Cost Explorer and the Cost and Usage Report.

From alert to repair, via the DevOps Agent

Health events can be routed to the AWS DevOps Agent for deeper investigation. The agent reasons over accelerator telemetry, CloudWatch logs, recent infrastructure changes, job configuration, and any runbooks you have authored. It distinguishes failure modes that call for different actions: a node with an accelerating memory-error trend that needs replacing, versus a driver issue that clears with a restart.

By default the agent presents an evidence-backed action plan for approval. For a faster loop, you can opt in to let it carry out well-understood remediations through AWS APIs on its own. This is the logical extension of the CloudWatch Omni approach we covered earlier for observing AI agents — see our dedicated analysis.

Enabling it in minutes

The install path is short, provided you start from the official add-on. If you run self-hosted inference on EKS, install or upgrade the Amazon CloudWatch Observability EKS add-on to v7.0.1 or above. The add-on discovers every NVIDIA GPU and Inferentia device in the cluster and starts emitting OpenTelemetry data, with no per-node setup and no pipelines to configure.

helm repo add aws-observability https://aws-observability.github.io/helm-charts
helm install amazon-cloudwatch-observability \
  aws-observability/amazon-cloudwatch-observability \
  --namespace amazon-cloudwatch --create-namespace

Once telemetry is flowing, open CloudWatch Omni. The view there is application-centric rather than infrastructure-centric: inference workloads appear next to the services and AI agents that call them, with each accelerator’s health, utilization, and waste tied to the workload. When an endpoint shows low throughput or a high time to first token, Omni traces the cause down to the workload, node, or single card responsible — and shows what to change, without switching tools.

The capability completes a foundation laid through 2025 and 2026: OTel Container Insights (GA in June 2026) for infrastructure metrics, Application Performance Monitoring enabled by default (February 2026), and container network observability (November 2025). The accelerator layer was the missing piece; it is now in place.

What it changes versus the DCGM + Prometheus setup

Plenty of teams already export their card metrics through NVIDIA’s DCGM exporter into Prometheus and Grafana. That setup, now a de facto standard, solves collection: utilization, memory, temperature, per GPU. What it does not solve is the rest of the chain.

Accelerated compute insights differs on four things the DCGM + Prometheus pair does not join by default: Kubernetes metadata (who consumes what, per team), automatic partitioning detection (MIG, DRA, time-slicing), the join with AWS pricing that turns a metric into dollars, and guided remediation through the DevOps agent. Collection was already a solved problem; what was not solved is correlating the metric with the tenant and the cost.

The positioning is therefore simple. If you already run a mature DCGM + Grafana stack, the added value concentrates on the pricing/waste layer and automated remediation — the rest you already have. If you have nothing, accelerated compute insights is the fastest path, because it delivers collection, cross-vendor normalization, and application-level correlation in one move.

Verdict

If you run GPUs or Trainium/Inferentia on EKS — especially if you pool them across teams — install the CloudWatch Observability add-on v7.0.1 and turn on accelerated compute insights before your next cost review. It is the first tool that joins per-device telemetry, workload boundaries, and pricing in one view, and that is exactly what it takes to replace the “cycle the whole instance” reflex with “fix the faulty card.” If you have no accelerators in production yet, just note the dependency for the day you do — and don’t pay for GPU observability your fleet doesn’t justify yet.

References

• The New Stack — “Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights” (November 11, 2026)
• ETTAYEB — “AWS ships CloudWatch Omni to observe and evaluate AI agents”
• ETTAYEB — “CloudWatch centralizes multi-account logs without a homegrown pipeline”