Google Cloud bakes AI inference into Pub/Sub and removes the intermediary microservice
The Pub/Sub AI Inference SMT reaches general availability: it applies a Gemini Enterprise Agent Platform model to every message in flight. For real-time enrichment you no longer need a separate inference service.
Early September 2026. Google Cloud’s “What’s new” page announces general availability of the AI Inference SMT in Pub/Sub. SMT stands for Single Method Transform. September 2026. The messaging bus — historically a dumb pipe — becomes an inference point. Why it matters: to enrich an event stream with a prediction, a classification, or an anomaly score, you no longer need to deploy and operate an intermediary inference microservice — the bus does it for you.
What the AI Inference SMT does
An SMT transform is a function Pub/Sub runs on a topic or a subscription. The AI Inference SMT applies that logic to inference: it sends the message to a model hosted on Gemini Enterprise Agent Platform, then appends the prediction to the message before delivering it downstream — to BigQuery, Bigtable, or any other consumer.
The usable models cover two cases. Self-deployed models: open, partner, or custom models deployed to a shared or dedicated Agent Platform endpoint. Model-as-a-Service (MaaS) models via Model Garden — Gemini, Claude, or the Anthropic, Mistral AI, and AI21 models, without managing the deployment. In practice, the message must be a JSON request to the model; the transform merges extra parameters (for example {"temperature": 0.2}) with the message data, calls the endpoint, and returns an enriched message structured as {"original_message": …, "model_output": …}.
The argument: three links fewer
The architectural value is what you no longer have to build. Without this transform, a real-time enrichment pipeline forces an intermediary service: a consumer that reads the topic, calls the model, and republishes the result. That is a service to deploy, scale, and monitor, plus an extra network hop on every event.
The AI Inference SMT removes those three links. Fewer services: the bus calls the model itself. Lower latency: removing intermediary hops shortens the end-to-end path. Less overload risk: Pub/Sub optimizes the request rate toward the endpoint so you do not saturate it — flow control you no longer have to write.
For anomaly detection, the use case Google Cloud calls out, the posture shift is real: a model’s prediction is attached to the event as it passes through the bus, available to downstream processing with no dedicated service in between.
Topic or subscription: the choice matters
The transform is configured in two places, and the difference is not cosmetic. On a topic, inference happens before the message is stored: on failure, the publish request itself fails, directly affecting your publisher’s availability. On a subscription, inference happens before delivery to the consumer: on failure, Pub/Sub retries according to the subscription’s retry policy and can forward the original message to a dead-letter topic.
Google Cloud recommends the subscription. The reason is operational: if the model endpoint is throttled, hits latency spikes, or goes down, the impact on the pipeline is far more contained on the subscription side, where ingestion stays available and a dead-letter preserves the data. On a topic, model downtime becomes ingestion downtime. Either way, a dead-letter topic is recommended: if inference fails, the original message is forwarded to the dead-letter with a CloudPubSubDeadLetterSourceSMTErrorMessage attribute — no data is lost.
The same week, the streaming layer matures
The AI Inference SMT does not arrive alone. In the same September window, Google Cloud generalized several pieces of its streaming data layer: the PostgreSQL source connector for Managed Service for Apache Kafka is now generally available (capturing changes into Cloud SQL, AlloyDB, or self-managed PostgreSQL), Dataflow’s pause-on-failure is GA (freezing a batch job before failure so you can resume without losing completed work), and BigQuery’s legacy insertAll API is rebranded as the Storage Write API (REST), fully backward compatible.
The common thread is legible: the streaming layer stops being a mere transport and becomes the place where you transform, infer, and survive failures — without leaving the flow.
Chain, pre-process, and watch the cost
The AI Inference SMT combines with other transforms. A JavaScript UDF placed upstream can reformat the message to fit the model’s expected input — for example a Chat Completions call — and a second one downstream can post-process the output before delivery. That is the answer when your publisher does not natively produce the required JSON structure.
The open question is cost. Every enriched message triggers a model call, billed according to the model you choose: on a high-volume stream, per-event inference can cost more than the pipeline it simplifies. Pub/Sub’s flow control limits calls to avoid saturating the endpoint, but it does not reduce their number — it spreads them out. The comparison is therefore not “transform or dedicated service” but “managed transform or Cloud Functions/Dataflow”: for inference on every event, the transform wins on simplicity; for selective inference triggered by a business condition, a separate function stays cheaper.
Limits to know before enabling it
The transform has documented boundaries worth folding into the design. Message size: the transformed message, enriched with the model output, must stay under the Pub/Sub message-size limit, or the transform fails. Regional constraint: for a pull subscription, the request must be issued from the endpoint’s region or it is rejected — the docs recommend a locational endpoint. Unary pull: flow control is not supported with the unary pull API. And quotas: the transform is subject to the Agent Platform endpoint quotas; flow control adjusts the pace but cannot exceed the model’s quota.
Those boundaries define the right use. The transform excels at uniform enrichment — every event receives a prediction — and at continuous anomaly detection. It fits poorly for conditional inference, where only a few events deserve a model call: there, a dedicated function that filters first stays cheaper, and the transform becomes a fixed cost rather than a gain.
One operational caveat follows from moving inference into the bus: the model’s behavior — its latency, its error rate, its quota consumption — now lives inside Pub/Sub rather than in a service you own. Treat the model endpoint as a dependency of the bus and monitor it accordingly, because a degraded model becomes a degraded pipeline.
Verdict
The AI Inference SMT is a quiet but structural change: it moves inference out of the dedicated service and into the bus, removing the most expensive part of a real-time enrichment pipeline.
If you are on Pub/Sub and already enrich events with a model, adopt it: you delete a service, a network hop, and flow-control code. If your model endpoint is fragile or saturated, configure it on a subscription, never on a topic, and always attach a dead-letter topic. If you have no real-time inference today, that is not a reason to add it: the transform only earns its keep where a per-event prediction has a downstream consumer — otherwise you pay for model calls that go nowhere.