Google ships HEIR, the compiler that runs AI inference on encrypted data

2023. Google announces its intent to expand its fully homomorphic encryption offering. August 14, 2026. Jeremy Kun’s post on blog.google presents HEIR as “the latest powerful tool added to our Private Computing Toolkit.” The same post publishes four private-inference applications compiled with HEIR, with source code on GitHub.

The message fits in one sentence: running a model on encrypted data is no longer a cryptography-team problem — it has become a compiler problem. That is exactly what HEIR — short for Homomorphic Encryption Intermediate Representation — claims to solve, and what Google is now demonstrating with concrete workloads.

What HEIR changes: homomorphic encryption becomes a compiler problem

Homomorphic encryption (FHE) lets you compute directly on ciphertexts: the server processes encrypted data and returns an encrypted result, without ever seeing the plaintext input. The guarantee is purely cryptographic — unlike hardware enclaves (TEEs), whose security rests on trust in a silicon vendor, FHE trusts no hardware at all.

The historical blocker was never the cryptography; it was the engineering. Converting an existing program to run homomorphically required, in Google’s words, “a team of cryptographers”: choosing a scheme (BFV, BGV, CKKS…), managing the noise that accumulates with every operation, tuning parameters (polynomial degree, modulus), and generating code against a library. Each step is a specialty in its own right.

HEIR absorbs that entire chain. Built on MLIR, the modular compiler infrastructure from the LLVM world, HEIR describes inference as an intermediate representation that passes transform into a form executable against backends such as OpenFHE or Lattigo. The developer no longer writes the cryptography: they annotate which arguments are secret, and the compiler handles scheme selection and code generation. It is the same move that turned GPU programming into a compiler problem twenty years ago — the complexity shifts from the developer to the toolchain.

The trajectory is long, and Google says so. The intent dates back to 2023; since then, the company has partnered with makers of hardware accelerators for FHE — Belfort, Niobium, Cornami, Optalysys — and HEIR has served as a platform for four peer-reviewed publications, with academic collaborations at Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, Edinburgh and Tsinghua. The project matured quietly before being shown.

Four demos: what already runs encrypted

Google’s post does more than promise: it publishes four applications compiled with HEIR, each with latency numbers on a single-threaded CPU, and the source code in the google/fully-homomorphic-encryption repository.

• Content recommendation. A Deep Learning Recommendation Model serves private recommendations — the exact demo mentioned above, where the provider recommends without ever seeing the user’s features. Joint work with Belfort Labs, LG and NYU.
• Credit-card fraud detection. A fraud detector compiled with Niobium and hardshell.ai that classifies a transaction without the server reading its attributes.
• Threat intrusion. The Kitsune anomaly-detection system, compiled to detect anomalies in encrypted network traffic — the provider sees an alert, never the packet contents.
• Hotword detection. A hotword detector compiled with Belfort Labs that would let a voice-triggered agent recognize an activation word without exposing the audio recording.

What the four cases have in common is instructive: they are all narrow, structured models — a recommender, a classifier, an anomaly detector — not 27-billion-parameter transformers. Today’s FHE does not encrypt an LLM; it encrypts the triage layer around one.

# The compiler and the official examples, both on GitHub
git clone https://github.com/google/heir
git clone https://github.com/google/fully-homomorphic-encryption

Why this replaces neither the TEE nor the plaintext LLM

Read the announcement without over-selling it, because the post itself sets the boundaries. FHE carries “a nontrivial cost overhead”: encrypted inference remains orders of magnitude slower than plaintext, and the published numbers are on a single CPU thread — the slowest possible configuration — pending demonstrations of the accelerator gains.

The comparison with enclaves is nuanced too. A TEE offers strong performance, but its security is hardware-bound — dependent on a vendor, and it has been bypassed before. FHE offers a mathematical guarantee at a performance cost that is prohibitive for many workloads. The two techniques are not rivals; they occupy different segments of the same confidentiality spectrum.

And there is the attack surface this kind of tooling addresses. FHE promises to process data without seeing it — which, for a CISO, means the data no longer leaves the perimeter in plaintext. It is precisely the answer to the risk that shadow AI made visible in 2026: employees pasting sensitive data into unsanctioned SaaS models, as documented here this summer. FHE does not remove the governance problem — it offers a path where inference and confidentiality stop being a trade-off.

A compliance question before a performance question

For teams handling healthcare or financial data, FHE answers a regulatory question first: how do you run useful inference without the data changing hands? GDPR, sectoral rules and sovereignty clauses often require that certain data never leaves the organization’s perimeter — which rules out sending it to an external SaaS model in the first place.

That is where HEIR changes the nature of the problem. Until now the alternative was binary: process locally (and forgo the power of a cloud-served model) or anonymize (and lose accuracy). FHE opens a third path: the data stays encrypted end to end, the model runs wherever you want, and only the key holder can read the result.

The guarantee is also stronger than enclaves on one precise point: it depends on no hardware vendor. For a CISO who must justify decisions before a regulator, “the security is mathematical, not contractual” is an argument that carries weight — provided you accept the latency price.

The real constraint: latency, not cryptography

The operational takeaway comes down to one question: what is the price of a mathematical confidentiality guarantee on your triage models? The answer depends entirely on the workload.

For a fraud classifier or a recommender, homomorphic latency is still measured in seconds where plaintext is measured in milliseconds — unacceptable for real time, tolerable for batch or pre-filtering. For an LLM, the question does not even arise: the compute volume makes FHE unthinkable in the near term.

That is nonetheless the point Google is making: the cost of FHE is falling fast, and it has been turned into an engineering problem — industrializable, optimizable by the hardware accelerators already on board, and measurable. A problem you can compile is a problem whose cost curve you can chart.

Verdict

If you operate in a regulated sector — healthcare, finance, sovereign data — that today cannot send certain data to a model at all, HEIR removes a real constraint: inference on encrypted data stops requiring a cryptography team. Track the project and run the official demos to gauge latency on your own data.

If you need an LLM or real-time inference, build nothing on FHE today: the latency remains disqualifying. Your confidentiality rests on other bricks — enclaves, controlled retention, self-hosting — that HEIR does not replace.

The bigger signal is that inference confidentiality is becoming a compiler problem rather than a cryptography problem. That is what a niche technology looks like right before it industrializes — and the moment to map its use cases is now, not once the accelerators have driven the latency down.

References

• Google — How Google is Making Private AI Practical with Homomorphic Encryption, Jeremy Kun, August 14, 2026
• HEIR — Homomorphic Encryption Intermediate Representation
• GitHub — google/heir: A compiler for homomorphic encryption
• GitHub — google/fully-homomorphic-encryption
• arXiv — HEIR: A Universal Compiler for Homomorphic Encryption