Google open-sources HEIR, a compiler for encrypted AI inference

Google has released HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler that converts pre-trained AI models to run inference directly on encrypted data. Homomorphic encryption lets a server compute on ciphertexts and return encrypted results without ever seeing the underlying inputs, which sidesteps the usual privacy trade-off: end-to-end encryption protects user data but blocks server-side features like spam detection, while shipping models to local devices risks leaking proprietary IP and hits device limits. The catch has always been cost overhead and the need for a team of cryptographers to hand-tune each program — HEIR targets the second problem by aiming to make encrypted inference a near one-click step for non-experts.

Since Google signaled its intent in 2023, HEIR has drawn in hardware-accelerator partners (Belfort, Niobium, Cornami, Optalysys) and become a research platform underpinning collaborations with Georgia Tech, CMU, UC Santa Barbara, Purdue, Edinburgh, Tsinghua and others, plus four peer-reviewed papers. To show maturity, Google shipped four compiled demos with single-threaded CPU latency figures and source on GitHub: a deep-learning recommendation model, credit card fraud detection, the Kitsune network-anomaly detector running over encrypted traffic, and a hotword detector for audio agents.

The significance is less about a cryptographic breakthrough than about tooling and momentum. Homomorphic encryption’s guarantees are purely cryptographic rather than dependent on trusted hardware or enclaves, making it attractive for tightly regulated sectors like healthcare and finance where data-sharing rules are strict. If accelerators keep driving latency down, HEIR could lower the barrier enough to move encrypted inference from research demos toward production use — the remaining question is whether the cost overhead falls fast enough to matter at scale.