Updates on HEIR, the homomorphic encryption compiler project

On 2026-08-14 I published an article on the Google Security blog with an update on HEIR, our homomorphic encryption (HE) compiler. This is a companion article, in which I have no limits on word count or jargon, and I can feel free to be honest. So strap in.Assuming you won’t read the linked corporate blog post, HEIR is a compiler that converts an input program to a program that operates directly on encrypted data. The guarantee of homomorphic encryption is that, assuming you haven’t cracked the cryptography, at no point does the computer running the program get even a single bit of information about the cleartext data used to generate the encrypted inputs. No information about the inputs, outputs, or any intermediate values.1The blog post focuses on HEIR’s ability to compile pre-trained ML models, and gives four examples of small, but nontrivial models that it can compile. Hence, homomorphic encryption can enable services to provide perfectly private inference. I’ll try to say more about when and where this is useful later in this article. First I wanted to give a more concrete sense for how HEIR works in the context of these examples, and outline (my view on) the project’s roadmap for the future. I won’t do a deep dive on HEIR’s internals by any means, since that would make the article too long. Give me a shout if you want that, but there are plenty of docs to read through at heir.dev and you can see a recent (fast-paced) talk I gave at ASPLOS this year.Table of Contents:The repo behind the blog postThe blog post ends with a list of examples compiled with HEIR. Those examples point to a GitHub repository2 that you can clone and run yourself. The biggest hurdle is installing bazel, and then bazel hermetically manages everything else.3Some simple runtime comparisonsThe simplest and fastest example to try is the credit card fraud detector. This is a simple three-layer feed-forward network with sigmoid activations, trained on a Kaggle dataset. The linear layers have dimensions 128, 64, and 2 (the last being the logits for the two classes, fraud and not-fraud).You can run the basic example in one line:bazel run -c opt //demos/cc_fraud/lattigo:evaluate_fhe This command will compile the (pre-trained, checked-in) cc_fraud model to the Lattigo backend, and then run it on a sample input. The command above outputs:Loading test row 0 from /home/jeremy/fully-homomorphic-encryption/demos/cc_fraud/data/test_rows.csv... Took 83.226µs Expected label (is_fraud): 0 Feature vector size: 82 First 5 features: [-0.31676582 0.85089076 -0.40874073 -0.1833772 -1.7155787] Configuring Lattigo context... Took 2.164051998s Encrypting input features... Took 19.430069ms Running preprocessing... Took 573.537941ms Running FHE evaluation (preprocessed)... Took 2.020821739s Decrypting output... Took 488.237µs Decrypted logits: [16.464235 -16.781752] Predicted class: 0 SUCCESS: Predicted class matches expected label! The central point here is that the evaluation of the model on encrypted inputs took about 2 seconds on a single-threaded CPU.Compare this to the same execution on cleartext inputs, noting that this is the latency of a single inference, so it doesn’t benefit from amortization. (This requires fetching and encoding the original dataset, which is explained in the README; I’ll skip that part here).$ bazel run -c opt //demos/cc_fraud/cleartext:evaluate_cleartext Loading model from: demos/cc_fraud/data/mlp_fraud_model_sigmoid.pt Evaluating Credit Card Fraud Sample Index: 0 True Label: 0 (LEGITIMATE) Predicted Label: 0 (LEGITIMATE) Fraud Probability: 0.000000 Result: CORRECT Latency: 0.5233 ms Anyone who has heard of HE may have heard that it is slow, but I want to pause here to compare this (single-threaded CPU, non-amortized!) runtime: 2 seconds for HE inference vs 0.5 ms for cleartext. This is a 4,000x slowdown, and the computation involves two matrix-vector products (where the matrix is not private), with two evaluations of a sigmoid function.There are many caveats to this demo worth briefly noting:This model is small enough that the private information fits in a single (CKKS) ciphertext. Inputs tensors with more than, say, 32k elements will require multiple ciphertexts and naturally more overhead.This model is small enough that bootstrapping (the slowest part of HE) is not required.I did not include a simulated network overhead, nor did I include the per-user one-time setup required to generate and upload the relevant key material that makes HE work.The server is doing a decent amount of (one-time, model-specific) pre-computation in the “Running preprocessing” step.The other examples in the repo are more complex, and hence have longer latencies and worse overhead vs cleartext (and a memory requirement of 60-90 GiB). In particular:network_anomaly, an ensemble of auto-encoders: 30 seconds for inference.criteo, a recommender model adapted specifically for HE: 5 minutes for inference.hotword, a 10-layer convolutional network: 20 minutes for inference.4This sounds bad, but remember it’s single-threaded CPU execution. Our colleagues working to integrate HEIR with GPUs have reported that the criteo workload runs in ~500ms on a single GPU (similar to an H100). Compare that to the criteo/cleartext:evaluate_cleartext demo which runs in 10ms, and you’re down to a 50x slowdown (again, the baseline is non-amortized CPU execution). That work wasn’t able to make it into the Google blog post, but they are compiled by HEIR with some pending upstream PRs. My point is that the execution times are continuing to improve, and for some small problems they could be called reasonable if you squint. And this does not even breach the topic of HE accelerated by FPGAs and ASICs, which are even more promising performance-wise.So instead of the Google corporate blog post showcasing raw performance, it was meant to showcase the expressiveness of HEIR: it can compile a lot of models, and the performance on some of them is decent.As far as showcasing features, the repo also shows how one might use HEIR to:Target and compare multiple HE backends (the repo has Lattigo and OpenFHE as examples, try bazel run -c opt //demos/cc_fraud/openfhe:evaluate_fhe.Explore per-layer timing: HEIR has the ability to insert debugging callbacks into the compiled program, try bazel run -c opt //demos/cc_fraud/lattigo:evaluate_fhe_timing. In that command you’ll see the first linear layer takes the majority of the runtime, ~1.2 seconds out of 2 seconds total.Explore an incorrect inference: Similar to timing, the debug callback can decrypt ciphertexts and inspect them for correctness or precision loss. Try bazel run -c opt //demos/cc_fraud/lattigo:evaluate_fhe_debug and you’ll see output that shows that by the end of the inference, the use of HE has caused about 2 bits of precision loss compared to the cleartext model. This precision loss can be tuned with compiler flags and trades against performance. Unfortunately, navigating that trade-off requires HE expertise.Getting a model into HEIRThe process of getting a pre-compiled model to be something that HEIR can process is not yet automated. The main two constraints are:You need to be able to convert the pre-compiled model to MLIR, which is the intermediate representation that HEIR uses to represent programs. Many ML frameworks such as PyTorch and JAX have tools to convert to MLIR.You need to manually annotate your program with HEIR-specific annotations that say (a) what inputs to the inference are secret and (b) what are bounds on the input ranges to each activation function.The demos in the repository show how to do this for PyTorch, and I’m working with the maintainers of torch-mlir to add a feature that will enable me to automate this (given a validation set to use to estimate ranges).That said, much of the early stages of the compiler pipeline (recognizing activations, fusing linear layers, etc.) is based on how torch-mlir exports models to MLIR, so if we want to add ONNX or JAX support (both have great MLIR exporters), it will likely not work out of the box just yet. Moreover, we don’t even have complete coverage of torch operators yet. A lot of the details of how to support linear algebraic operators in HE are both tricky and active research topics (for MLIR enthusiasts, we don’t support linalg.generic in full generality).Invoking the compilerThis is a bit of a tangent, but if you look at the build rules for the examples in the repo, you’ll see some slightly messy calls to a macro that invokes the compiler.load("@rules_heir//heir:lattigo.bzl", "heir_lattigo_lib") HEIR_OPT_FLAGS = [ "--annotate-module=backend=lattigo scheme=ckks", "--torch-linalg-to-ckks=min-slot-count=8192 greedy-level-budget=15 greedy-modulus-switch-after-mul=true experimental-disable-loop-unroll=true first-mod-bits=30 scaling-mod-bits=24", "--scheme-to-lattigo", ] heir_lattigo_lib( name = "fraud_model_lattigo", go_library_name = "fraud_model_lattigo", heir_opt_flags = HEIR_OPT_FLAGS, importpath = "fully_homomorphic_encryption/demos/cc_fraud/lattigo/fraud_model_lattigo", mlir_src = "//demos/cc_fraud/data:model_annotated.mlir", split_preprocessing = True, ) go_binary( name = "evaluate_fhe", srcs = [ "evaluate_fhe.go", "utils.go", ], data = [ "//demos/cc_fraud/data:test_rows.csv", ], pure = "on", deps = [ ":fraud_model_lattigo", ":fraud_model_lattigo_utils", "//demos/common/go/pathutils", ], ) What’s going on here is that HEIR’s interface is a lot more like LLVM than clang. There are two binaries, heir-opt and heir-translate, which handle running compiler passes and codegen, respectively, matching LLVM’s opt and translate. Those two binaries are wrapped into bazel rules, which I published as rules_heir, and then further wrapped the rules in macros that correctly stitch together the optimizer and codegen binaries, and wrap the results into a cc_library, go_library, or