Survey of verifiable AI inference as of May 2026. API contracts typically do not bind the model billed to the weights executed; the sections below survey mechanisms that address it.
- zkML (ZK proofs): Cryptographic binding, no honest-party assumption. Systems include zkLLM, NANOZK, TensorCommitments, EZKL, DeepProve, zkPyTorch/Expander, JSTprove. zkLLM: ~15 min proving for 13B (2024); NANOZK: 24 ms layer verification, ~70× smaller proofs than EZKL at depth 128 (2026).
- TEEs: Remote attestation rooted in hardware (Anthropic/Irregular whitepaper, June 2025). NVIDIA H100/Blackwell GPU-CC is the practical deployment path; Tinfoil Modelwrap binds weight commitments to enclave measurement.
- opML: Optimistic fraud proofs (ORA); bisection dispute games from rollup designs. Low cost at scale; ~7-day dispute window excludes interactive latency targets.
- SVIP: Statistical checks on hidden activations; sub-10 ms overhead, <5% error in reported evaluations, requires API access to internals.
- Verde / Gensyn: Refereed delegation with RepOps for bitwise-reproducible ops; requires ≥1 honest replica, negligible proving overhead. On Gensyn mainnet (April 2026).
- OTR: TEE attestation plus optimistic challengers and stochastic ZK spot-checks (DGrid); targets the Verifiability Trilemma with sub-second TEE finality.
Decentralised integration: see §11 (EigenLayer AVS, Gensyn).
When a client pays for frontier-model inference through a commercial API, the protocol does not prevent the provider from substituting a cheaper or quantised model and returning outputs as if the billed model ran. Providers can do this with straightforward operational changes; standard terms of service do not supply a technical countermeasure.
Verifiable inference covers mechanisms that bind a response to execution of specified weights: zero-knowledge proofs, TEE remote attestation, optimistic fraud proofs, statistical checks on internal activations, and hybrids. This survey covers zkML, TEEs, opML, behavioural fingerprinting, and combined architectures, with emphasis on papers and deployed systems through mid-2026.
Machine Learning as a Service (MLaaS) follows a simple bilateral structure: a client sends an input (prompt, image, structured data) to an inference endpoint, and a server processes it using a pre-trained model and returns a result. The client pays for the capability level of a specific model. Nothing in the protocol binds the response to execution of the claimed model.[1][2]
A malicious provider can:
- Substitute a smaller, cheaper model (a 7B parameter model instead of a 70B one)
- Apply aggressive quantisation that degrades quality
- Return cached responses from a prior run
- Return outputs from a model that has been fine-tuned away from the claimed base weights
All of these attacks are undetectable to the client using standard API interactions. The 2026 paper NANOZK articulates this directly: "Service providers could substitute cheaper models, apply aggressive quantization, or return cached responses — all undetectable by users paying premium prices for frontier capabilities."[3][4][5]
Any system attempting to prove inference integrity faces a trilemma among three desirable properties:[6][7]
- Computational Integrity: The proof must be cryptographically sound, binding the output to the execution of specific weights
- Low Latency: The proof must be generated and verified without prohibitive delay (seconds, not hours)
- Low Cost: Proof generation must not cost orders of magnitude more than the inference itself
No existing scheme simultaneously satisfies all three. This tension organises the entire design space.[8][9]
Five approaches dominate the design space; they differ in trust assumptions, latency and cost overhead, deployment maturity, and what must change on the client relative to a standard OpenAI-style POST /v1/chat/completions integration.
| Approach | Trust assumption | Latency overhead | Cost overhead | Integrity | Client vs OpenAI Chat Completions |
|---|---|---|---|---|---|
| ZK proofs (SNARK/STARK) | None (cryptographic) | Very high (minutes–hours) | Very high | Cryptographic | Non-native: async proof delivery; client verifies (vk, proof, public_inputs) locally or on-chain. Request body may need input/output commitments; response carries proof metadata or a separate verify endpoint. |
| TEEs | Hardware vendor | Minimal (~0%) | Low | Hardware-rooted | Often drop-in body; connection layer changes: fetch/verify attestation (/.well-known/…, RA-TLS, or SDK preflight), pin measurement/model-root policy, optional E2EE (HPKE/EHBP) before messages are sent. |
| Optimistic / fraud proofs | ≥1 honest challenger | Dispute-window delay | Very low | Game-theoretic | Not chat HTTP: client is a smart contract calling requestCallback(…); app reads finalized output from chain/events after the challenge period. Wallet/RPC/ABI required. |
| Behavioural fingerprinting | Statistical / proxy | Negligible | Negligible | Probabilistic | Depends on variant: SVIP needs non-standard response fields (hidden/proxy features); black-box needs logprobs + top_logprobs; local forensic needs full runtime control (see §2.1). |
| Hybrid (OTR, TEE+ZK) | Mixed | Sub-second (TEE tier) | Low | Layered | TEE client steps plus attestation/receipt on the response; optional on-chain subscription for disputes and sampled ZK checks. |
The OpenAI-compatible surface is the de facto client contract: JSON with model, messages, sampling parameters, and a streaming or non-streaming completion in choices[].message.content. None of the verification approaches listed above are satisfied by that exchange alone; each adds obligations on the client, the provider, or both. The following assumes a client that today points an OpenAI SDK at https://api.openai.com/v1 (or a compatible proxy).
The inference request can remain familiar, but verification is a second phase. The provider (or a prover service) runs the model, then generates a proof that the circuit for model M produced output y on committed input x. The client must:
- Obtain a verification key
vkfor the committed model (published out-of-band, on-chain registry, or IEEE-style model registry [71]). - Receive
proofand public inputs (hashes of prompt, output, and sometimes sampled tokens) either in an extended response envelope (non-standard fields such asproof,public_inputs,vk_hash) or via webhook/poll after async proving (typical for zkLLM-scale latencies). - Run a verifier (local library, zkVerify, or L1/L2
verifyprecompile) before treating the completion as final.
Commitment schemes (TensorCommitments, NANOZK layerwise proofs) add cross-layer hash chaining: the client checks that each layer’s output commitment matches the next layer’s input commitment. There is no widely deployed OpenAI extension for this; Ritual-style stacks expose verification as a separate service path rather than inline in chat/completions.
TEE-backed products usually keep the chat JSON schema and move trust to the transport and attestation layer.
- Tinfoil-style clients wrap the OpenAI SDK: before or during the first request, the client fetches
/.well-known/tinfoil-attestation, validates the hardware quote, Sigstore predicate, and TLS/HPKE key binding, and checks the attested model Merkle/dm-verity root against a published commitment for the requestedmodelname. Request bodies match OpenAI; EHBP or pinned TLS provides encrypted direct-to-enclave channels. - Confidential-inference designs (Anthropic/Irregular, Azure AI confidential inferencing) require the client to verify KMS/TEE attestation and policy, obtain an HPKE public key, and encrypt the prompt (often via OHTTP) so only the attested enclave decrypts it. That is not a drop-in OpenAI SDK configuration: the client implements RA-TLS or attested-key HPKE setup before posting completions.
- For GPU CC (H100/Blackwell), the client’s job is policy verification (MRENCLAVE/RTMR allowlists, non-debug enclave) and, where offered, per-response receipts tying the completion to a measurement.
ORA-style opML is an on-chain oracle protocol, not an extended chat API. The integrating “client” is a smart contract that inherits AIOracleCallbackReceiver, calls aiOracle.requestCallback(modelId, input, address(this), gasLimit, callbackData), and implements aiOracleCallback(requestId, output, callbackData) when the result is posted after the challenge window. An off-chain application that wants a chat UX must either (a) submit calldata derived from the user prompt to the contract and wait for the callback event, or (b) use an indexer that watches opML finalization—still fundamentally different from streaming chat/completions. Wallets, chain selection, and dispute-window latency are mandatory client concerns.
| Variant | Request changes | Response changes | Client verification |
|---|---|---|---|
| SVIP | Prompt as usual; provider must accept an embedded secret (from a setup phase) | Standard text plus proxy_task_feature (compressed hidden-state features), not choices[].message alone |
User holds secret, labeling network, and proxy head; compares L2 distance to expected label vector (<0.01 s). Requires prior trusted setup to train the proxy on the target model. |
| Black-box fingerprinting | logprobs: true, top_logprobs: n (supported on OpenAI; often disabled or restricted on third-party APIs) |
choices[].logprobs.content[].top_logprobs |
Offline compatibility test on reconstructed logit vectors; no provider protocol beyond exposing logprobs. |
| Domain watermarking | Usually none if the provider serves a watermark-trained checkpoint | Standard completions | Statistical detection on outputs; client-side or auditor-side tests. |
| Local forensic | Full control of seed, runtime, and model artefact paths |
N/A (local inference) | Re-run with hashed weights and compare bit-identical outputs. |
SVIP is the furthest from OpenAI compatibility: three-party setup (trusted trainer, provider, user), non-standard response fields, and fixed-length constraints in published evaluations.
The client does not send chat messages to a single /v1/chat/completions endpoint. It submits the same ML program (inference job) to multiple compute providers through the Gensyn/Verde network API, with RepOps-enabled operators for bitwise-reproducible execution. The client compares outputs and checkpoint hashes, and initiates Verde dispute arbitration when replicas diverge. Integration is job-spec and SDK-centric (AXL/Chain/REE), not OpenAI message arrays.
Clients inherit TEE integration (attestation before connect, optional E2EE) and treat the inference response as provisionally final when accompanied by a TEE attestation \sigma_{TEE} binding trace, binary, and hardware. For full OTR security, infrastructure clients also run or subscribe to challenger (“Fishermen”) services and occasional ZK spot-check verification on sampled requests—typically operator/indexer roles rather than every end-user app. EigenAI narrows the gap for app developers: OpenAI-compatible /v1/chat/completions with extra response fields (receipt.req_hash, receipt.out_hash, receipt.sig, eigendalink, system_fingerprint, deterministic seed) and optional request signing/encryption to the operator key; verification replays the request against EigenDA with deterministic decoding metadata.
Zero-knowledge proofs (ZKPs) allow a prover \mathcal{P} to convince a verifier \mathcal{V} that a statement is true without revealing anything beyond the statement's validity. Applied to ML inference, the goal is to prove: "I ran model \mathcal{M} with parameters \theta on input x and obtained output y ," without disclosing \theta , x , or any intermediate activations unless desired.[10][11]
Succinctness matters for deployment: verification of a proof must be significantly cheaper than re-running the computation. zk-SNARKs (Succinct Non-Interactive Arguments of Knowledge) and zk-STARKs (Scalable Transparent Arguments of Knowledge) are the two dominant proof systems, with SNARKs offering smaller proofs and STARKs offering transparency without a trusted setup.[12][11]
Neural network inference presents unique challenges for ZK circuits:
- Non-arithmetic operations: Softmax, GELU, LayerNorm, and ReLU do not naturally encode as finite-field arithmetic circuits
- Scale: A 70B parameter model involves billions of multiply-accumulate operations, all of which must be arithmetised
- Floating-point: ZK circuits operate over finite fields; IEEE 754 floating-point must be simulated or replaced with fixed-point arithmetic[13]
Liu, Xie, and Zhang's zkCNN (ACM CCS 2021) applied GKR interactive proofs to convolutional neural networks. The work introduced techniques to handle the sumcheck protocol for convolution operations and established the theoretical basis for subsequent LLM-scale work.[14]
Sun, Li, and Zhang (arXiv:2404.16109) present zkLLM, a ZKP scheme for LLMs. Main components:[15]
- tlookup: A parallelised lookup argument for non-arithmetic tensor operations in deep learning, providing a solution with no asymptotic overhead. It is designed for the GPU parallel execution model.[16]
- zkAttn: A specialised ZKP for the attention mechanism, balancing running time, memory usage, and accuracy.[17]
With a fully parallelised CUDA implementation, zkLLM can generate a correctness proof for a 13B parameter LLM inference in under 15 minutes, producing a proof smaller than 200 KB. This marked the transition from toy-scale models to transformer-scale verification, though 15 minutes per query remains far from real-time.[15][16]
Wang's NANOZK (arXiv:2603.18046, March 2026) proves inference layerwise: transformer forward passes decompose into independent layers, each with a constant-size proof (5.5 KB/layer: 2.1 KB attention, 3.5 KB MLP; 24 ms verification).[5][3]
Lookup tables approximate softmax, GELU, and LayerNorm without reported accuracy loss; Fisher-information-guided selective verification applies when full-layer proofs are too costly. At depth 128, NANOZK reports 70× smaller proofs and 5.7× faster proving than EZKL, with \epsilon < 10^{-37} soundness.[5]
A concurrent approach from Zeng et al. (arXiv:2602.12630, February 2026) proposes TensorCommitments (TCs), a tensor-native proof-of-inference scheme. TCs bind LLM inference to a commitment — an irreversible tag organised via multivariate Terkle Trees — that breaks under tampering. For LLaMA 2, TC adds only 0.97% prover overhead and 0.12% verifier time over baseline inference while improving robustness to tailored LLM attacks by up to 48% over prior work.[1]
Pal et al. (March 2026) from Ritual present a sampling-based framework that replaces full cryptographic proofs with statistical properties of neural network execution traces. The prover commits to the inference trace via Merkle-tree vector commitments and opens only a small number of entries along randomly sampled paths from output to input. Soundness is traded for cost: proving drops from minutes to milliseconds, useful when repeated queries raise detection probability.[18]
EZKL (zkonduit) is the most widely used zkML toolchain for ONNX → Halo2 circuits. It accepts ML models in ONNX format, automatically converts them into ZKP-compatible circuits using the Halo2 proof system, and generates proofs of correct model execution. The three core use cases are: public model on private data, private model on public data, and public model on public data (particularly for blockchain co-processors). As of 2025-2026, EZKL proves a linear regression in 0.118 seconds on commodity hardware; larger models on A100 GPUs cost USD 0.02–0.50 per proof.[19][20][12]
DeepProve (Lagrange Labs) uses sumcheck and logup GKR for sublinear proving on MLPs and CNNs in ONNX. Published benchmarks claim 54–158× faster proving and 671× faster verification than EZKL.[21][22][23]
Polyhedra's zkPyTorch compiler (March 2025) lowers PyTorch/ONNX models to ZK circuits. Expander, the backend, uses GKR with polynomial commitments at linear prover time. Benchmark performance on their architecture shows VGG-16 (15M parameters) at approximately 2.2 seconds per image proof; Llama-3 (8B parameters) at approximately 150 seconds per token on single-core CPU. The Expander system achieves up to 9,000 ZK proofs per second with CUDA 13.0 and GPU-accelerated KZG commitments.[24][25][26]
JSTprove (arXiv:2510.21024, October 2025) wraps Expander with CLI tooling and auditable artefacts for verifiable inference pipelines.[27][28]
JOLT is a zkVM (zero-knowledge virtual machine) targeting the RISC-V instruction set, developed by a16z. The lookup-centric approach converts CPU instructions into lookups in large pre-defined tables via the Lasso lookup argument, rather than heavy algebraic constraint systems per operation. JOLT Atlas adapts this for ML operations, enabling arbitrary ML programs compiled to RISC-V to be proved — at the cost of universal VM overhead compared to circuit-specific systems.[29][30][13]
A Trusted Execution Environment is a hardware-isolated enclave within a processor (or GPU) that provides three guarantees: data confidentiality, data integrity, and code integrity. The key mechanism enabling verifiable inference is remote attestation — a cryptographic protocol by which the TEE generates a signed report proving:[31]
- The hardware is a genuine TEE (using manufacturer-embedded keys)
- The TEE is running specific, unmodified code (measured as a hash of the loaded binary)[32][33]
For AI inference, this means a user can receive a cryptographic attestation proving: "NVIDIA H100 hardware, running binary hash X, processed your query." If binary X corresponds to the inference code for the claimed model weights, the user has hardware-rooted assurance of execution integrity.[34][31]
CPU-only TEEs are insufficient for AI inference because weights and activations live in GPU memory — outside the CPU enclave boundary. An attacker with GPU memory access can extract weights. The NVIDIA H100 GPU introduced the first GPU-level confidential computing support, extending the TEE boundary to include the accelerator itself. It encrypts GPU execution memory, register states, model weights, the KV cache, and outputs, providing hardware attestation for GPU workloads.[35][36][37][31]
The Anthropic / Irregular (Pattern Labs) joint whitepaper on Confidential Inference Systems (June 2025) formally describes the design principles and security considerations for confidential inference, noting that:
- The threat model must assume the service provider is adversarial with full machine control
- CPU-only TEEs are insufficient — the confidential boundary must extend to the GPU accelerator
- NVIDIA H100 and Blackwell GPUs ship with native TEE support, enabling encrypted weight loading, protected in-flight computation, and encrypted output egress[38][33]
Anthropic's architecture involves: encrypting the request before it reaches Anthropic servers; decrypting, processing, and re-encrypting within the trusted loader; and never releasing model weights from the enclave.[33]
| Platform | Vendor | GPU Support | Status |
|---|---|---|---|
| Intel SGX | Intel | No (CPU only) | Production |
| Intel TDX | Intel | Via NVIDIA CC pairing | Production |
| AMD SEV-SNP | AMD | Via NVIDIA CC pairing | Production |
| ARM TrustZone / CCA | ARM | No (edge focus) | Production/Research |
| NVIDIA H100 CC | NVIDIA | Native | Production |
| NVIDIA Blackwell | NVIDIA | Native (enhanced) | Production |
The 2026 survey paper "When Agents Handle Secrets" provides a unified taxonomy covering all six major TEE platforms, analysing deployment roles and performance trade-offs for agentic AI specifically.[39]
Tinfoil's Modelwrap (February 2026) binds model identity to TEE measurement:[40][41]
- Public Merkle root over weight files
- Boot-time attestation (Linux
dm-verityrecomputes block hashes on read; mismatch fails I/O) - Client verification of attestation root against the published commitment
Each response can carry a receipt tying the attestation to the weight commitment.[41]
TEEs shift trust from the inference provider to the hardware manufacturer (Intel, AMD, NVIDIA). A compromised or backdoored chip manufacturer could theoretically defeat attestation. Additionally, side-channel attacks (power analysis, cache timing) remain a theoretical vector, with RAND noting that current GPU TEEs lack physical attack protections. The Confidential Computing Transparency Framework (arXiv:2409.03720) explicitly notes that attestation cannot guarantee the absence of vulnerabilities or backdoors in TEE firmware.[42][43][31]
opML (Optimistic Machine Learning), introduced by Hyper Oracle (ORA) in 2023 (arXiv:2401.17555), adapts the optimistic rollup design pattern to ML inference. The protocol operates as follows:[44][45]
- A submitter executes ML inference natively (with full hardware acceleration) and posts results on-chain
- The result is assumed correct by default (optimistic assumption)
- Validators check results; if a discrepancy is found, a bisection dispute game is initiated
- The game binary-searches the computation trace to isolate a single disputed instruction
- A smart contract arbitrates only that single instruction, making on-chain adjudication cheap[46][47]
The core components are a Fraud Proof Virtual Machine (FPVM) that can execute individual ML operations and expose them to on-chain verification, and a Machine Learning Engine (MLE) for efficient off-chain execution. Memory is managed as a Merkle tree, enabling efficient single-instruction state proofs.[47][45]
opML can run 7B LLaMA models on standard PCs without GPUs, requires no expensive proof generation per request, and is cost-effective for large model scales. However, the dispute window — analogous to the 7-day challenge period in Ethereum optimistic rollups — prevents real-time or sub-second finality. This makes opML unsuitable for interactive applications, DeFi, or real-time AI agents. Security also requires at least one honest challenger to be monitoring and willing to dispute.[48][7][44]
A 2026 Springer paper (zk-OPML) explores combining ZK proofs with the opML framework to reduce the challenge period by providing probabilistic ZK spot-checks during the dispute window. This trades marginal ZK proving cost for significantly shortened settlement times.[49]
6.1 SVIP: Hidden Representation Verification
SVIP (arXiv:2410.22307, NeurIPS 2025; Secret-based Verifiable Inference Protocol) requires the provider to return generated text and selected hidden activations from internal transformer layers.[4][50]
A small proxy task model is trained exclusively on the hidden states of the specified model (e.g., Llama 3.1 70B). If the provider runs the correct model, the proxy task performs well on the returned hidden states; if a cheaper model was substituted, the proxy task fails to generalise to those states.[51][4]
The verifier holds undisclosed probe queries so the provider cannot tailor activations to pass a fixed check.[52][51]
Results across models from 13B to 70B parameters: false negative rate below 5%, false positive rate below 3%, and verification latency of less than 0.01 seconds per query. The primary limitation is that SVIP requires the provider to expose hidden representations, which may not be supported by all APIs, and is most robust for fixed-length inputs.[4][51]
Black-box fingerprinting techniques (arXiv:2407.01235, 2024) exploit the fact that the output distribution of an LLM spans a unique vector space associated with each model. By reconstructing the vector space from API responses (using top-k probabilities or full vocabulary logits), an auditor can perform a compatibility test to determine whether a suspect model's outputs lie in the same space as the target model's outputs. This requires no model internals, only the probability distribution over output tokens.[53]
Domain-specific watermarking for model fingerprinting (ICML 2025) introduces the idea of training a model to embed output watermarks only within specified subdomains (particular languages, topics, or code styles). This provides strong statistical guarantees for model provenance, with controllable false positive rates and robustness to real-world deployment variability, fine-tuning, and post-processing.[54]
A practical forensic approach for local LLM inference (IEEE 2026) achieves 100% bit-identical reproducibility across 40 controlled inference runs by recording cryptographic hashes of model artefacts, capturing environmental metadata, and logging inference-time seeds. A separate verification process instantiates a clean runtime, reloads the hashed model, and performs bit-for-bit comparison to produce forensic pass/fail reports. This approach is primarily applicable to open-weight models in controlled environments.[55]
Verde (arXiv:2502.19405, February 2025, developed by Gensyn) introduces the cryptographic notion of refereed delegation adapted to the ML setting. Rather than a single trusted provider, a computationally limited client delegates the same program to multiple untrusted compute providers. If at least one provider is honest, the client obtains the correct result.[56][57]
The two technical innovations enabling this are:
- Dispute Arbitration (Verde Protocol): When providers disagree, a dispute game pinpoints the exact operator in the neural network's computational graph where results first diverge. The referee (which can be a smart contract or jury of verifiers) recomputes only that single disputed operator, not the entire computation. This reduces verification cost from O(N) to O(1) dispute adjudication.[58][59]
- RepOps (Reproducible Operators): A library implementing bitwise-reproducible versions of ML operators by enforcing a fixed execution order of floating-point operations. This solves the inherent hardware non-determinism of GPU computation (CUDA kernel scheduling, floating-point associativity) that would otherwise prevent honest nodes from producing identical outputs on different hardware.[60][59][56]
Verde is the foundation of Gensyn's mainnet (launched April 22, 2026), supporting models up to 72B parameters with bitwise-consistent outputs across hardware.[60]
Verde trades cryptographic soundness for practical efficiency. Unlike ZK proofs — which provide mathematical certainty with no honest-party assumption — Verde's guarantees hold only if at least one participating node is honest. Verde adds only checkpoint hashing on the compute path—no dedicated proving hardware.[59]
DGrid AI's Optimistic TEE-Rollups (OTR; arXiv:2512.20176, December 2025) stack three verification layers:[7][6]
- Provisional finality: TEE-signed attestation \sigma_{TEE} with MRENCLAVE (or equivalent) binding trace, binary, and hardware; sub-second.[9]
- Optimistic fallback: decentralised challengers ("Fishermen") dispute attestations.[9]
- Stochastic ZK spot-checks at rate \rho < 0.01 on sampled transactions.[9]
Proof of Efficient Attribution (PoEA) ties payment to the weight set loaded in the enclave. Reported simulations: ~99% of centralised throughput, ~0.07 s inference latency.[61][8]
Ritual (arXiv:2602.17223, February 2026) reuses privacy-preserving inference primitives (SMPC, FHE) for verified inference at small marginal cost. Infernet and Ritual Chain expose ZKML, optimistic proofs, and TEE-backed paths behind a common interface.[62][63][64]
Conan (IEEE 2025, arXiv:2503.07466) addresses a subtler problem: even if inference was run on the correct model, the provider might use a model that was fine-tuned to reduce quality while maintaining architecture. Conan requires the server to first commit to the model and prove in zero-knowledge that the committed model achieves the claimed accuracy, and then perform secure inference on that committed model using maliciously-secure two-party computation (2PC) protocols. This simultaneously achieves accuracy verification, inference integrity, and input/output privacy.[65][66]
FHE allows computation over encrypted data — a client could submit an encrypted query and receive an encrypted output, with the server never seeing plaintext. EncryptedLLM (ICML 2025, arXiv) presents a GPU-accelerated CKKS-based FHE implementation for GPT-2 forward passes, achieving runtimes over 200× faster than CPU baselines. Full LLM inference via FHE remains impractical at production scale (bootstrapping, ciphertext expansion). For near-term confidential inference, TEE paths dominate; FHE is primarily a research track.[67][68][69]
An IETF Internet-Draft (draft-sharif-ai-model-lifecycle-attestation-00, February 2026) proposes a standards-track protocol for cryptographic attestation across the full AI model lifecycle — from training data provenance through inference output. If ratified, this would provide interoperability between attestation implementations across vendors and enable regulatory compliance based on verifiable provenance chains.[70]
A proposed AI model registry (IEEE, November 2025) uses Ethereum and IPFS to provide on-chain model registration with dual verification paths: ZKML for strong cryptographic guarantees and opML via dispute resolution for efficiency. The system enables users to confirm both model identity and execution correctness through a single registry lookup.[71]
The major AI labs have published their own positions:
- Anthropic / Irregular: Joint whitepaper on Confidential Inference via Trusted VMs (June 2025), explicitly noting CPU-only TEEs are insufficient and requiring GPU-level confidential computing[38][31][33]
- OpenAI: May 2024 post "Reimagining Secure Infrastructure for Advanced AI" called for confidential computing primitives to extend beyond CPU hosts into AI accelerators[31]
- Google DeepMind: Frontier Safety Framework (May 2024) lists TPUs with confidential compute capabilities at its highest security level[31]
Several networks expose verifiable inference as infrastructure rather than a bilateral API feature:
EigenLayer's EigenAI (mainnet live late 2025) provides an AVS (Actively Validated Service) for LLM inference where operators post ETH restaking collateral as a slash-able economic bond. Every input, output, and model version is cryptographically guaranteed through on-chain attestation. EigenCompute (January 2026 mainnet alpha) handles off-chain execution verification as a separate AVS. By 2026, over 280 crypto-AI projects require trust-minimised model evaluation, making AI verification the largest AVS category.[72][73]
Allora uses zkML to allow deployed models to supply predictions alongside ZK proofs that the value was generated by the claimed algorithm — without revealing model weights — enabling permissionless, verifiable AI oracles for DeFi and governance.[74]
Gensyn's three-layer architecture (AXL for communication, Chain for identity and settlement, REE for authentication) uses Verde and RepOps to provide a fully decentralised ML network where any third party can reproduce and verify any computation.[75][60]
| System | Model Scale | Proving Time | Proof Size | Approach |
|---|---|---|---|---|
| EZKL | Linear regression | 0.12 seconds | ~KB | SNARK (Halo2) |
| DeepProve | CNN | ~milliseconds | ~KB | GKR sumcheck |
| zkLLM | 13B params | ~15 minutes | < 200 KB | CUDA tlookup |
| NANOZK | per transformer layer | 24 ms verification | 5.5 KB/layer | Layerwise ZK |
| zkPyTorch (Expander) | 8B params (Llama-3) | ~150 sec/token | — | GKR |
| SVIP | Any open-weight | < 0.01 sec | (API overhead) | Statistical proxy |
| Verde / Gensyn | Up to 72B | Near-zero overhead | Checkpoint hashes | Refereed delegation |
| TEE (H100 CC) | Any (hardware-bound) | ~0% overhead | Attestation report | Hardware |
| OTR | Any LLM | Sub-second (TEE tier) | Attestation + spot ZK | Hybrid |
On-chain ZK proof verification adds a fixed Ethereum gas cost of approximately 200,000 to 600,000 gas units per proof, translating to roughly USD 3–18 per verification call at current mainnet prices. A zkML MVP (lightweight model, testnet deployment, end-to-end integration) typically requires 4–6 weeks and USD 40,000–80,000 depending on optimisation depth.[20]
Open problems:
- Billion-parameter ZK proving: No current system can generate a ZK proof for a 70B+ model inference in real-time. The NANOZK layerwise decomposition and lightweight trace commitment schemes represent the frontier but have not yet been demonstrated at GPT-4 scale.
- Hardware non-determinism: Floating-point non-determinism across GPU hardware makes bit-identical reproducibility a hard engineering constraint. RepOps addresses this but at operator-level granularity with custom CUDA kernels — not universally applicable.
- Compound attestation for agent chains: As AI operates as multi-hop agent pipelines, the attestation problem compounds — each hop's correctness must be proven, and the chain of attestations must be efficiently aggregated. The survey "When Agents Handle Secrets" identifies this as a major open challenge.[39]
- Side-channel attacks on GPU TEEs: Current H100 Confidential Computing does not protect against physical attacks, and RAND's SL4 requirements are not fully met. The Anthropic whitepaper acknowledges fallback architectures bridging CPU enclaves to GPUs as non-airtight.[31]
- Model commitment binding at fine-tuning: If a provider fine-tunes from a base model, the committed base model hash no longer corresponds to the running weights. Auditable Fine-Tuning and Inference for Proprietary AI (arXiv:2603.07466, March 2026) addresses this by requiring providers to publicly release base model hash commitments before fine-tuning contracts are established and providing recomputation-based verification protocols for fine-tuned deltas.[76]
- Non-deterministic sampling: LLMs sample stochastically at generation time. A ZK proof of inference proves the forward pass given a fixed sequence of sampled tokens, not the distribution. The prover must also commit to the sampling randomness — a non-trivial extension.
| Constraint | Direction |
|---|---|
| Cryptographic proof required; minutes of proving acceptable | EZKL, zkPyTorch, or zkLLM for models ≲13B; run proving off the request path |
| Near-native latency; trust hardware vendor | H100/Blackwell CC with remote attestation; weight commitments (e.g. Modelwrap); Anthropic/Irregular confidential-inference design |
| On-chain enforcement; economic slashing | EigenLayer EigenAI AVS; opML where dispute-window latency is acceptable |
| API-scale statistical check; open weights | SVIP-style hidden-state probes (logprobs or custom fields per §2.1) |
| Decentralised compute; large models; no ZK prover fleet | Gensyn Verde + RepOps (≥1 honest replica) |
| Regulated / high-stakes; layered assurance | OTR: TEE finality + challengers + sparse ZK audits |
Between 2024 and 2026, several results moved verifiable inference from prototypes toward deployable components: zkLLM at 13B scale, NANOZK's per-layer proofs, Verde on Gensyn mainnet, and GPU confidential-compute designs from Anthropic and Irregular.[77][3][38][5]
No single mechanism satisfies the Verifiability Trilemma. Frontier-scale ZK proving is not yet on the inference latency path; TEEs import trust in the silicon vendor; optimistic and refereed schemes need at least one honest participant. Hybrids such as OTR combine attestation, economics, and sampled ZK.[6][8]
Governance pressure (NIST, EU AI Act) and audit requirements in finance, healthcare, and legal workflows are driving attestation standards (IETF draft-sharif-ai-model-lifecycle-attestation) and vendor alignment on GPU CC.[70][31]
State of the art as of May 2026. Primary references: EZKL, DeepProve, Verde, NANOZK, zkLLM, Anthropic confidential inference, awesome-zkml.
- TensorCommitments: A Lightweight Verifiable Inference for Language Models - Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, an...
- The Inference Interference - Everything Bagel - The solution to this is Verifiable inference, using mechanisms to verify the use of specific models ...
- NANOZK: Layerwise Zero-Knowledge Proofs for Verifiable Large Language Model Inference - When users query proprietary LLM APIs, they receive outputs with no cryptographic assurance that the...
- SVIP: Towards Verifiable Inference of Open-source Large Language Models - The ever-increasing size of open-source Large Language Models (LLMs) renders local deployment imprac...
- NANOZK: Layerwise Zero-Knowledge Proofs for Verifiable ... - arXiv - We present METHOD, a zero-knowledge proof system that makes LLM inference verifiable: users can cryp...
- Optimistic TEE-Rollups: A Hybrid Architecture for Scalable and Verifiable Generative AI Inference on Blockchain - The rapid integration of Large Language Models (LLMs) into decentralized physical infrastructure net...
- Optimistic TEE-Rollups: Solving the Verifiability Trilemma ... - DGrid AI - ... integrity, sub-second latency, and near-native cost. The Verifiability Trilemma: Why Decentraliz...
- Tee-rollups Enable 0.07 Second Blockchain Inference - This trilemma states that systems struggle to simultaneously achieve high computational integrity, l...
- [論文評述] Optimistic TEE-Rollups: A Hybrid Architecture for Scalable ... - Optimistic TEE-Rollups (OTR) is a novel hybrid verification protocol designed to address the Verifia...
- Zero-knowledge Machine Learning (ZKML) - Introduction to Nesa - For model verification, P \mathcal{P} P demonstrates that M \mathcal{M} M executed correctly on inpu...
- An introduction to zero-knowledge machine learning (ZKML) - A zero-knowledge (ZK) proof is a cryptographic protocol in which one party, the prover, can prove to...
- ZKML: Verifiable Machine Learning using Zero-Knowledge Proof - ZKML offers a solution by providing cryptographic proofs that the training procedures were executed ...
- JOLT Atlas Reaching For SOTA In Zero Knowledge Machine ... - Kinic - When we talk scalability in zkML, we could mean two things: handling large models (many layers, many...
- zkCNN: Zero Knowledge Proofs for Convolutional Neural Network ... - zkCNN: Zero Knowledge Proofs for Convolutional Neural Network Predictions and Accuracy. Security and...
- zkLLM: Zero Knowledge Proofs for Large Language Models - arXiv - Addressing the persistent challenge of non-arithmetic operations in deep learning, we introduce tloo...
- [PDF] zkLLM: Zero Knowledge Proofs for Large Language Models - We propose tlookup, a unique ZKP protocol for universal non-arithmetic operations in deep learning, ...
- [Literature Review] zkLLM: Zero Knowledge Proofs for Large ... - The paper introduces "zkLLM," a novel framework for generating zero-knowledge proofs (ZKPs) tailored...
- Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference - When large AI models are deployed as cloud-based services, clients have no guarantee that responses ...
- The EZKL System - EZKL is a developer-friendly system for verifiable AI and analytics. Analytics can be descriptive (a...
- zkML Proof Generation Costs: Benchmark Analysis 2026 - Ancilar - EZKL proves a linear regression in 0.118 seconds (EZKL Benchmarks, 2025). Operators who separate the...
- GitHub - Lagrange-Labs/deep-prove - Welcome to DeepProve, a cutting-edge framework designed to prove neural network inference using zero...
- DeepProve — Verifiable AI Inference & zkML | Lagrange - Lagrange's DeepProve generates cryptographic proofs for AI inferences — up to 158× faster than the l...
- Lagrange's DeepProve - A Pioneer in AI Verification for - Binance - DeepProve of @Lagrange Official supports multi-layer perceptrons (MLP) and convolutional neural netw...
- Polyhedra Boosts zkML with Expander: 9000 zk Proofs Per Second - Polyhedra upgrades its Expander system with CUDA 13.0 support, 1 TB/s bandwidth, and GPU-accelerated...
- Polyhedra Introduces a Breakthrough in AI Trust Infrastructure - Polyhedra is building foundational infrastructure for trust and scalability in AI and blockchain sys...
- Expander - Polyhedra Network - Expander is open-source and continuously evolving, delivering unmatched performance to help develope...
- JSTprove: Pioneering Verifiable AI for a Trustless Future - arXiv - DeepProve [23] is a zkML framework that supports multi-layer perceptrons (MLPs) and convolutional ne...
- Daily Papers - Hugging Face - In this paper, we introduce JSTprove, a specialized zkML toolkit, built on Polyhedra Network's Expan...
- A Technical Dive into Jolt: The RISC-V zkVM - ZK/SEC Quarterly - In our latest post, we take you inside the workings of Jolt, a zero-knowledge virtual machine for th...
- GitHub - a16z/jolt: The simplest and most extensible zkVM. Fast and ... - This repository currently contains an implementation of Jolt for the RISC-V 64-bit Base Integer Inst...
- Confidential Computing for AI Workloads - Elsie's Blog - Substack - Confidential computing resolves this by running inference inside a hardware-secured TEE. The hardwar...
- What Is a Trusted Execution Environment (TEE)? - Chainlink - A Trusted Execution Environment (TEE) is a secure area of a main processor that guarantees code and ...
- Confidential Inference via Trusted Virtual Machines - Anthropic - Confidential Inference is a set of tools we can use to process encrypted data and to show that such ...
- Enhancing AI inference security with confidential computing - TEEs use hardware-backed cryptographic technologies to create an isolated, verified execution enviro...
- Confidential Computing on NVIDIA H100 GPUs for Secure and ... - The NVIDIA H100 Tensor Core GPU is the first ever GPU to introduce support for confidential computin...
- AI Security with Confidential Computing - NVIDIA - It protects GPU execution, memory, and register states while keeping models, training data, and infe...
- NVIDIA GPU Confidential Computing Demystified - arXiv - In this paper, we aim to demystify the implementation of NVIDIA GPU-CC system by piecing together th...
- Irregular and Anthropic Publish Whitepaper on Confidential AI ... - TEEs provide cryptographic attestation to verify code integrity, allowing parties to confirm the com...
- When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AI - Agentic AI systems, specifically LLM-driven agents that plan, invoke tools, maintain persistent memo...
- Modelwrap: Cryptographic Assurance of AI Model Identity - UBOS - Modelwrap combines three proven techniques to create an end‑to‑end guarantee of model integrity: Pub...
- How Tinfoil Proves Exactly What Model Is Running - With Modelwrap we end up attesting two things: The cryptographic commitment to the model weights (a ...
- Confidential Computing for AI: TEEs, Attestations, and Limits - Confidential computing now encompasses GPU-enabled Trusted Execution Environments (TEEs) which provi...
- A Confidential Computing Transparency Framework for a Comprehensive Trust Chain - Confidential Computing enhances privacy of data in-use through hardware-based
Trusted Execution Envi...
- opML: Optimistic Machine Learning on Blockchain - The integration of machine learning with blockchain technology has witnessed increasing interest, dr...
- opML - ORA - opML (Optimistic Machine Learning) is an innovative framework that enables efficient and scalable ma...
- opML - Projects & Protocols - IQ.wiki - This framework enhances transparency and fosters trust in machine learning inference by allowing for...
- [Literature Review] opML: Optimistic Machine Learning on Blockchain - opML uses a fraud-proof system instead of zero-knowledge proofs to guarantee the correctness of ML r...
- opML: A Method for Verifying Machine Learning Output Using ... - Objective: opML aims to port AI model inference and training/fine-tuning into blockchain using an op...
- zk-OPML: Using zero-knowledge proofs to optimize OPML
- SVIP: Towards Verifiable Inference of Open-source Large Language... - TL;DR: We propose SVIP, a secret-based protocol for verifiable LLM inference using hidden representa...
- SVIP: Towards Verifiable Inference of Open-Source Large ... - GitHub - This repository contains the implementation of SVIP, a secret-based verifiable LLM inference protoco...
- State of Verifiable Inference & Future Directions - Equilibrium Labs - Verifiable inference enables proving the correct model and weights were used, and that inputs/output...
- A Fingerprint for Large Language Models - arXiv - This paper to propose a novel black-box fingerprinting technique for LLMs, which requires neither mo...
- Robust LLM Fingerprinting via Domain-Specific Watermarks - Our evaluations show that domain-specific watermarking enables model fingerprinting with strong stat...
- Deterministic LLMs: a Practical Forensic Framework for Verifiable and Reproducible Local LLM Inference - Large Language Models (LLMs) have a growing role in legal, investigative, and enterprise workflows, ...
- Verde: Verification via Refereed Delegation for Machine Learning Programs - Machine learning programs, such as those performing inference, fine-tuning, and training of LLMs, ar...
- Verde: Verification via Refereed Delegation for Machine Learning ... - We design Verde, a dispute arbitration protocol that efficiently handles the large scale and graph-b...
- Verification via Refereed Delegation for Machine Learning Programs - A cryptographic refereed delegation approach ensures correct results for machine learning programs r...
- Verde: a verification system for machine learning over untrusted nodes - This is an academic paper describing Verde, a verification protocol for machine learning programs, a...
- Competition for AI upper-layer applications is fierce; Gensyn has ... - ... decentralized AI field: ... The second type is zkML , such as Ritual and Giza, which use cryptog...
- Optimistic TEE-Rollups: A Hybrid Architecture for Scalable and ... - ... Trilemma, which posits that a decentralized inference system cannot simultaneously achieve high ...
- Igniting the Decentralized AI Revolution on the Blockchain With Ritual - ... Ritual supports various proof systems, including ZKML, optimistic proofs, and trusted execution ...
- Blockchain X AI, 6 Must-Know Infra Projects - DeSpread Research - Ritual Chain – a Layer 1 blockchain optimized for AI workloads; Infernet – a decentralized oracle ne...
- Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference ... - These include methods such as secure multi-party computation (SMPC) and fully homomorphic encryption...
- Conan: Secure and Reliable Machine Learning Inference Against Malicious Service Providers - In the Machine Learning as a Service paradigm, a service provider (e.g., a server) hosting a model o...
- [PDF] Conan: Secure and Reliable Machine Learning Inference against ... - The second is secure inference on the committed model against malicious servers, built on our generi...
- Homomorphic encryption for LLM inference: Is it viable, or are TEE ... - FHE for full LLM inference is still researchy; bootstrapping and ciphertext bloat kill latency and c...
- Privacy-Preserving Large Language Model Inference via GPU ... - Fully homomorphic encryption can, in principle, evaluate any function over encrypted data. One appro...
- Privacy-Preserving Large Language Model Inference via GPU ... - This paper presents a GPU-accelerated implementation of CKKS-based fully homomorphic encryption (FHE...
- draft-sharif-ai-model-lifecycle-attestation-00 - IETF Datatracker - Cryptographic Attestation for AI Model Lifecycle: From Training Data to Inference Output.
- Demo: Verify, Don’t Trust: Open-Source AI Is Not Enough - Artificial intelligence (AI) is increasingly used in critical domains, yet models are often executed...
- EigenLayer Crosses $18B in Restaked ETH — How Vertical AVS ... - EigenAI went live on mainnet in late 2025, providing verifiable AI inference — allowing blockchain a...
- EigenLayer's Double Launch: 4x Rewards and Verifiable AI ... - P2P.org reduced commission from 10% to 5% through end of 2025 for all $EIGEN restakers. EigenLayer m...
- Allora Network: A comprehensive overview of a self-improving ... - Gensyn focuses on large-scale AI model training by leveraging decentralized compute power with crypt...
- Introducing Judge - Gensyn - Verde ensures every judgment can be independently checked. It uses refereed delegation: multiple unt...
- Auditable Fine-Tuning and Inference for Proprietary AI - arXiv - We require the cloud provider to publicly release the hash commitment of the base model before any f...
- The zkML Singularity: A Comprehensive Analysis of the 2025 ... - The ability to compress the intelligence of a Neural Network into a succinct cryptographic proof (De...