Google ships Gemma 4 QAT checkpoints, shrinking E2B to under 1GB for mobile

Google released Quantization-Aware Training checkpoints for Gemma 4, trading the usual post-training quantization penalty for models trained with quantization simulated in the loop. The result is Q4_0 weights across the lineup plus a mobile-specific format that pushes the E2B variant’s memory footprint below 1GB, small enough to run on phones and modest consumer GPUs without the quality cliff typical of aggressive PTQ.

The mobile schema is built around how edge silicon actually executes math: static activations skip runtime scaling, channel-wise layout matches mobile accelerator design, token-generation layers are squeezed to 2-bit while reasoning layers stay at higher precision, and embeddings plus KV cache get targeted compression to keep long conversations in memory. Text-only deployment (dropping the audio and vision encoders) cuts the footprint further.

Weights are on Hugging Face in GGUF and compressed-tensor formats, with day-one support across llama.cpp, Ollama, LM Studio, vLLM, SGLang, MLX, Transformers.js, and Google’s LiteRT-LM runtime. MTP-compatible QAT checkpoints are also available so the earlier inference speedup stacks with the new compression.