Gemma 4: running Google's AI models on your own hardware
Gemma 4 launched with four downloadable models. How they differ, what hardware they need, and what to check before running them locally.
On this page
If an app needs to work without a connection, calling a hosted model on every request is a poor fit. A local model removes that dependency, but it introduces another constraint: the model, its working memory, and the app all have to fit on the device.
Gemma 4 gives developers several sizes to work with. Google DeepMind's April 2, 2026 launch included two smaller models designed for phones and similar devices, plus two larger models for more powerful hardware. The model files are available under the Apache 2.0 license, which allows commercial use subject to its terms.
Choosing among them starts with the task and available hardware. The strongest benchmark result in the family does not describe the model that fits on a phone.
TL;DR Gemma 4's four launch models span edge devices and larger local systems. E2B and E4B support audio as well as visual input; the larger launch models focus on text and vision. Google's benchmarks show substantial gains over Gemma 3, but memory use and reliability still need to be measured with the model size, quantization, and context your app will use.
September 2026 update: This article covers the original April launch family. Google's current model overview also includes a 12B model.
The four Gemma 4 launch models
An open-weight model provides its trained parameters for download. You can run those weights through a compatible local runtime or host them on your own infrastructure. Local execution is an option, not a requirement: Google also provides hosted ways to try the models.
The model card distinguishes the launch models as follows:
| Model | Parameter count | Context window | Launch role |
|---|---|---|---|
| E2B | 2.3B effective; 5.1B with embeddings | 128K | Mobile and edge |
| E4B | 4.5B effective; 8B with embeddings | 128K | Mobile and edge |
| 26B A4B | About 26B total; about 4B active | 256K | Larger local systems |
| 31B | 30.7B dense | 256K | Workstations and servers |
The effective parameter label on E2B and E4B needs care. They use per-layer embedding tables, which add stored parameters without requiring the same computation as running every parameter through the main model layers. “2.3B effective” therefore does not mean that the entire model contains only 2.3 billion parameters.
The 26B model uses mixture of experts: it activates a subset of its expert components for each token. That reduces computation relative to activating the whole model, but the inactive weights still need to be stored or made available. The active parameter count alone does not tell you whether it fits in GPU memory.
What the models can read
All four launch models take text and images and generate text. They can process video as a sequence of frames. E2B and E4B also support audio input, including speech recognition and understanding. Audio support is not shared by all launch sizes.
Google says the family was trained on more than 140 languages. That describes training coverage, not equal accuracy across every language or task. For an app serving a specific language, test actual user inputs, including spelling variation and mixed-language text.
For example, an offline receipt assistant could take a photo and return a proposed merchant, date, and total. The application would still validate those fields and ask the user to correct uncertain results. A voice-based version also needs a model and runtime with audio support; choosing the largest model would not automatically provide it.
How Gemma 4 compares with Gemma 3
Google's launch comparison shows strong gains for Gemma 4 31B with thinking enabled over Gemma 3 27B without thinking. The current model card reports:
| Evaluation | Gemma 3 27B | Gemma 4 31B |
|---|---|---|
| AIME 2026 | 20.8% | 89.2% |
| Codeforces Elo | 110 | 2,150 |
| τ2-bench, average across three domains | 16.2% | 76.9% |
The source image reports the Retail subset of τ2-bench, 86.4% versus 6.6%, rather than the three-domain average above. The full launch table also makes the smaller-model tradeoff visible:
| Gemma 4 variant, thinking enabled | AIME 2026 | τ2-bench Retail |
|---|---|---|
| 31B | 89.2% | 86.4% |
| 26B A4B | 88.3% | 85.5% |
| E4B | 42.5% | 57.5% |
| E2B | 37.5% | 29.4% |
A smaller model does not inherit the flagship's scores. These rows are specific evaluations, not a general prediction of application quality.
τ2-bench evaluates tool-using tasks under its own environments and rules. A higher score is a reason to test a model on your workflow, not evidence that it will reliably finish your workflow. Model size, thinking settings, tool definitions, and the application around it all affect the result.
What local execution requires
Google's launch post describes the E models as suitable for phones and edge devices, including Raspberry Pi and Jetson hardware. It offers AI Edge Gallery as a way to try compatible models on a device.
For the larger models, Google says its unquantized bfloat16 weights fit on a single 80GB H100 and that quantized versions can run on consumer GPUs. Quantization stores model values at lower precision, reducing memory use at a possible cost to output quality. The exact trade depends on the format and runtime.
A weight download fitting on disk does not establish that a workload fits in RAM. Inference also needs working memory, including cached attention state that grows with the context and concurrent requests. Image processing and longer prompts add their own demands.
For a device test, record the model variant, quantization, runtime, input length, peak memory, and time to a usable answer. On a phone, check sustained use as well as the first response: thermal limits and battery consumption matter to the experience.
The useful checks before building around it
Start with representative inputs and outputs you can inspect. For the receipt example, that means blurry images, unfamiliar layouts, missing totals, and cases where the correct answer is “I cannot read this.” Compare the extracted values with known answers.
Then test the longest context and conversation your app actually needs. A 128K or 256K limit describes what the model can accept; it does not guarantee that every fact is retained or every instruction followed throughout that window. Repeat the evaluation after changing quantization or the runtime.
Apache 2.0 permits commercial use and modification, subject to its license and notice requirements. Local inference avoids an API bill per token, but it still uses hardware and engineering time. Likewise, keeping inference local does not automatically keep the entire app offline if telemetry, storage, or other features call external services.
Gemma 4 is useful when control over the model and its execution environment solves a real constraint. Pick the smallest launch model that handles the required inputs, measure it on the target device, and move up only when the output quality justifies the extra resources.
Sources
- Google: Gemma 4 launch
- Google AI for Developers: Gemma 4 model card
- Google AI for Developers: Current Gemma model overview
- Google AI Edge: Gallery
Common questions
What is Gemma 4?
Gemma 4 is a family of AI models from Google DeepMind that you can download and run on your own hardware under the Apache 2.0 license. The April 2026 launch included E2B, E4B, 26B, and 31B, with sizes suited to devices ranging from phones to workstations.
Is Gemma 4 free to use?
Its weights are available under the commercially permissive Apache 2.0 license, which still has license and notice obligations. Running downloaded weights locally avoids a hosted model's per-token API charge, but hardware, electricity, and deployment still have costs.
Can Gemma 4 run on a phone?
Google designed the E2B and E4B launch models for edge devices and provides access through AI Edge Gallery. Whether a model fits and runs well depends on its quantization, runtime, context length, and the device's available memory.
Which Gemma 4 launch models support audio?
E2B and E4B support audio input, while all four launch models support text and images, with video handled as frames. The models generate text; the larger launch models should not be assumed to have the smaller models' audio capabilities.
How did Gemma 4 compare with Gemma 3 at launch?
Google's comparison reports AIME 2026 scores of 89.2% for Gemma 4 31B with thinking and 20.8% for Gemma 3 27B without thinking. These are different model sizes and inference settings, and the result does not establish the performance of the smaller phone models.
What does the E in Gemma 4 E2B mean?
E means effective parameters: E2B has a 2.3B effective parameter count, but its total rises to 5.1B with embeddings. The effective count describes computation and should not be treated as the model's total storage or RAM requirement.