Fireship

Google Just Casually Disrupted the Open-Source AI Narrative…

Google’s latest surprise is Gemma 4, a 31‑billion‑parameter large language model released under the permissive Apache 2.0 license. Unlike most “open‑weight” offerings that carry restrictive clauses, Gemma 4 is truly free to use, modify, and commercialize, and it can run on a single consumer‑grade GPU. The model’s standout claim is its efficiency: a 20 GB download runs at roughly 10 tokens per second on an RTX 4090, whereas comparable models such as Kimmy K2.5 require 600 GB, hundreds of gigabytes of RAM, and multiple H100 accelerators. Google attributes this compression to two innovations—TurboQuant, a novel quantization that stores weights in polar coordinates and applies a Johnson‑Lindenstrauss‑type transform, and per‑layer embeddings (the “E” variants) that give each transformer layer its own token‑specific cheat sheet, cutting redundant information. In practice, Gemma 4 matches the performance of larger, more resource‑hungry open models while staying orders of magnitude smaller. The presenter demonstrated the model on an RTX 490 using Ollama, noting solid all‑round capabilities and suitability for fine‑tuning with frameworks like Unsloth. However, it still lags behind specialized coding assistants such as Code Rabbit for high‑precision programming tasks. The release lowers the barrier to entry for developers and enterprises seeking to run powerful LLMs locally, fostering broader experimentation and reducing reliance on costly cloud APIs. By proving that memory bandwidth—not raw compute—is the primary bottleneck, Google may shift industry focus toward smarter compression techniques, accelerating the open‑source AI race.