Huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning
•October 27, 2025
0
Why It Matters
The new version dramatically improves performance, scalability, and developer ergonomics, reinforcing Hugging Face’s role as the de‑facto backbone of the AI ecosystem.
Key Takeaways
- •v1.0 introduces httpx backend, boosting request performance.
- •New Typer‑based CLI replaces deprecated huggingface‑cli.
- •Xet protocol cuts large file transfer size by chunking.
- •Over 113M monthly downloads, 200k+ corporate users.
- •Dependency for 200k GitHub repos and 3k PyPI packages.
Pulse Analysis
The Hugging Face Hub has evolved from a simple model‑sharing shortcut into a full‑stack platform that underpins modern AI workflows. Over the past five years, huggingface_hub grew from a thin Git wrapper to a robust library that abstracts authentication, versioned storage, and community features. Its adoption across more than 200 k GitHub repositories and thousands of PyPI packages illustrates how the open‑source community has embraced a unified interface for models, datasets, and interactive Spaces, turning the Hub into a critical piece of AI infrastructure.
Version 1.0’s technical overhaul addresses the scaling pressures of today’s generative AI boom. By swapping the legacy requests library for httpx, the package gains async support, connection pooling, and faster retries, which translates into lower latency for model pulls and uploads. The new Typer‑based CLI offers a richer, more discoverable command set, while the hf_xet integration replaces Git‑LFS with chunk‑level deduplication, cutting bandwidth usage for multi‑gigabyte artifacts by up to 70 %. These changes not only speed up everyday data scientists’ pipelines but also lower operational costs for enterprises that rely on massive model repositories.
From a business perspective, huggingface_hub’s reach—over 113 million monthly downloads and adoption by Fortune 500 firms—means that any shift in its stability or performance reverberates across the AI supply chain. The library’s expanded API surface now supports automated deployment of Spaces, inference endpoints, and job orchestration, enabling companies to embed model serving directly into CI/CD workflows. As AI models become larger and more collaborative, the hub’s Xet‑driven storage and httpx‑based networking position it to handle petabyte‑scale data while preserving the open, community‑first ethos that fuels rapid innovation.
huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning
huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning
Published
October 27, 2025
Authors
Lucain Pouget Wauplin
Célina Hanouti
Lysandre
Julien Chaumond
TL;DR:
After five years of development, huggingface_hub has reached v1.0 – a milestone that marks the library’s maturity as the Python package powering 200,000 dependent libraries and providing core functionality for accessing over 2 million public models, 0.5 million public datasets, and 1 million public Spaces. This release introduces breaking changes designed to support the next decade of open machine learning, driven by a global community of almost 300 contributors and millions of users.
🚀 We highly recommend upgrading to v1.0 to benefit from major performance improvements and new capabilities.
pip install --upgrade huggingface_hub
Major changes in this release include the migration to httpx as the backend library, a completely redesigned hf CLI (which replaces the deprecated huggingface-cli) featuring a Typer-based interface with a significantly expanded feature set, and full adoption of hf_xet for file transfers, replacing the legacy hf_transfer. You can find the full release notes here.
We’ve worked hard to ensure that
huggingface_hubv1.0.0 remains backward compatible. In practice, most ML libraries should work seamlessly with both v0.x and v1.x versions. The main exception istransformers, which explicitly requireshuggingface_hubv0.x in its v4 releases and v1.x in its upcoming v5 release. For a detailed compatibility overview across libraries, refer to the table in this issue.
The Story Behind the Library
Every major library has a story. For huggingface_hub, it began with a simple idea: what if sharing machine learning models could be as easy as sharing code on GitHub?
In the early days of the Hugging Face Hub, researchers and practitioners faced a common frustration. Training a state‑of‑the‑art model required significant compute resources and expertise. Once trained, these models often lived in isolation, stored on local machines and shared via (broken) Google Drive links. The AI community was duplicating work, wasting resources, and missing opportunities for collaboration.
The Hugging Face Hub emerged as the answer to this challenge. Initially, it was primarily used to share checkpoints compatible with the transformers library. All the Python code for interacting with the Hub lived within this library, making it inaccessible for other libraries to reuse.
In late 2020, we shipped huggingface_hub v0.0.1 with a simple mission: extract the internal logic from transformers and create a dedicated library that would unify how to access and share machine learning models and datasets on the Hugging Face Hub. Initially, the library was as straightforward as a Git wrapper for downloading files and managing repositories. Five years and 35+ releases later, huggingface_hub has evolved far beyond its origins.
Let’s trace that journey.
The Foundation Years (2020‑2021)
The early releases established the basics. Version v0.0.8 introduced our first APIs, wrapping Git commands to interact with repositories. Version v0.0.17 brought token‑based authentication, enabling secure access to private repositories and uploads. These were humble beginnings, but they laid the groundwork for everything that followed.
The Great Shift: Git to HTTP (2022)
In June 2022, version v0.8.1 marked a pivotal moment: we introduced the HTTP Commit API. Instead of requiring Git and Git LFS installations, users could now upload files directly through HTTP requests. The new create_commit() API simplified workflows dramatically, especially for large model files that are cumbersome to use with Git LFS. In addition, a git‑aware cache file layout was introduced. All libraries (not only transformers, but third‑party ones as well) would now share the same cache, with explicit versioning and file deduplication.
This wasn’t just a technical improvement. It was a philosophical shift. We were no longer building a Git wrapper for transformers; we were building purpose‑built infrastructure for machine learning artifacts that could power any library in the ML ecosystem.
An Expanding API Surface (2022–2024)
As the Hub grew from a model repository into a full platform, huggingface_hub kept pace with an expanding API surface. Core repository primitives matured: listing trees, browsing refs and commits, reading files or syncing folders, managing tags, branches, and release cycles. Repository metadata and webhooks rounded up the offering so teams could react to changes in real time.
In parallel, Spaces emerged as a simple yet powerful way to host and share interactive ML demos directly on the Hub. Over time, huggingface_hub gained full programmatic control to deploy and manage Spaces (hardware requests, secrets, environment configuration, uploads). To deploy models on production‑scale infrastructure, Inference Endpoints were integrated as well. Finally, the Jobs API came later (Q3 2025) to complete our compute offering.
The social and community layers became first‑class citizens too: from APIs for pull requests and comments, to user and organization info, repository likes, following and followers, all the way through Collections to curate and share sets of related resources. Everyday ergonomics improved too: seamless authentication in Colab, resumable downloads, reliable uploads of large‑scale folders, and more.
Then came version v0.28.0 and the Inference Providers ecosystem. Instead of a single inference backend, we partnered with multiple serverless providers (Together AI, SambaNova, Replicate, Cerebras, Groq, and more) to serve one API with transparent routing. We adopted a pay‑per‑request inference architecture that matched how people actually wanted to work.
Ready. Xet. Go! (2024‑2025)
Version v0.30.0 introduced Xet, a groundbreaking new protocol for storing large objects in Git repositories. Unlike Git LFS, which deduplicates at the file level, Xet operates at the chunk level (64 KB chunks). When you update a large file in a dataset or a model, only the changed chunks are uploaded or downloaded, not the entire file.
The migration was massive, starting with 20 petabytes across over 500 000 repositories. Yet it happened transparently, with full backward compatibility. One year later, all 77 PB+ over 6 000 000 repositories have been migrated to the Xet backend, allowing for much faster (and smarter!) uploads and downloads. This happened with no user intervention, and no disruption to existing workflows 🔥
Measuring Growth and Impact
Measuring the growth and impact of an open‑source library is a tricky task. Numbers tell a story of their own:
-
113.5 million monthly downloads, 1.6 billion total (October 2025).
-
Powers access to 2 M+ public models, 500 k+ public datasets, 1 M+ public Spaces, and about twice as much when accounting for private repos.
-
Used by 60 k+ users daily, 550 k+ monthly.
-
Trusted by 200 k+ companies from startups to Fortune 500.
But the real scale becomes clear when you look at the ecosystem. huggingface_hub is a dependency for over 200 k repositories on GitHub and 3 000 packages on PyPI, powering everything from major third‑party frameworks like Keras, LangChain, PaddleOCR, ChatTTS, YOLO, Google Generative AI, Moshi, NVIDIA NeMo, and Open Sora, to countless smaller libraries and tools across the ML landscape. Our own ecosystem (transformers, diffusers, datasets, sentence‑transformers, lighteval, gradio, peft, trl, smolagents, timm, lerobot, etc.) benefits from this foundation as well.
The remarkable part? Most of the third‑party integrations happened organically, and we played no role in them. The Hugging Face Hub empowers the ML community in countless ways, yet we’re continually humbled by how far it has gone and how widely it’s used.
Building for the Next Decade
Version 1.0 isn’t just about reaching a milestone. It’s about building the foundation for the next decade of open machine learning. The breaking changes we’ve made aren’t arbitrary; they’re strategic decisions that position huggingface_hub to scale with the explosive growth of AI while maintaining the reliability that millions of developers depend on.
Modern HTTP Infrastructure with httpx and hf_xet
The most significant architectural change in v1.0 is our migration from requests to httpx.
0
Comments
Want to join the conversation?
Loading comments...