GitHub Redesigns Git Storage for Agent-Scale Workloads

GitHub unveiled a new storage architecture designed to handle thousands of concurrent writes from autonomous agents. The system exploits a fundamental property of Git: repository objects are immutable and content-addressed, so multiple writers can store them simultaneously without coordination. Only reference updates—branch and tag pointers—require agreement between writers. By separating these concerns, GitHub decoupled storage, compute, and serving so each can scale independently.

The architecture, called Project Rubicon, splits persistent state between Azure Blob Storage for Git objects and a separate Ref-CAS log service for references and repository metadata. A fleet of disposable virtual replicas handle reads and writes, pulling data into local caches as needed. When repository load increases, additional replicas are automatically provisioned. Background compaction nodes continuously optimize stored data without competing for replica resources. The system targets support for repositories handling thousands of writes per second and thousands of concurrent agents.

This redesign addresses a fundamental shift in development patterns: autonomous systems now generate sustained parallel write traffic rather than bursty human activity. GitHub’s planning reflects expectations that agent-driven workloads will grow exponentially, requiring architecture that scales orders of magnitude beyond current peak loads while maintaining familiar Git semantics.