Source control for the age of agents

A companion to my JJ Con talk, Exploring a Jujutsu without git.

At Cosine, we hosted more than 80,000 repositories. Working on their storage meant dealing with Git’s packing, repacking, and repository maintenance. That experience shaped what I wanted to build next.

For Composal, I wanted blob storage, such as S3, to be the durable home for repository data. I wanted us to control how that data was stored, retrieved, and maintained. Building around Git meant continuing to accommodate its storage machinery, or taking responsibility for replacing parts of it.

I don’t believe Git should remain the default foundation for everything we build in source control. Its place in our tooling makes compatibility important. But we should be able to reconsider the implementation underneath it, especially as agents change how often code is checked out, revised, and submitted for review.

Operating a Git repository

Git packs many objects—file contents, directory trees, and commits—into compressed files. It can store differences between similar objects to save space. Indexes help it find those objects again. As a repository grows, maintenance combines packs, updates indexes, and eventually removes redundant data.

That work consumes CPU, I/O, and temporary disk space. Git has incremental and geometric repacking to limit how much gets rewritten. Across a hosting service, you still have to schedule the work, provide resources for it, and keep repositories available while it happens.

Saving a completed pack in S3 is straightforward. Serving an actively changing repository from object storage requires more: deciding what to cache locally, which packs to read, when to compact them, and how to publish a new pack inventory without disrupting readers. The storage bill is only part of the system you have to operate.

That’s the work I wanted more freedom to design. Git’s repository format and maintenance tools were a substantial commitment to carry into a new platform.

Keeping Git at the boundary

There are serious efforts to change this architecture while retaining Git. WAL-Git, for example, makes a write-ahead log in object storage authoritative and treats its servers as disposable caches. It still uses upstream Git for operations including repacking, but changes where the durable state lives and how servers coordinate around it.

That is a useful example of how far you can go without changing the client. It also makes the choice clearer: which parts of Git do we want to retain, and which parts do we want to own?

For Composal, that question extends to the version-control model. Jujutsu, or JJ, gives a change an ID that survives amendments and rebases, and can rebase dependent changes when their parent is edited. We use those relationships in Composal’s review workflow. A developer can revise a change in their checkout and upload another version to the same review without separately describing its place in a stack.

JJ’s backend interface also lets us implement object storage while reusing its merge and rebase engine. We can work on storage and transfers through that interface. We don’t have to persuade Git to adopt our design or maintain a replacement for the whole version-control engine.

That’s why we chose JJ. We wanted its change model and the freedom to build the hosting system around it. Those JJ features also work with Git storage; choosing a native backend is a further decision we made for Composal.

Why we came back to packing

Our first ambition was a packless engine. We would store each object under its own content hash and retrieve it by that key. A file edit would create a new object; previously stored objects would remain unchanged.

The appeal was a small storage interface: put an object, get an object. Then we measured a checkout with a cold client cache.

One trace recorded 2,930 separate requests for file contents, with 32 reads already running at a time. Concurrency overlapped the requests, but every file still incurred request and cache work. A repository with many small files could spend substantial effort moving very little data in each request.

We needed to group objects for download. That brought packing back into the design, with a much clearer reason for it.

Individual requests

1. Fetch a file
2. Cache that file

Repeat for each file the checkout needs.

Grouped download

1. Fetch a chunk
2. Decode it
3. Cache its files

One request can deliver multiple files.

We explored grouping and compression, learning from work such as Meta’s Mononoke Packblob. Composal’s native transfer format carries complete objects in compressed chunks that the client can verify and decode into its cache.

There are costs to that choice. Complete objects can take more space than good delta compression. Someone still has to build the chunks, manage caches, and reclaim storage. Immutable data alone isn’t an advantage over Git: completed Git packs are immutable too.

The freedom we gained is in choosing those trade-offs ourselves. We can change how related objects are grouped and transferred while keeping JJ’s object interface intact. The experiment changed our goal from eliminating packs to controlling the cost of building, reading, and maintaining them.

The workload we’re building for

When an agent starts in a fresh sandbox, it can’t rely on the warm checkout a developer has used for months. Downloading the repository and writing files to disk are part of the time before it can start a task. A shared cache may help, but its effectiveness becomes something the platform has to engineer and measure.

We expect teams using agents to create and discard more working environments and to produce more revisions for review. That makes the cost of preparing each environment, retaining the resulting work, and serving it again more consequential. It’s why I want control over both the storage system and the workflow built on it.

Teams should be able to adopt that work gradually. Composal supports Git import, configured GitHub synchronization, and submission from Git checkouts alongside the native JJ workflow. Git cannot represent every JJ state, so compatibility has boundaries. We can keep supporting Git tools while developing the native path.

At JJ Con, I’ll go through the backend and the experiments behind these choices. I want to contribute that experience to JJ’s work on backend APIs and hosting integrations, so more people can build services around it. Composal is our opportunity to pursue that direction with the teams who will use it.