Quick Links
Large Git repositories can use a lot of resources and slow down your computer. This can sometimes be mitigated by using shallow cloning, which throws away all but the most recent commits, making it easier to download Git repos with long histories.
What is Shallow Cloning?
Cloning a repository not only creates a copy of the entire repository, but also its complete commit history. For most repositories, the overall size and length of the commit history is not a problem. However, for really large repositories, cloning all that data can be time-consuming and resource-intensive.
For example, the Linux kernel has used the same repository since the dawn of time. It has over 1.1 million commits. Cloning this can take upwards of an hour on older hardware, and use multiple gigabytes of RAM just for the Git process alone. Not every repository is this huge of course, but some come close, and even Linux kernel contributors don’t want to sit at their keyboard for an hour watching it clone.
Shallow cloning is a technique that fetches only a limited number of recent commits. This results in a lightweight and faster clone that still retains the necessary functionality for your work, since most people don’t care about older commits, and you can still submit PRs and push changes without the entire history.
This also has great benefits for Continuous Integration pipelines (CI/CD), since the build server likely does not need to know the entire Git history just to build the application. If you’re writing scripts that work with Git repos and only care about the contents, it may be worth it to use shallow cloning instead.
Related:How to Clone a GitHub Repository
Besides just cloning time, shallow repositories run faster in general, since there’s less history to process when running everyday commands. Really long histories can also cause GUI Git clients to lag, which can ruin the experience.
Shallow Cloning a Git Repository
The easiest way to shallow clone is to just use the–depthparameter ongit clone. This will limit the clone to a certain number, like 100 commits before the current repository HEAD. All other older commits will be truncated from the history.
You can also shallow clone everything after a certain date, which is much more useful if you don’t know how many commits you need.
The date parameter can acceptmany different formats, but a simple “X years/months ago” generally works fine.
There are also methods toshallow-ize an existing Git repository, but they’re usually messy and involve rewriting history and manually clearing all old objects from Git’s garbage collection cache. We would recommend just pushing all changes, removing the old repository, and re-cloning from the remote if you want to reduce your existing repo size.
Shallow Cloning Only a Single Branch
Shallow cloning can be combined with another Git feature that limits the clone to only a single branch. If you’re just downloading a repository to check out a feature branch, you can run this command instead:
Blobless and Treeless Clones
Another similar option that can still preserve commit history isusing blobless clones. In Git, the actual file contents are stored as “blobs,” and Git adds on top of this trees and commits that track how those blobs go together.
Since Git mostly only cares about the commits and the tree structure, you can choose to omit the blobs, and download them on-demand when they’re needed. This can significantly speed up cloning times while also maintaining history. You can also combine blobless and shallow clones.
To do this, rungit clonewith a filter set:
Treeless clones are similar, and often used for automation, but are typically not recommended for daily use since they can actually be slower for everyday Git operations.
Typically, however, shallow clones will still be faster, and blobless clones will grow in size over time as more parts of the repo need to be accessed.
