A while ago, I was working on a software development project which kept several version branches active in its source code repository. Their team checked in changes rapidly, as I puttered away on my part of the code. I was swamped by the effort of pulling their changes, branch by branch. So I found one way to make it easier: the git worktree feature. I set up another way: the “big sister” repository. Let me describe them to you. Maybe it will help you with your projects.

The project I was working on used git for version control. It had several version branches active. Let’s call them S2_6, S3_0, and main for the sake of illustration. (I will assume you know git well enough for everyday version control, and understand terms like “repo” and “remote”.)

When you clone the repository in the simple, straightforward way, git gives you a single local repository, with a work tree containing the .git/ repository directory hidden within. It is straightforward to work on any one version branch at a time. Check out version S2_6. Perform a git pull from the upstream repo. Make a branch with the S2_6 version of my changes. Repeat the git pull from time to time to get updates from the project.

But this became tiresome when I tried to work in, or read through, or test with, multiple version branches at the same time. To switch branches, I had to check in any changes to the previous version branch, check out the other version branch, and do a git pull or git merge to update my copy of the version branch. Then I found out that the previous version had left some files in the work tree which my other version branch did not use. They looked like unmanaged files which needed to be checked in. And it was hard to compare with the previous version branch’s code, because those files were no longer checked out.

What I wanted was a single repository with multiple work trees, each with a different branch checked out. Git has exactly this capability: the git worktree feature (and command).

I could, from my local repo directory, give a command like: 

% git worktree add ../S2_6
% cd ../S2_6

This put me in a directory named for the version branch S2_6, which contained a work tree, with branch S2_6 checked out. Meanwhile, my original project directory remained. And both S2_6 and my original directory shared a repository. That meant, I could perform a git fetch once, to get updates from upstream into the repository. Then from each work tree, I could to a git merge to merge the changes for that branch from upstream to my work tree.

There was a difficulty, however. I use a distribution of the Eclipse IDE to simplify git usage, and to help me code. Eclipse’s plugin for git does not support the git worktree feature and its multiple work trees. This limitation has been the subject of Eclipse enhancement request 477475, git 2.5 worktree support. The request was opened in 2015. Seven years later, it has attracted partial patches, but still no proper fix. This forces a choice: either use git worktree, but don’t use Eclipse even for the simple git commands which it normally can do just fine; or don’t use git worktree, and keep using git through Eclipse.

I chose the latter. I came up with a workaround I call the “big sister remote”.

The idea is to have an overall directory which contains the main project repository directory (e.g., project_main), and also a separate cloned repository for each version branch (e.g., project_S2_6). The structure looks like this:

- project_overall
   |-- project_main
   |-- project_S2_6
   +-- project_S3_0

The project_main repo has a remote named upstream , which points to the project’s upstream repository in the usual way. Each of the separate repositories has an additional remote named bigsister , which refers to the project_main repository via a file path (not a URL). It also has the usual remote named upstream.

This lets me do all the usual operations on each repo separately, including fetching from upstream. But I can also automate the fetching from upstream, via a simple shell script. Its essence is the following:

pushd "/.../project_overall/project_main"
git fetch upstream
if [[ $(git branch --show-current) == 'main' ]] ; then
    git merge upstream/main
fi
cd ..
# For each child repo corresponding to a version branch,
# fetch new commits from bigsister repo (../project_main/)
foreach child (S2_6 S3_0 S3_1) do
    cd "project_$child"
    git fetch bigsister upstream/"$child":remotes/upstream/"$child"   
	if [[ $(git branch --show-current) == "$child" ]] ; then
	    git merge upstream/"$child"
	fi
	cd ..
done
# Return to previous working directory
popd

So, when I want to get the latest from upstream in all my version repositories, I just run this script. It fetches and merges correctly, without requiring me to remember the details of git merge. As a bonus, the slow task of fetching code changes over the network from the remote upstream repository happens only once; the rest of the fetching is local, from the main repository.

So, if you find yourself wanting to have multiple branches checked out at once from the same project, consider git worktree, or the big sister repo, will be helpful for you.

No Comments »