working with git submodules in Ceph

loic

The gf-complete and jerasure libraries implement the erasure code functions used in Ceph. They were copied in Ceph in 2013 because there were no reference repositories at the time. The copy was removed from the Ceph repository and replaced by git submodules to decouple the release cycles.

If a change needs to be made urgently (shortly before a Ceph release for instance), it may be inconvenient to wait for a pull request to be reviewed and merged. Copies of the repositories were created in the ceph namespace for both jerasure and gf-complete.
A stable branch was created for jerasure v2 and gf-complete v1. The git submodules point to these release branches as follows:

git submodule add -b v1 https://github.com/ceph/gf-complete.git src/erasure-code/jerasure/gf-complete git submodule add -b v2 https://github.com/ceph/jerasure.git src/erasure-code/jerasure/jerasure

When checking out a branch with the new submodules, it is necessary to update them to clone and checkout the required branch.

git submodule update

To work with jerasure upstream, three remotes are defined : one for the upstream (jimplank read only), one for a fork of the upstream at a location suitable to issue a pull request (loic read-write), one for the mirror from the ceph (ceph read-write).

$ git remote add loic git@bitbucket.org:dachary/jerasure.git $ git remote -vv ceph git@github.com:ceph/jerasure.git (fetch) ceph git@github.com:ceph/jerasure.git (push) jimplank http://jerasure.org/jerasure/jerasure.git (fetch) jimplank http://jerasure.org/jerasure/jerasure.git (push) loic git@bitbucket.org:dachary/jerasure.git (fetch) loic git@bitbucket.org:dachary/jerasure.git (push)

A local branch wip-compilation-warnings is created from the upstream master branch jimplank/master.

$ git checkout -b wip-compilation-warnings jimplank/master Branch wip-compilation-warnings set up to track remote branch master from jimplank. Switched to a new branch 'wip-compilation-warnings' loic@fold:~/software/ceph/jerasure$ git branch -vv * wip-compilation-warnings 87f3010 [jimplank/master] remove unused variables

When the work is completed, it is pushed

git push loic wip-compilation-warnings

and a pull request is sent upstream. When merged into master, the commits are backported to the stable branch v2:

git checkout -b wip-compilation-warnings-v2 jimplank/v2 git cherry-pick -x da3b767 git push loic wip-compilation-warnings-v2

and a pull request is sent upstream.

The integration with Ceph can be tested localy by modifying the URL of the upstream from the ceph namespace to the location where the pull request is waiting to be reviewed:

diff --git a/.gitmodules b/.gitmodules index b51c509..e22aaac 100644 --- a/.gitmodules +++ b/.gitmodules @@ -13,5 +13,5 @@ branch = v1 [submodule "src/erasure-code/jerasure/jerasure"] path = src/erasure-code/jerasure/jerasure

The submodule must be updated to take this modification into account and point to the branch including the latest change with:

git submodule sync git submodule update --remote src/erasure-code/jerasure/jerasure

If the upstream acceptance takes time, the backport can be pushed to the Ceph namespace to be used immediately:

git checkout ceph/v2 git cherry-pick -x da3b767 git push ceph v2

And the the Ceph repository updated to point to the latest commit of the v2 branch with:

git submodule update --remote src/erasure-code/jerasure/jerasure git add src/erasure-code/jerasure/jerasure git commit -m 'sync latest upstream'