Bacula Git Usage

This chapter is intended to help you use the Git source code repositories to obtain, modify, and submit Bacula source code.

Bacula Git repositories

As of September 2009, the Bacula source code has been split into three Git repositories. One is a repository that holds the main Bacula source code with directories bacula, gui, and regress. The second repository contains the directories docs directory, and the third repository contains the rescue directory. All three repositories are hosted by UKFast.

Previously everything was in a single SVN repository. We have split the SVN repository into three because Git offers significant advantages for ease of managing and integrating developer's changes. However, one of the disadvantages of Git is that you must work with the full repository, while SVN allows you to checkout individual directories. If we put everything into a single Git repository it would be far bigger than most developers would want to checkout, so we have separted the docs and rescue into their own repositories, and moved only the parts that are most actively worked on by the developers (bacula, gui, and regress) to a the Git Bacula repository.

Each major version of Bacula uses a specific branch name rather than the default git “master” branch. For example, the version 11.0.1 code will be accessible via the branch Branch-11.0. In this section, the word “master” refers to the current active development branch.

Bacula developers must now have a certain knowledege of Git.

Git Usage

Please note that if you are familiar with SVN, Git is similar, (and better), but there can be a few surprising differences that can be very confusing (nothing worse than converting from CVS to SVN).

The main Bacula Git repo contains the subdirectories bacula, gui, and regress. With Git it is not possible to pull only a single directory, because of the hash code nature of Git, you must take all or nothing.

For developers, the most important thing to remember about Git and the bacula.org repository is not to "force" a push to the repository. Doing so, can possibly rewrite the Git repository history and cause a lot of problems for the project.

You can get a full copy of the Bacula Git repository with the following command:

  git clone http://git.bacula.org/bacula.git bacula
  git checkout -b Branch-11.0 origin/Branch-11.0

This will put a read-only copy into the directory bacula in your current directory, and bacula will contain the subdirectories: bacula, gui, and regress. Obviously you can use any name an not just bacula. In fact, once you have the repository in say bacula, you can copy the whole directory to another place and have a fully functional git repository.

The above command needs to be done only once. Thereafter, you can:

cd bacula
git pull     # refresh my repo with the latest code

As of August 2009, the size of the repository (bacula in the above example) will be approximately 55 Megabytes. However, if you build from source in this directory and do a lot of updates and regression testing, the directory could become several hundred megabytes.

Learning Git

If you want to learn more about Git, we recommend that you visit:
book.git-scm.com/.

Some of the differences between Git and SVN are:

• Your main Git directory is a full Git repository to which you can and must commit. In fact, we suggest you commit frequently.
• When you commit, the commit goes into your local Git database. You must use another command to write it to the bacula.org repository (see below).
• The local Git database is kept in the directory .git at the top level of the directory.
• All the important Git configuration information is kept in the file .git/config in ASCII format that is easy to manually edit.
• When you do a commit the changes are put in .git rather but not in the main bacula.org repository.
• You can push your changes to the external repository using the command git push providing you have write permission on the repository.
• We restrict developers just learning git to have read-only access until they feel comfortable with git before giving them write access.
• You can download all the current changes in the external repository and merge them into your master branch using the command git pull.
• The command git add is used to add a new file to the repository AND to tell Git that you want a file that has changed to be in the next commit. This has lots of advantages, because a git commit only commits those files that have been explicitly added. Note with SVN add is used only to add new files to the repo.
• You can add and commit all files modifed in one command using git commit -a.
• This extra use of add allows you to make a number of changes then add only a few of the files and commit them, then add more files and commit them until you have committed everything. This has the advantage of allowing you to more easily group small changes and do individaual commits on them. By keeping commits smaller, and separated into topics, it makes it much easier to later select certain commits for backporting.
• If you git pull from the main repository and make some changes, and before you do a git push someone else pushes changes to the Git repository, your changes will apply to an older version of the repository you will probably get an error message such as:

   git push
   To git@github.com:bacula/bacula.git
    ! [rejected]        Branch-11.0 -> Branch-11.0 (non-fast forward)
    error: failed to push some refs to 'git@github.com:bacula/bacula.git'
  

  which is Git's way of telling you that the main repository has changed and that if you push your changes, they will not be integrated properly. This is very similar to what happens when you do an "svn update" and get merge conflicts. As we have noted above, you should never ask Git to force the push. See below for an explanation of why.

• To integrate (merge) your changes properly, you should always do a git pull just prior to doing a git push.
• If Git is unable to merge your changes or finds a conflict it will tell you and you must do conflict resolution, which is much easier in Git than in SVN.
• Resolving conflicts is described below in the github section.

Step by Step Modifying Bacula Code

Suppose you want to download Bacula source code, build it, make a change, then submit your change to the Bacula developers. What would you do?

• Tell git who you are:
  

  git config --global user.name "First-name Last-name"
  git config --global user.email "email@address.com"
  

  Where you put your real name and your email address. Since this is global, you only need to do it once on any given machine regardless of how many git repos you work with.

• Download the Source code:
  

  git clone http://git.bacula.org/bacula.git bacula
  

• Configure and Build Bacula:
  

  ./configure (all-your-normal-options)
  make
  

• Create a branch to work on:

  cd bacula/bacula
  git checkout -b bugfix Branch-11.0
  

• Edit, build, Test, ...
  

  edit file jcr.h
  make
  test
  

  Note: if you forget to create a working branch prior to making changes, and you make them on the development branch, this is no problem providing that you create the working branch before your first commit. So assuming that you have edited “master” instead of your bugfix branch, you can simply:

  git checkout -b bugfix master
  

  and a new bugfix branch will be created and checked out. You can then proceed to committing to your bugfix branch as described in the next step.

• commit your work:

  git commit -am "Short comment on what I did"
  

• Possibly repeat the above two items

• Switch back to the master branch:
  

  git checkout master
  

• Pull the latest changes:
  

  git pull
  

• Get back on your bugfix branch:
  

  git checkout bugfix
  

• Merge your changes and correct any conflicts:
  

  git rebase master bugfix
  

• Fix any conflicts:
  You will be notified if there are conflicts. The first thing to do is:

  git diff
  

  This will produce a diff of only the files having a conflict. Fix each file in turn. When it is fixed, the diff for that file will go away.

  For each file fixed, you must do the same as SVN, inform git with:

  git add (name-of-file-no-longer-in-conflict)
  

• When all files are fixed do:

  git rebase --continue
  

• If you find that it is impossible to reconcile the two branches or you made a mistake in correcting and adding files, before you enter the:

  git rebase --continue
  

  you can instead enter:

  git rebase --abort
  

  which will essentially cancel the the original git rebase and reset everything to the beginning with no changes to your bugfix branch.

• When you have completed the rebase and are ready to send a patch, do the following:
  

  git checkout bugfix
  git format-patch -M master
  

  Look at the files produced. They should be numbered 0001-xxx.patch where there is one file for each commit you did, number sequentially, and the xxx is what you put in the commit comment.

• If the patch files are good, send them by email to the developers as attachments.

• Then you can continue working on your code if you want, or start another branch with a new project.

• If you continue working on your bugfix branch, you should do a git rebase master from time to time, and when your changes are committed to the repo, you will be automatically synchronized. So that the next git format-patch will produce only the changes you made since the last format-patch you sent to the developers.

More Details

Normally, you will work by creating a branch of the development branch of your repository, make your modifications, then make sure it is up to date, and finally create format-patch patches or push it to the bacula.org. Assuming you call the Bacula repository bacula, you might use the following commands:

cd bacula
git checkout bacula
git pull 
git checkout -b newbranch bacula
(edit, ...)
git add <file-edited>
git commit -m "<comment about commit>"
...

When you have completed working on your branch, you will do:

cd bacula
git checkout newbranch  # ensure I am on my branch
git pull                # get latest source code
git rebase master       # merge my code

If you have completed your edits before anyone has modified the repository, the git rebase master will report that there was nothing to do. Otherwise, it will merge the changes that were made in the repository before your changes. If there are any conflicts, Git will tell you. Typically resolving conflicts with Git is relatively easy. You simply make a diff:

git diff

Then edit each file that was listed in the git diff to remove the conflict, which will be indicated by lines of:

other text

where text is what is in the Bacula repository, and other text is what you have changed.

Once you have eliminated the conflict, the git diff will show nothing, and you must do a:

git add <file-with-conflicts-fixed>

Once you have fixed all the files with conflicts in the above manner, you enter:

git rebase --continue

and your rebase will be complete.

If for some reason, before doing the -continue, you want to abort the rebase and return to what you had, you enter:

git rebase --abort

Finally to make a set of patch files

git format-patch -M master

When you see your changes have been integrated and pushed to the main repo, you can delete your branch with:

git checkout master
git branch -D newbranch

Forcing Changes

If you want to understand why it is not a good idea to force a push to the repository, look at the following picture:

The above graphic has three lines of circles. Each circle represents a commit, and time runs from the left to the right. The top line shows the repository just before you are going to do a push. Note the point at which you pulled is the circle on the left, your changes are represented by the circle labeled Your mods. It is shown below to indicate that the changes are only in your local repository. Finally, there are pushes A and B that came after the time at which you pulled.

If you were to force your changes into the repository, Git would place them immediately after the point at which you pulled them, so they would go before the pushes A and B. However, doing so would rewrite the history of the repository and make it very difficult for other users to synchronize since they would have to somehow wedge their changes at some point before the current HEAD of the repository. This situation is shown by the second line of pushes.

What you really want to do is to put your changes after Push B (the current HEAD). This is shown in the third line of pushes. The best way to accomplish this is to work in a branch, pull the repository so you have your master equal to HEAD (in first line), then to rebase your branch on the current master and then commit it. The exact commands to accomplish this are shown in the next couple of sections.