This document is still a work in progress.

Revision Control at Code-Host with Subversion and Insurrection

Scope

The scope of this document is to describe and define some basic methods of using Subversion in the development process. These proceedures are mainly focused on the software development process for product release and support. This document is intended to be an adjunct to the Subversion documentation and not a replacement for it. This is also not a a replacement for a more user-oriented introduction to Subversion and the TortoiseSVN integration into the Windows Explorer.

Subversion Revision Control

The revision control system we have instituted here is known as Subversion. It is a very powerful, modern revision control solution that works with many different platforms and tools. It was build as a better revision control system than CVS while still keeping as much of the feel and desireable features of CVS. This means that for the basic, day-to-day operations, Subversion is very much like CVS.

Subversion does have a number of improvements over CVS, with the major ones being atomic operations such as commits and the tracking of meta-data, including rename operations and filesystem flags. Subversion also brings with it a much more flexible branching and tagging system. In fact, it is this extreem flexibility which makes it important to define specific practices as Subversion does not directly enforce any specific or single view of those practices.

For more detailed technical documentation on the actual use of Subversion, please see the Subversion book in HTML format or PDF format. The book is also available as an O'Reilly printed book but the digital versions tend to be more up-to-date due to physical publication overhead.

Tools you will need

One of the powerful features of Subversion is that it operates via HTTP/WEBDAV protocols. This means that any modern browser can browse the current version of files in the repository. However, to make full use of the revision control features, you will need a Subversion client tool. The current client code is available on the Insurrection Server or should be installed as part of your development environment. If you are running a Windows environment, please install the Subversion command line tools and the TortoiseSVN GUI. Many Linux systems should already have the base command line tools installed.

The Subversion Server also has access controls in place. In order to access the various repositories, you will need an account defined. Once you have an account, you will have read-only access to all repositories and read-write access to those repositories that you have been associated with a manager. All users are also given access to the example repository within which you can play around without affecting actual active repositories. The example repository will be where some of the examples within this document will be shown.

Getting Started

If you don't have an account on the Insurrection server, please get one now. The username of the account should be the same as your EMail address. This is important as Insurrection will be sending EMail to you with your new account password. (There may be other important EMail from the Insurrection server in the future.)

Once you have your account, go to password change web interface and log in using your user name and password. You should also take a moment to change the password to something secure. Note that the Insurrection server passwords are stored in a one-way encrypted format and thus can not be recovered but they may be reset by an administrator.

After having set your password, you should go to the example repository and browse around. Note that the repository browsing features require a web browser that is XML and XSLT compatible. (This means most any modern web browser, albeit Safari needs to be Mac OS-X 10.3.9 or later.)

Differences from Visual SourceSafe

Subversion is modelled after CVS (Concurrent Version System) which is designed for concurrent development. This model has proven itself to scale very well with both the size of the project and the number of developers. This is not the model of Visual SourceSafe (VSS) and thus there are some differences in the work flow that due to this core difference in operation.

The major difference is that when working with Subversion (and CVS), you do not normally have "locks" on files. That is, the local files are always available for you to edit/update. The normal process is to edit what you need to and when you are ready to "commit" the changes back to the repository, you would issue the commit operation. This may tell you that some files that you changed have also be changed by others and you will need to merge the changes. Usually this process goes very smoothly and the changes merge automatically.

A major technical benefit of Subversion is the fact that commits are atomic. That means, if you have changed 5 files as part of an update, doing the commit of the whole will commit all 5 of the changed files as a single, atomic operation. In VSS this would end up having each file individually commited and thus potentially split and not trackable as a single item.

Another difference is the user interface. There are "VSS-like" user interfaces available for Subversion, however the TortoiseSVN interface which integrates right into the Windows Explorer and Desktop provides for nearly seemless interaction with Subversion. There is also the very powerful command line tool which can do even more (albeit without the fancy GUI as the command line tool really is a command line tool).

The other major change is the tagging and branching support. Subversion has very advanced branching and tagging support, albeit the features are so flexible that they do not enforce any one operational convention. The following sections will describe what I feel is the best convention for our use. This will change as we find new and different processes that help our productivity.

Subversion Basics

This section will describe the usage of Subversion in the terms of the SVN commands and operations. These operations may be named slightly differently in the various GUIs that are available. I will use the terms as they exist in the core "SVN" command line tools.

One note of importance that I would like to state here, even before we run into it: When dealing with files that are under revision control, it is important that you use the revision control system to do file operations such as rename, delete, copy, etc. This is because Subversion actually tracks these things in the repository and unless it is told about this, it can not track them.

The Repository

Subversion uses HTTP protocol to talk with the repository. As such, repository location identifiers look like standard URLs and can be accessed in a simple manner with a standard web browser. The full URLs are only needed when referencing a repository location you do not currently have checked out on your local system.

We actually have multiple repositories at MKSoft in order to easily support different access rules for each repository. Everyone with a Subversion access account gets full access to the Example repository which this document will make use of.

Each repository is broken into 4 core directories that have some special meaning as to the rules I have established for the revision control system. Subversion's flexibility here is so great that it actually does not impose any structure at all so this structure is something we impose on ourselves. The structure is documented the the Readme.txt file that is part of every repository. It basically reads:

```
/trunk
```
This is the trunk of the source revision tree. It contains the current development branch. Most new work should happen here.
```
/branches
```
This tree is for the branches that we build. A branch is just a special "copy" of the source tree within Subversion that started at some specific revision of the code. A simple branch is done by just using svn copy of the /trunk to some named branch in /branch/somename_1.0. Even more interesting branches can be made by using svn copy to copy from your local checkout to the branch. This will make sure that the exact versions in your local checkout is what the branch contains.
- ```
/branches/personal
```
  Within the /branches tree is the /branches/personal tree for personal branches, such as when doing some in-depth work on something and wish to not clutter the main tree with it until it is working enough to do so.
```
/tags
```
This tree is for revision tags. This is the way Subversion likes to deal with tags. You name them by the directory you create within this tree. Thus, the tag name is the name of the directory in the tags directory. Note that tags and branches are exactly the same from Subversion's standpoint. It is up to us to make sure that tags are tags, and branches are branches. That is, you should never commit to a tag but rather first create a branch and do your work there.
- ```
/tags/personal
```
  Within the /tags tree is the /tags/personal tree for personal tags, such as bookmarking some work in progress for future diff work.
```
/releases
```
This tree is where we svn copy a tag that actually made it to production. We should name it with something that ties it to the product version or ChangeControl process (such as using the ChangeControl number as the first part of the name) This is here such that, while we may have multiple tags for a specific project, only a few of them actually made it to production and we can then easily go here to see what is in production.

What the structure of the tags and releases directories should look like is up to your project needs. I would recommend sub directories in those trees by the name of the product if you have more than one to maintain. This then gives a natural separation of the tags/releases. The same could be done in the branch tree if you want to segrigate your branches.

Checking out a part of a repository means that you are getting a local copy of that section of the repository within which you can work. This does not involve locking the repository. Once local, you can then do operations on the object within the repository. The standard flow would look something like this:

svn checkout http://svn.code-host.net/svn/example/trunk/fold/

A  fold/fold.js
A  fold/Base_More.gif
A  fold/Base_Last.gif
A  fold/Minus_More.gif
A  fold/Minus_Last.gif
A  fold/Item_More.gif
A  fold/Item_Last.gif
A  fold/Sub_Closed.gif
A  fold/Sub_Opened.gif
A  fold/fold.html
A  fold/Plus_More.gif
A  fold/Plus_Last.gif
Checked out revision 40.

At this point the directory fold was created to hold the local image of the repository as seen from the url. Unless you get rid of this directory, there is no more need to do a checkout.

For more detailed and compete examples, please see chapter 3, section 4 of the Subversion book.

Simple update/edit/commit cycle

Working with the same fold that we got from the repository above, we want to make sure we are up to the latest version. If a "VSS get latest" is needed, one would just:

cd fold
svn update

U  fold.js
U  fold.html
A  subdir
A  subdir/readme.txt
Updated to revision 47.

This would update all of the files that have newer revisions on the server and add any files that have been added to the server and remove any files that have been removed. In this case, two files were updated, a directory was added and a file in that directory was added.

After editing files, one would just issue the commit operation at the outer most directory containing the changes. By doing so, all of the changes will be part of the same atomic operation and thus will be seen/tracked/logged together. This does not mean you can not manually get only one of the changes, but it does make it much easier to get all of the related changes and less likely that something goes unnoticed. The commit operation asks for a commit message. It is important that this message be useful and meaningful as they are used to find out what the changes are in a repository. For example, the output (minus the commit message entry) would be done like this:

cd fold
svn commit

Sending        fold.html
Sending        subdir/readme.txt
Transmitting file data ..
Committed revision 48.

As can be seen here, only two files were updated for this changed.

For more detailed and compete examples, please see chapter 3, section 5 of the Subversion book.

You can see the history of a given file or a whole tree by using the log operation. The history of a file would look like this:

cd fold
svn log subdir/readme.txt

------------------------------------------------------------------------
r48 | msinz | 2005-01-18 10:28:15 -0500 (Tue, 18 Jan 2005) | 2 lines

Some minor changes to show how a commit process works

------------------------------------------------------------------------
r47 | msinz | 2005-01-18 10:02:44 -0500 (Tue, 18 Jan 2005) | 2 lines

Added a subdir to the tree for some example work

------------------------------------------------------------------------

You can also do a whole tree:

cd fold
svn log .

------------------------------------------------------------------------
r48 | msinz | 2005-01-18 10:28:15 -0500 (Tue, 18 Jan 2005) | 2 lines

Some minor changes to show how a commit process works

------------------------------------------------------------------------
r47 | msinz | 2005-01-18 10:02:44 -0500 (Tue, 18 Jan 2005) | 2 lines

Added a subdir to the tree for some example work

------------------------------------------------------------------------
r41 | msinz | 2004-11-29 11:38:58 -0500 (Mon, 29 Nov 2004) | 2 lines

Just some example changes - no real impact

------------------------------------------------------------------------
r40 | lwang | 2004-11-17 12:13:10 -0500 (Wed, 17 Nov 2004) | 2 lines

Just an example of how to commit

------------------------------------------------------------------------
r2 | (no author) | 2004-10-18 06:57:09 -0400 (Mon, 18 Oct 2004) | 2 lines

This is a simple example directory containing my Folding HTML tree code

------------------------------------------------------------------------

Even more powerful is the fact that the logs can contain more details, such as the files that were changed/added/deleted as part of the operations. For example, if I do this operation on a single file, I see:

cd fold
svn log -v subdir/readme.txt

------------------------------------------------------------------------
r48 | msinz | 2005-01-18 10:28:15 -0500 (Tue, 18 Jan 2005) | 2 lines
Changed paths:
   M /trunk/fold/fold.html
   M /trunk/fold/subdir/readme.txt

Some minor changes to show how a commit process works

------------------------------------------------------------------------
r47 | msinz | 2005-01-18 10:02:44 -0500 (Tue, 18 Jan 2005) | 2 lines
Changed paths:
   A /trunk/fold/subdir
   A /trunk/fold/subdir/readme.txt

Added a subdir to the tree for some example work

------------------------------------------------------------------------

Here I see what all was related with that commit. For example, the commit for repository revision 48 had not just the readme.txt file but also the fold.html file as part of the commit. The same information can be had for a whole tree of files.

For more detailed and compete examples, please see chapter 3, section 6 of the Subversion book.

Branches and Tags

This is where correct use of Subversion can significantly improve the development process relative to VSS. If you already read about tagging and branching in the Subversion book, you already know that there really is no difference as far as Subversion is concerned. There is a semantic difference that is, with some extra scripts on the server, enforced for our repositories.

Branches are cheap

Subversion makes branching cheap. They happen very quickly and are very low overhead on the server. Thus do not worry about the number of branches you may wish to make.

A branch in Subversion is a copy of part of the repository into another place in the repository via the svn copy command. The branch still is associated with the original files and has their whole history available to it.. It also knows where in that history it was made.

Branches should be made at a minimum for each line of code development. Taking an example from the Backoffice development process, when we start to get ready for a new product release, we cut a branch for that specific release target from the trunk. For example, we made a branch called Backoffice_2.0.1 in the branches tree when we started the Backoffice 2.0.1 stabilization process. Future work that was outside the scope of 2.0.1 continued in the trunk while the 2.0.1 work was going on. At some point defined by the team, the changes from 2.0.1 branch work will be merged back into the trunk via the Subversion three-way merge tool. In most cases this will "just work" but if a section of code was changed in the trunk and the branch, some intervention will be needed. The amount of intervention depends on how much simular code changes were done in both places.

Since branches are so cheap to manage, you can even cut a branch for each bug that is being worked on. What this does is provide a stable and isolated revision tree for that bug work. Once the bug has been fixed, the branch can be the merged back to the source tree or trees that need that fix. Since the branch had only the bug fix in it, the Subversion differences within that branch will be the "change set" for the bug fix and thus the source code "patch" for the bugfix.

Once branches are no longer active and the changes have been merged back or abandoned, you can then "delete" the branch. This does not actually delete the information but it removes the branch from the active view of the repository. The branch is still available in the repository history and thus the can be called back to life at any time by revision id or date.

Using the fold example again, lets make a branch to work on. Note that we are using the current local tree as the source for the branch as that will then make sure that the branch is what you have locally, including any local changes that you may not have committed yet. (Thus, you can choose to make the branch after you have realized that the changes you are making are too complex or too risky to do in the current location.) Here we make a branch at the root of the branches tree called fold_test_1.0.0:

cd fold
svn copy . http://svn.code-host.net/svn/example/branches/fold_test_1.0.0

Committed revision 49.

Note that the command will ask you to enter a comment of what is being done. Please be descriptive.

Now, to get the branch locally, we can do a number of things. The commonly used method would be to check out the branch into its own local directory tree. You then would be able to work on the new branch in that location and the original one in the original location. The process of the checkout is just as before. There is no special behavior for dealing with a branch other than the checkout needs the initial URL to point to the branch. This would be the same URL used to make the branch.

There is another option - you can take your current source tree and "switch" it to a different branch. This is very useful if you don't want to change the directory the source is located in due to some external tool or IDE. Using the switch feature can also be much faster in that Subversion knows what needs to be changed and will only transmit the changes needed to execute the switch. For this example, we will use this method as it is different than the checkout method already described.

cd fold
svn switch http://svn.code-host.net/svn/example/branches/fold_test_1.0.0

At revision 49.

At this point, your local tree (starting at the fold directory) is connected with the branch we just made. You can make your changes here and the commit operations will be to the branch.

For more detailed and compete examples, please see chapter 4, section 2 and chapter 4, section 5 of the Subversion book.

After making some changes to the files in the branch and committing the changes, we want to merge that back to the trunk. First we will want to know how much to merge. Since we have not merged since creating the branch, this can easily be found by doing a log that traces back to the branch creation.

cd fold
svn up

At revision 52.

svn log --stop-on-copy

------------------------------------------------------------------------
r52 | msinz | 2005-01-19 10:39:09 -0500 (Wed, 19 Jan 2005) | 2 lines

A new file created in the branch

------------------------------------------------------------------------
r51 | msinz | 2005-01-19 10:32:20 -0500 (Wed, 19 Jan 2005) | 2 lines

Some more changes to play with the branch/merge features

------------------------------------------------------------------------
r50 | msinz | 2005-01-19 10:31:37 -0500 (Wed, 19 Jan 2005) | 2 lines

A change to the readme for showing what happened in the branch

------------------------------------------------------------------------
r49 | msinz | 2005-01-19 10:10:28 -0500 (Wed, 19 Jan 2005) | 2 lines

Make a test branch of the fold tree

------------------------------------------------------------------------

The svn update was done before this in order to refresh information from the server. It is generally a good idea to do this before doing operations where you want to know what is within a branch. As you can see the branch starts at revision 49.

Next we need to switch back to the trunk:

cd fold
svn switch http://svn.code-host.net/svn/example/trunk/fold

U  fold.js
U  fold.html
U  subdir/readme.txt
D  subdir/note.txt
Updated to revision 52.

Note that this shows all of the changes that were done to your local system in order to convert from the branch back to the trunk. Thus this gives you a look at what the scope of the changes were.

Now, we merge that branch into our trunk. First, lets see what would be done by using the dry-run option. Note that we want all of the changes in the branch starting at the start of the branch until the last revision in the branch, which in this case happens to be the head of the tree, but it is best never to assume that. The dry-run does not change any files, it just is useful to get a guess as to the extent of the merge.

cd fold
svn merge --dry-run -r 49:52
    http://svn.code-host.net/svn/example/branches/fold_test_1.0.0

U  fold.js
U  fold.html
A  subdir/note.txt
U  subdir/readme.txt

(Due to complexities of the merge, this is only a very good guess as there are special cases it can not notice during a dry run.)

Now, lets actually do the operation:

cd fold
svn merge -r 49:52
    http://svn.code-host.net/svn/example/branches/fold_test_1.0.0

U  fold.js
U  fold.html
A  subdir/note.txt
U  subdir/readme.txt

At this point, the files have been changed but not committed into the trunk. You can see the status of the files:

cd fold
svn status

M      fold.js
M      fold.html
A  +   subdir/note.txt
M      subdir/readme.txt

It would be at this time any conflicts would be addressed. This example does not have any, so all that is needed is a commit to get the merged changes into the repository.

cd fold
svn commit

Sending        fold.html
Sending        fold.js
Adding         subdir/note.txt
Sending        subdir/readme.txt
Transmitting file data ...
Committed revision 53.

During the commit process you will be asked to enter a commit note. You should describe what was done, which in this case was a merge from a branch. Including the branch merge information in this note is a good idea such that you can always go back to the log and figure out what the merge was and even try to reproduce or reverse the merge.

For more detailed and compete examples, please see chapter 4, section 3 and chapter 4, section 4 of the Subversion book.

Tags are very cheap

Tags are just like branches within Subversion but we add an extra semantic rule that says a tag never changes. That is, if you tag something, every time you get data from that tag it will be exactly the same. This is a critical concept for revision control systems as you would want tags for each release of a product. The Subversion repositories have be configured to enforce the rule that tags can not change. This way we can be sure that once a tag is set there is nothing that can change it.

Using the Backoffice 2.0.1 release project as an example, every time we do a build that goes to QA, we tag that. Those tags are of the form Backoffice_2.0.1.n were the n is the build number. This way, we are able to always get back to exactly the source and build process that produced the release. Once a specific build gets out of QA and into production, we copy/tag that specific release into the releases tree under the formal release name.

Using the fold tree as an example, lets make a tag:

Once we finish with the branch and have merged all of the changes back to the trunk, we normally delete the branch such that we know it is not active.

cd fold
svn copy . http://svn.code-host.net/svn/example/tags/fold_1.0.0-1

Committed revision 54.

Note that the command will ask you to enter a comment of what is being done. Please be descriptive.

For more detailed and compete examples, please see chapter 4, section 6 of the Subversion book.

Note - I have run into cases where the system thought that your local files were out of date and would not let a tag be made. This is always addressed by doing an svn up command but that may update files you do not want updated. You can update without recursion using the -N option. I do not know why this condition exists other than it is exposed due to our strict handling of tags.

Changes from a Tag

As stated before, tags can not change - they are immutable. Sometimes, however, you need to make changes to a specific release of the code. To do this, you make a branch from the given tag you need to change. All changes are done in the branch and once completed, a new tag can be made from that branch. The new tag must have a new, unique name, but that is what we want since it is not exactly same as the previously tagged release. The rule is revision control is that you can not change history. In fact, it is the history that is the main value of revision control. Subversion has no support for changing history short reinitializing the whole server. The rule of no changes to what a tag or release defines is equally as important.

Merging with conflicts

The Subversion book does a very good job explaining the process of conflict resolution during a merge. Since much of the actual process depends on the code editing process, the book's description is as simple as it gets. See chapter 3, section 5, "Resolve Conflicts" for details on how to resolve conflicts during the merge.

Advanced Management

Branches as change sets

Since branches are so easy to deal with, one form of development management has a new branch made for each bug fix. What this does is give a complete history of the work for a given bug fix and an identifiable change set that implements the bug fix. Thus, if a bug fix needs to be applied to another version of the product, the process is basically a merge of that specific bug fix branch into the product. This is useful when needing to support older releases of the software. Now, this does not magicly make things work, but it does provide a level of tracking and support from the tools to implement.

Branches as bookmarks

Sometimes when working on some code, you try to implement something and then choose not to go that route at this time. If you wish to keep that work but not in the main tree, you can make a branch of your as of yet un-committed changes. When making the branch, do not first commit your new changes. Just use the svn copy command with the local directory as the source and the target your new branch for this bookmark. Subversion will make the branch and then check in the local changes onto that branch all in one operation. You can then revert your local copy to the mainline code again, knowing that you can always get your changes back by merging from your bookmark branch. (This is bettern than committing the code on the mainline and then undoing the changes since it is harder to merge those types of conditions than from a self-contained branch)

This document is still a work in progress.