Insurrection Web Tools

One of my pet projects has been to build a better web based interface to a revision control system known as Subversion. The Insurrection project is the result of that effort. While it is still under development, it already is a very compelling web interface to Subversion and is in active use in a number of places.

The project page is at http://insurrection.tigris.org/ but the source code can be accessed via the Insurrection tools itself at http://svn.sinz.com/. If you use Subversion, you can get the latest code via: svn co http://svn.sinz.com/svn/Insurrection/trunk

Features

Easily browse the repository
Access any version of any file
View revision histories
View differences at various revisions
Annotate files with respect to author and revision
Get RSS and Atom feeds of changes in a repository or part of a repository or a specific file
All repository features are via repository consistent URLs
Administrate user access to repositories

Design Goals

Must work without the need to install anything on the host OS. This means that all of the functionality of this system must work in a user process with a locally built Apache and Subversion code base. We do assume some basic features of a "unix" like operating environment, such as having a basic Perl installation.
Make use of modern browser technology to reduce server side overhead and bandwidth usage. This means using client-side XSLT transforms to embellish XML output into a presentable form. This means heavy use of XSLT, CSS, and JavaScript where needed to product a better result. No aiming at the lowest possible version.
Try not to use any browser or platform specific extensions if at all possible. This means that none of the JavaScript will do browser detection. There is one case were we have to check for a Microsoft-ism due to the lack of a common standard but the check is written such that if IE does start supporting the standard it will automatically used that form. The minimum browser support is for Firefox 1.0, Mozilla 1.7.x, and IE 6 with IE 5+, Opera, and Safari to be supported where reasonable. Safari is partially working due to bugs in Safari. Opera is not working due to lack of an XSLT implementation. (See these browser functionality tests that I have put together for details on client side requirements.)
To provide a way to administrate and support multiple repositories on the same server. Currently the code does not support a single repository only format since it assumes a multi-repository structure. Support for single repository configuration will be looked at if it does not adversely impact the support for multiple repositories.
To make sure that we don't use any more bad puns as we have used our allowance in the naming of this project.
...more to come...

Project Status

The current code is in active use on a number non-public Subversion servers. This does not mean that the project is complete but it is relatively stable. Many things have not been cleaned up or as fully supported as I would like. There is also a lack of clean documentation as to how to install the code. This is because, so far, it has always been installed by the same person who wrote it.

I have put up a production server at http://svn.sinz.com/ that has the Insurrection code on it so you can see it in action. I have also made some "Browser Shots" that show a small bit of what using Insurrection is like. (Click to view screen shots)

While it is best to actually see Insurrection in action at http://svn.sinz.com/, I have put together a few quick snapshots of the system working in my browser:

The main page is template driven, with the repositories the logged in user has access to being displayed. Annonymous users will only see repositories that have annonymous access enabled. As you can see, each repository gets its own RSS and Atom feed links in addition to a customizable description. Login can be handled via HTTP/SSL or just normal HTTP. The access user name and password is exactly those of Subversion's WebDAV system such that there is no confusion or possibility of them getting out of sync.

RSS and Atom feeds are not restricted to whole repositories. Subtrees and individual files can also be selected for RSS/Atom feeds. The directory feeds can easily be selected during the browsing of the repository.

When browsing the repository, you get a view of the current state of the repository. As you can see in the second screen shot, the repository browser is internally dynamic in that you can expand folders/directories in place. This is done using async requests from the browser and the bandwidth needed is only the minimal XML structure of the information. All formatting is done either via XSLT and/or JavaScript. (In some browsers JavaScript is used in place of XSLT, in others both XSLT JavaScript is used)

The Get revision history icon icon is the button to select the revision history entries of the repository that pertain to that given item. If the item is a directory, the history will include any changes made within that directory tree. If the item is a file, the history will include any revision which updated that file.

Here we got the revision history of "/trunk" from within the Insurrection repository that was being browsed in the screen shots above. The log shows the commit message for each revision and, when clicked on, the details as to what files where added, deleted, modified, etc. The second picture shows what happens after having clicked on revision 264.

For each file listed in the revision details, a popup menu is available that lets you select various operations on that file. In this screen shot we are selecting the "Diff from previous" option which shows us the differences from the previous version of the file. The "Patch from previous" would produce a patch file that would be usable to patch the previous version into the new version. The second screen shot shows the HTML diff output.

One can also select a specific revision that can then be used to diff against elsewhere in the page. Note that this selection is currently on a specific filename. I am going to be working on removing that requirement.

The second screen shot here shows the result of the "annotate" or "blame" feature. It shows the author and revision that the given lines were last modified/added. Thus, if you want to know who last touched a bit of code you can easily do so. Clicking on a revision in the annotate display will get you the revision entry for that revision of the file.

This is the administrative page for a repository. Users who are defined as admins for a given repository will be able to access this page. From here the administrator can change the repository description, change user access rights, add new users, download a dump/backup of the repository, and configure certain hook features.

There is also a bandwidth usage report link in the admin page that depends on external usage tools that I have not released yet. Here are two screen shots of the usage pages of the public Insurrection repository. This depends on a number of tools and scripts to produce this result and is, of course, optional and not required for the full functionality of the Insurrection tools. (I just like having such information available to me.)

I have put together some simple tests of the client-side/browser capabilities that the Insurrection system needs. You can try these tests at http://svn.sinz.com/browser-tests/ and the source for these tests are in the source repository.

The goal of this set of tools is to become a reasonably complete Subversion repository browsing system plus a simple administration system for repository access controls.

Why reinvent the wheel?

While, in some ways I believe that I have built something that already exists, I have done so mainly due to direct needs or issues with what already exists. See some of the design goals above...

The main reason was the need for running within a standard Linux server install without the need for any root access. This makes this a clean install onto already supported NOC servers within a company without special needs or configuration. The current setups I have installed allows for the migration of the whole system to another machine by just moving the user account and directories. No special backup procedures, system installs, or extra patching for securty or administration. (For example, my setup runs on a plain-vanilla RedHat AS2.1 install)

One of the key components that made this possible is the Subversion Filesystem database backend support. In my testing of the FSFS repositories, I have found that they are actually more reliable than the BDB ones. (Less operator intervention in the case of a lights-out environment.)



	Insurrection Web Tools One of my pet projects has been to build a better web based interface to a revision control system known as Subversion. The Insurrection project is the result of that effort. While it is still under development, it already is a very compelling web interface to Subversion and is in active use in a number of places. The project page is at http://insurrection.tigris.org/ but the source code can be accessed via the Insurrection tools itself at http://svn.sinz.com/. If you use Subversion, you can get the latest code via: `svn co http://svn.sinz.com/svn/Insurrection/trunk` Features Easily browse the repository Access any version of any file View revision histories View differences at various revisions Annotate files with respect to author and revision Get RSS and Atom feeds of changes in a repository or part of a repository or a specific file All repository features are via repository consistent URLs Administrate user access to repositories Design Goals Must work without the need to install anything on the host OS. This means that all of the functionality of this system must work in a user process with a locally built Apache and Subversion code base. We do assume some basic features of a "unix" like operating environment, such as having a basic Perl installation. Make use of modern browser technology to reduce server side overhead and bandwidth usage. This means using client-side XSLT transforms to embellish XML output into a presentable form. This means heavy use of XSLT, CSS, and JavaScript where needed to product a better result. No aiming at the lowest possible version. Try not to use any browser or platform specific extensions if at all possible. This means that none of the JavaScript will do browser detection. There is one case were we have to check for a Microsoft-ism due to the lack of a common standard but the check is written such that if IE does start supporting the standard it will automatically used that form. The minimum browser support is for Firefox 1.0, Mozilla 1.7.x, and IE 6 with IE 5+, Opera, and Safari to be supported where reasonable. Safari is partially working due to bugs in Safari. Opera is not working due to lack of an XSLT implementation. (See these browser functionality tests that I have put together for details on client side requirements.) To provide a way to administrate and support multiple repositories on the same server. Currently the code does not support a single repository only format since it assumes a multi-repository structure. Support for single repository configuration will be looked at if it does not adversely impact the support for multiple repositories. To make sure that we don't use any more bad puns as we have used our allowance in the naming of this project. ...more to come... Project Status The current code is in active use on a number non-public Subversion servers. This does not mean that the project is complete but it is relatively stable. Many things have not been cleaned up or as fully supported as I would like. There is also a lack of clean documentation as to how to install the code. This is because, so far, it has always been installed by the same person who wrote it. I have put up a production server at `http://svn.sinz.com/` that has the Insurrection code on it so you can see it in action. I have also made some "Browser Shots" that show a small bit of what using Insurrection is like. (Click to view screen shots) While it is best to actually see Insurrection in action at http://svn.sinz.com/, I have put together a few quick snapshots of the system working in my browser: The main page is template driven, with the repositories the logged in user has access to being displayed. Annonymous users will only see repositories that have annonymous access enabled. As you can see, each repository gets its own RSS and Atom feed links in addition to a customizable description. Login can be handled via HTTP/SSL or just normal HTTP. The access user name and password is exactly those of Subversion's WebDAV system such that there is no confusion or possibility of them getting out of sync. RSS and Atom feeds are not restricted to whole repositories. Subtrees and individual files can also be selected for RSS/Atom feeds. The directory feeds can easily be selected during the browsing of the repository. When browsing the repository, you get a view of the current state of the repository. As you can see in the second screen shot, the repository browser is internally dynamic in that you can expand folders/directories in place. This is done using async requests from the browser and the bandwidth needed is only the minimal XML structure of the information. All formatting is done either via XSLT and/or JavaScript. (In some browsers JavaScript is used in place of XSLT, in others both XSLT JavaScript is used) The icon is the button to select the revision history entries of the repository that pertain to that given item. If the item is a directory, the history will include any changes made within that directory tree. If the item is a file, the history will include any revision which updated that file. Here we got the revision history of "/trunk" from within the Insurrection repository that was being browsed in the screen shots above. The log shows the commit message for each revision and, when clicked on, the details as to what files where added, deleted, modified, etc. The second picture shows what happens after having clicked on revision 264. For each file listed in the revision details, a popup menu is available that lets you select various operations on that file. In this screen shot we are selecting the "Diff from previous" option which shows us the differences from the previous version of the file. The "Patch from previous" would produce a patch file that would be usable to patch the previous version into the new version. The second screen shot shows the HTML diff output. One can also select a specific revision that can then be used to diff against elsewhere in the page. Note that this selection is currently on a specific filename. I am going to be working on removing that requirement. The second screen shot here shows the result of the "annotate" or "blame" feature. It shows the author and revision that the given lines were last modified/added. Thus, if you want to know who last touched a bit of code you can easily do so. Clicking on a revision in the annotate display will get you the revision entry for that revision of the file. This is the administrative page for a repository. Users who are defined as admins for a given repository will be able to access this page. From here the administrator can change the repository description, change user access rights, add new users, download a dump/backup of the repository, and configure certain hook features. There is also a bandwidth usage report link in the admin page that depends on external usage tools that I have not released yet. Here are two screen shots of the usage pages of the public Insurrection repository. This depends on a number of tools and scripts to produce this result and is, of course, optional and not required for the full functionality of the Insurrection tools. (I just like having such information available to me.) I have put together some simple tests of the client-side/browser capabilities that the Insurrection system needs. You can try these tests at http://svn.sinz.com/browser-tests/ and the source for these tests are in the source repository. The goal of this set of tools is to become a reasonably complete Subversion repository browsing system plus a simple administration system for repository access controls. Why reinvent the wheel? While, in some ways I believe that I have built something that already exists, I have done so mainly due to direct needs or issues with what already exists. See some of the design goals above... The main reason was the need for running within a standard Linux server install without the need for any root access. This makes this a clean install onto already supported NOC servers within a company without special needs or configuration. The current setups I have installed allows for the migration of the whole system to another machine by just moving the user account and directories. No special backup procedures, system installs, or extra patching for securty or administration. (For example, my setup runs on a plain-vanilla RedHat AS2.1 install) One of the key components that made this possible is the Subversion Filesystem database backend support. In my testing of the FSFS repositories, I have found that they are actually more reliable than the BDB ones. (Less operator intervention in the case of a lights-out environment.) Anonymous