Why Git?

Please note that I have posted a follow-up article, which presents a working solution using Git sub-projects to overcome the git-cat-file bad file error when git-svn dcommitting with submodules.

I do alot of my work (or at least the fun Ruby work) on the train on my daily commute. During this time, Subversion has served me well for the last four years. One if its biggest pluses for me at the time was being able to have a snapshot of your latest commit, which is something no other SCM did in 2003.

No being able to access the entire SVN history was a pain when I needed to maintain separate branches of the same code or do merging between branches. This is something I had to stage on my laptop in separate directories and action once I get home and reconnect to my SVN repository.

Git has changed all of that. Using Git, I am able to have a copy of my entire subversion repository, courtesy of git-svn, on the train. This allows me to:

1. View the entire svn history whilst being disconnected from my SVN repository.
2. Branch and Merge as much as I want.
3. Keep track of my SVN branches via GIT as SVN does not track branch histories.

Once I get home I simply sync my laptop's GIT repository with my SVN repository via git-svn dcommit.

So What's the Problem?

There is one snag in this: my requirements dictate that my source be accessible to the outside world via SVN due to Capistrano deployments and clients needing to access my code (or a branch of my code). The problem here is that there is no direct equivalent of svn:external in Git.

My Rails sites are actually stored in two SVN repositories: the Rails app sits in its own repository whilst plugins reside in an SVN plugins repository. The separate plugins repository allows me to re-use my plugins and ensures that any code changes I introduce into a plugin filters across to all my Rails apps.

This is evident when first importing your SVN repository to Git via git-svn clone. git-svn will ignore all svn:externals. Result? Empty plugins directory.

Solution 1: Track plugins code in the same Rails App GIT Repository.

This is a no go for me. I will potentially end up with multiple versions of the same plugins across various GIT repositories.

Solution 2: Create a Dedicated GIT Repository for all Plugins.

At first glance this looks like a winner. I can have all my plugins in one place (a good thing ^tm) and just reference them in all my Rails app Git repositories.

There is one snag to this solution. Git does not support partial checkouts. You either checkout the whole repository... or nothing. In this scenario, all my Rails apps will have all my plugins.. which is not what I want...

Solution 3: Create a Dedicated GIT Repository for all Plugins and Use Symbolic Links.

This is sounding better. You have all your Rails plugins in one Git repository. These are clones locally on your development machine (say in /var/rails/plugins.git).

Simply create symbolic links from your rails app to the plugins your require:

ln -s link target
ln -s /var/rails/app1.git/vendor/plugins/acts_as_commented /var/rails/plugins.git/acts_as_commented

The above will create a symbolic link in your app1/vendor/plugins directory that actually points to the GIT managed plugins.git repository in /var/rails/plugins.git

Unfortunately, the above does not work for all plugins. For some reason, rspec_for_rails does not like being linked symbolically. Otherwise this solution would have worked for me.

Do note that you can link the plugins directory under vendors to the root of plugins.git but then we are back to solution 2; you are linking in all your plugins into a Rails app.

Solution 4: Clone the SVN Plugins repository Locally with Git.

Let me explain my plugin repository structure before I proceed:

SVN/trunc/active_merchant
SVN/trunc/acts_as_commented 
.
.
SVN/vendor/active_merchant
SVN/vendor/acts_as_commented

All modified plugins are stored in my trunc whilst tracking vendor code in the vendor branch.

Unfortunately you cannot:

git-svn clone svn://repos_url.com/svn/trunc 

as that will clone the whole trunc. What is required is the ability to clone each plugin into a separate GIT repository (yes a pain but bear with me).

git-svn clone svn://repos_url.com/svn/trunc/active_merchant .
git-svn clone svn://repos_url.com/svn/trunc/acts_as_commented .
.
.

At first glance this seems like alot of work.... but not if you write a little ruby script that does the hard work for you :).

#!/usr/bin/ruby
 
require 'fileutils'
 
GIT_SVN = 'git-svn'
 
plugins = `svn list svn://192.168.0.10/rails/plugins/trunc`
 
#create subs
plugins.each{|plugin| 
  #check if exists
  plugin = plugin[0,plugin.length-1]
  FileUtils.mkdir(plugin[0,plugin.length-1]) if !File.directory?(plugin)
}
 
#create git repos for each sub
plugins.each{|plugin|
  plugin = plugin[0,plugin.length-1]
  cmd = "#{GIT_SVN} clone svn://192.168.0.10/rails/plugins/trunc/#{plugin}"
  system(cmd)
}

Create a /var/ails/plugins.git (or /home/me/rails/plugins.git) directory, copy the above script into it and execute.

Git Submodules.

This is another possible solution.

Git allows you to break up a large project into independent sup projects. In other words... my plugins actually are independent and do need to be tracked separately.

Once sub modules are defined, the top level Git repository acts as a super repository that tracks your main rails app but also tracks your rails plugins. This, I believe, is more flexible than SVN's svn:externals.

To add a plugin to your rails app (from the root of your project dir_):

git submodule add /var/rails/plugins.git/active_merchant vendor/plugins/active_merchant

Repeat for each plugin your app needs.

Alternatively, write a bash script that will do this for you:

#!/bin/bash 
for i in active_merchant active_scaffold acts_as_commentable acts_as_favouriteable 
  acts_as_markable acts_as_rateable acts_as_taggable akismet backgroundrb 
  browser_filters exception_logger fck_editor flash_fixes nested_layouts query_cache 
  query_trace tabnav
do
  /usr/bin/git submodule add /var/rails/plugins.git/$i vendor/plugins/$i
done
 

Once added:

git submodule init
git submodule update

The last command should clone your Git plugins repository to your app's plugins directory.

This solution would have been perfect except that it will not work. As it stands, git-svn is not able to git-svn dcommit any repository that contains sub-modules.

Any such attempts results in:

Committing to file:///home/sysadm/devbox/testrepo/superproj …
     A    .gitmodules
     A    plugintest/myplugin
 fatal: git-cat-file 99e6127cfff6447645f01a50dc836081456d0753: bad file
 32768 at /usr/bin/git-svn line 450
 

I have posted a query to the Git mailing list but have received no replies which leads me to believe that this is not currently possible and that git-svn's maintainer is not inclined to provide a solution at present.

Graham Aston has been kind enough to post his comments on this article. He has had some success with using a symllink method, which he's wrapped up in a handy script.

I discuss this solution in Solution 3 and maintain that certain plugins are, for one reason or another, not symlink compatible. I was not able to utilise either rspec or rspec_on_rails, which failed to properly initialise when symlinked. YMMV on this one.

Solution 5: Git Sub-projects.

My understanding of Git subprojects is that they have preceded Git sub-modules.

A Git sub-project is simply a git repository in a subdirectory under a Git managed directory. Using this, I should be able to clone each plugin under vendor/plugin and still utilise git-svn dcommit.

In this instance, we are able to ignore vendor/plugin via .gitignore, which we were unable to when using sub-modules.

Unfortunately this, although does provide my first working solution, has a few disadvantages:

1. Git sub-projects are disconnected from the Git super-project. On the face of it, this is not such an issue as SVN maintains this information.
2. Lack of tools such as git-submodule to check on status of sub-projects.
   

We are required to write our own porcelain to maintain and push sub-project changes.

Please note that I have posted a follow-up article, which presents a working solution using Git sub-projects to overcome the git-cat-file bad file error when git-svn dcommitting with submodules.

Solution 6: Git Super-project with SVN Managed Vendor/Plugins.

The last possible workaround is to maintain the main Rails app in Git while continuing to utilise SVN to maintain the vendor/plugins directory. In this instance, the vendor/plugin directory can simple be ignore in .gitignore.

Although this is a working solution it is less ideal than Solution 5 in that I remain disconnected from my plugins code whilst being off the network.

Conclusions

My proffered solution for the time being is solution 5, in which my plugins code (which I host on a central server and re-use on different Rails apps) are tracked using Git sub-projects.

Geoffrey Grosenbach has posted a comment referring to Giston. Giston is perfect if you are tracking your own changes against the author's code on a single repository. I, however, have a collection of about a dozen plugins which have been slightly modified and are used in quite a few Rails applications.

After much experimentation, I've come to the conclusion that the ideal solution would be to let Git completely manage both the Rails application and the shared plugins source. Taking SVN and the lacking git-svn dependency out of the equation would allow me to best manage my code.

Closing Thoughts

Git is still relatively new so does lack some of polish when compared to Subversion (i.e. no shell integration under Windows).

It is, however, the next step in the evolution of the SCM toolset. It encourages more experimentation in your code by making branch a trivial as opposed to a daunting task.