GIT Migration

This page is a place holder for gathering information on the potential migration from SVN to GIT for the ScummVM source code repositories.

Please use this page to track any blockers that you may foresee (Windows users, I am looking at you ;-)) so the problems can be addressed and the work moved forward.

Windows users may want to take a look at http://code.google.com/p/gitextensions/ and http://code.google.com/p/tortoisegit/ --Fingolfin 12:47, 2 November 2010 (UTC)

Things to tweak in the repository conversion

Author names/emails: see this thread: http://sourceforge.net/mailarchive/message.php?msg_name=20100807142903.71414e9c%40annie .

Do we maybe want to use unknown@scummvm.org as email address for some of the unknown authors rather than username@users.sf.net? Wjp 13:27, 2 November 2010 (UTC)

Graft in FreeSCI history from vendor/, rewriting the commit messages

Maybe: make merges of branches back into trunk actual merges Wjp 22:35, 3 November 2010 (UTC)

Maybe: broken endlines in SCI r50329, r50331 (breaking annotations) Wjp 13:09, 2 November 2010 (UTC)

Broken endlines in the merge r54051, fixed in r54052

If possible: restore author/date of SWORD25 import commits? Wjp 13:09, 2 November 2010 (UTC)

Are there other external histories we can and want to graft in? (Sarien?) Wjp 13:09, 2 November 2010 (UTC)

From Fingolfin's mail at http://sourceforge.net/mailarchive/message.php?msg_name=AA4D038A-B948-43F0-87D7-22A2818BA3ED%40quendi.de:

There was also some cleanup I performed after the CVS -> SVN switch, which in retrospect is not nice to have in the history. E.g. I cleaned up various branches, and many SVN properties. So commits like <http://scummvm.svn.sourceforge.net/viewvc/scummvm?view=revision&revision=20471> would be nice to suppress. Those SVN properties wouldn't be visible in git anyway, I guess.

Also, what about fake branching commits created by cvs2svn, like <http://scummvm.svn.sourceforge.net/viewvc/scummvm?view=revision&revision=3488>, can we get rid of those, too?

Also, I think when we made the CVS conversion back then, something went slightly wrong with the very old history. Consider e.g. <http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/?pathrev=3486>, where the code that should be in trunk is hidden away in a subdirectory "scummvm-old" of trunk. Not terrible, but certainly annoying. Esp. since later on, at some point the two are mixed: Trunk is populated but "scummvm-old" is still there.This happened in revision 4785, see <http://scummvm.svn.sourceforge.net/viewvc/scummvm?view=revision&revision=4785>, and while it lists me as the committer with commit message "Initial revision", I am not quite sure why that is... probably because we switch to a new CVS repository back then, with a revised file structure... And things were never properly stitched together. The annoying part is that "scummvm-old" is still visible much, much later, see e.g. <http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/?pathrev=18000>. It was only removed in revision 20419, see <http://scummvm.svn.sourceforge.net/viewvc/scummvm?view=revision&revision=20419>, manually.

Something built on the following filters should take care of scummvm-old: Wjp 22:20, 3 November 2010 (UTC)

 git checkout -b remove_old_branch  `git svn find-rev r4784`
 git filter-branch --index-filter 'git ls-files -s | 
   sed "s-\tscummvm\-old\/\"*-\t-" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
   git update-index --index-info && test -f $GIT_INDEX_FILE.new &&
   mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE || true' HEAD
 git checkout -b remove_old_branch_2 `git svn find-rev r20419`
 git filter-branch -f --prune-empty --parent-filter \
   'sed s/`git svn find-rev r4784`/`git rev-parse remove_old_branch`/' \
   --index-filter 'git ls-files -s | grep -v scummvm-old |
   GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
   test -f $GIT_INDEX_FILE.new && mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE || true' HEAD

Where to host the GIT repository?

• SF.Net Internal GIT hosting? - Not ideal. Features lacking.

• From Fingolfin: One thing to settle is where to host the master repository: I for once would like using github, due to the many nice features it offers, like very easy code review. (FYI, I registered a scummvm project account there some time ago, in case we want to use it, and to prevent others from (ab)using it). ScummVM on GITHub.

Some of our devs already use github for their own ScummVM trees, and since github makes it super-easy for anybody to branch from another user's tree, and then push changes back (resp. file a "pull request)", this seems ideal to me.

Alternatives

• The mercurial client, which is written in Python, can perhaps provide a better user experience for Windows, together with tortoise-hg. Relevant guides can be found here:
  • http://kb.wowace.com/repositories/tortoise-hg-as-a-git-client/
  • http://kb.wowace.com/repositories/tortoise-hg-as-an-svn-client/

How does that compare to TortoiseGit and Git Extensions ? The latter seems to be pretty good (though I have not yet tried it myself, will see if I can do this tonight or tomorrow). It supposedly even features Visual Studio integration. --Fingolfin 14:38, 2 November 2010 (UTC)

TortoiseHG on Windows runs very well against a HG repo. I've also pulled a Git repo with hg-git and it worked but I haven't done any major development in that configuration yet (eg, branch/merge, etc). --ScottT 20:58, 2 November 2010 (UTC)

I've also been using TortoiseHG for lastexpress/create_project/scummvm development (no branching/merging used either yet). I'm also using the subversion bridge to commit back to the current repo. The svn<=>mercurial<=>git conversion is not perfect (putting extra HG info in the git comments), but that will be a moot point once the transition to git is complete. I'm also using a modified version of Posh-hg for better command-line integration.

I'm seeing performance problems (apparently caused by dulwich) when trying to push a converted scummvm repo to github though. It works great for small projects but will abort (out of memory) on huge repositories (not to mention the slowness in converting revisions and packing for sending). Is there a public scummvm tree on github so I can try importing it and pushing back to a new repo? (note that you can convert using hg-git and then push back the results directly with git, but that's painful).

For Visual Studio integration with Mercurial, there is Visual HG (and a couple other projects), which work relatively well (although lacking features compared to AnkhSVN) --Julien 14:35, 3 November 2010 (UTC)

Folks, you all write in reply to what I wrote, but without addressing at all the content of what I wrote. That's really a pity. Before debating which hg<->git bridge to use, could you maybe explain to us why TortoiseGit and Git Extensions (which, by the way, also includes Visual Studio integration) are to be dismissed out of hand? Thanks! :) --Fingolfin 21:53, 3 November 2010 (UTC)