Using Git with Subversion Mirroring for WordPress Plugin Development

wordpress-plugin-svn-to-git.md

Update: please note that I have since switched to using a set of bash scripts instead of poluting the Git repository with git svn.

Using Git and GitHub with Subversion (SVN) Mirroring for WordPress Plugin Development

Author: Kaspars Dambis
kaspars.net / @konstruktors

Getting Started

We'll assume that your plugin is already hosted on the official WordPress plugin Subversion repository, and we'll use the Easy Digital Downloads plugin as an example.

1. First, we need to get the revision number of the first commit to the WordPress Subversion repository, because otherwise Git will try to go through all the 100000+ commits in order to find it.

    $ svn log http://plugins.svn.wordpress.org/easy-digital-downloads/
   

   It is the last commit revision you seen on the screen:

    ------------------------------------------------------------------------
    r529177 | plugin-master | 2012-04-09 19:36:16 +0200 (Mon, 09 Apr 2012) | 1 line
   
    adding easy-digital-downloads by mordauk
    ------------------------------------------------------------------------
   

   In this case it is r529177. You can also use the official WordPress Trac browser to determine the first revision number.

2. Now, create a local Git repository and import the first commit from the SVN repository:

    $ git svn clone --no-minimize-url -s -r529177 http://plugins.svn.wordpress.org/easy-digital-downloads/
   

   which returns the following:

    Initialized empty Git repository in /Users/kaspars/svn2git/easy-digital-downloads/.git/
    r529177 = e18c66d09d77e4d8a923f2f300f73606791071e1 (refs/remotes/trunk)
    Checked out HEAD:
      http://plugins.svn.wordpress.org/easy-digital-downloads/trunk r529177
   

   Notice the --no-minimize-url flag which is required to stop git svn clone from moving into the base folder of the remote SVN repository and going through all plugins (see git-svn docs). We also use the -s flag which is a shorthand way of setting trunk, tags, branches as the relative paths, which is the Subversion default (from git-svn docs).

3. Finally, move inside the newly created Git repository:

    $ cd easy-digital-downloads
   

   and fetch all the other commits from the SVN repo:

    $ git svn fetch
   

   This step will take hours (!), so you can use the GIT_TRACE environment variable before the the command to see a verbose output:

    $ GIT_TRACE=1 git svn fetch
   

   To speed up git svn fetch especially with large SVN repositories you can try setting the --log-window-size parameter to request more commits per SVN request (read Experiences of Using Git-Svn on a Large Project). Increasing it to 10000 should decrease the fetch time significantly:

    $ git svn fetch --log-window-size 10000
   

   Once this process is finished you have a complete Git commit tree of the project. The final step is to move the SVN HEAD (files from SVN /trunk) into our Git master branch (I assume), which is done using:

    $ git svn rebase
   

   Now you should see all the latest plugin files in your local repository.

   

Inspecting the Newly Cloned Repository

Let's look at all the branches that were created:

$ git branch -a

which returns a list of all the available branches:

* master
  remotes/tags/1.0.1.1
  remotes/tags/1.0.1.2
  remotes/tags/1.0.1.3
  remotes/tags/1.0.1.4
  ...
  remotes/tags/1.1.8
  remotes/tags/1.2
  remotes/tags/1.2.1
  remotes/tags/1.2.1.1
  remotes/trunk	

Inside .git/config of our newly created Git repository:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
        ignorecase = true
        precomposeunicode = false
[svn-remote "svn"]
        url = http://plugins.svn.wordpress.org/easy-digital-downloads
        fetch = trunk:refs/remotes/trunk
        branches = branches/*:refs/remotes/*
        tags = tags/*:refs/remotes/tags/*

we can see that Git has automatically selected trunk as the destination for git svn dcommit which pushes all your Git changes back to the WordPress SVN repository.

Adding a Remote Git Repository

Let's create a new GitHub repository which we'll use as the main repository for our plugin developemnt from now on:

Once you have created the repository, simply add it as a remote origin of our local Git repository:

$ git remote add origin git@github.com:kasparsd/wordpress-svn2git-sample.git

and then push our local repository to GitHub:

$ git push origin --all

Now your GitHub repository should contain all your plugin files as well as all the revision history carried over from the Subversion repository.

Working with the Plugin

Now let's assume that we want to create a new release of the plugin in order to invite the plugin users to report bugs and get involved on GitHub. For that we need to edit the copy of readme.txt and bump the version number of the plugin.

Once we have made the changes, we can check the status of our local Git repository:

$ git status

# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   easy-digital-downloads.php
#	modified:   readme.txt
#
no changes added to commit (use "git add" and/or "git commit -a")

Let's commit the changes to our local repository:

$ git commit -am "Adding message about moving to GitHub, version bump"

Pushing Changes to GitHub

To publish our local commits to GitHub, we use

$ git push

	[master addb6cf] Adding message about moving to GitHub, version bump
 2 files changed, 12 insertions(+), 1112 deletions(-)
 rewrite readme.txt (99%)

Pushing Changes to WordPress Subversion Repository

Once you have switched to using Git as your version control system, you should always use git svn dcommit to publish changes to the SVN repository. Otherwise, you must run

$ git svn rebase

every time before pushing changes to SVN in order to merge changes from the SVN repository into your local Git repo.

If you are not using trunk as a release branch for your plugin, you may safely push all your Git changes to the WordPress repository without worrying about users getting update notices:

$ git svn dcommit

Now, because I don't have write access this particular repository, here is a response that I get when pushing one of my own plugins:

Committing to http://plugins.svn.wordpress.org/widget-context/trunk ...
	M	widget-context.php
Committed r601084
	M	widget-context.php
r601084 = a1c2a2f8f4f98ec5537aef48545e350ff6270f2b (refs/remotes/trunk)
No changes between b6098f5bb1751cc05309093cad1d5d982a964158 and refs/remotes/trunk
Resetting to the latest refs/remotes/trunk	

Tagging Releases

Tagging a release in Git is very simple:

$ git tag v1.0.2

To create an SVN tag, simply:

$ git svn tag 1.0.2

This will create /tags/1.0.2 in the remote SVN repository and copy all the files from the remote /trunk into that tag, so be sure to push all the latest code to /trunk before creating an SVN tag.

Updating and Trakcing Plugin Assets

Assets folder in the WordPress SVN repository can be used for storing plugin banners and screenshots. Make sure that the /assets folder actually exists in your SVN repository before trying to fetch it. Open this in your web browser:

http://plugins.svn.wordpress.org/easy-digital-downloads/

where easy-digital-downloads is the slug of your plugin, and you should see the root folder of your plugin SVN repository:

Revision 851441: /easy-digital-downloads

..
1.0.4/
assets/
branches/
tags/
trunk/

Notice that the /assets folder is present.

Let's track that folder as a seperate Git branch called assets. Every time you need to update the cover image or screenshots you, simply checkout that branch, do the changes and publish it to SVN using the same old git svn dcommit.

Add a new SVN remote called assets:

$ git config --add svn-remote.assets.url http://plugins.svn.wordpress.org/easy-digital-downloads/assets

and set it to use the assets branch on our remote SVN repository:

$ git config --add svn-remote.assets.fetch :refs/remotes/assets

This will have appended the following to our .git/config:

[svn-remote "assets"]
	url = http://plugins.svn.wordpress.org/easy-digital-downloads/assets
	fetch = :refs/remotes/assets

which means that we can now fetch the assets remote at its current HEAD:

$ git svn fetch -r HEAD assets

and switch to it using:

$ git checkout remotes/assets

Note: checking out 'remotes/assets'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 7e5b727... Updated banner image

Let's create a local branch called assets to track this remote:

$ git checkout -b assets

Now you can add and modify plugin assets, commit the changes and then publish the changes to the SVN, using:

$ git svn dcommit

Suggested Reading

• http://teleogistic.net/2011/05/revisiting-git-github-and-the-wordpress-org-plugin-repository/
• http://eamann.com/tech/how-to-publish-a-wordpress-plugin-git/
• http://ben.lobaugh.net/blog/90633/creating-a-synchronized-github-fork-of-a-wordpress-org-subversion-plugin-repository
• http://en.hetarena.com/archives/232

The SVN tags are converted to Git branches, you can make real Git tags with the following command (based on
this git ready tip):

git for-each-ref refs/remotes/tags | cut -d / -f 4- |
while read ref
do
git tag -a "$ref" -m"Tag $ref" "refs/remotes/tags/$ref"
done

@janfabry Is it possible to tag new versions in git then push them to wordpress SVN?

@janfabry any advice on managing tags/releases? Ideally I'd like to do
git tag 1.0.0 and have that be reflected on the svn side

git tag 1.0.0 && git svn tag 1.0.0 is the best i can come up with

I really like this for updating trunk but I don't see my assets directory (i.e. where I keep the screenshots for wordpress.org):

[~/svn]# ls
./  ../  .svn/  assets/  branches/  tags/  trunk/

[~/git]$git branch -a
* master
  test
  remotes/tags/1.0
  remotes/tags/1.1
  remotes/tags/1.2
  remotes/tags/1.3
  remotes/tags/1.4
  remotes/tags/1.5
  remotes/tags/1.6
  remotes/test/master
  remotes/test/test
  remotes/trunk

I can $ git svn dcommit from my git repo then # svn up from the svn repo and get all of the code changes in trunk. Then, I'm just managing tags in svn. Is there a better way to do this?

Oh, man. I wish there was some kind of comment notification for Gists. I never knew there were all these great comments until now when I find this article via a google search.

Thanks for the great suggestions @janfabry! I have included them into the article now.

Don't forget to push tags to github:

git push --tags origin

using git push --all origin did not push the tags when I did it

I had the same problem with git svn rebase hanging until I used the --all switch.

I am able to push new revisions to the trunk. I am however not able to create tags. Typing: git svn tag "1.0.0" returns the following error:

Authorization failed: POST of '/!svn/me': authorization failed: Could not authenticate to server: rejected Basic challenge (http://plugins.svn.wordpress.org) at /usr/local/Cellar/git/2.4.1/libexec/git-core/git-svn line 1196.

Did anyone of you experience this problem?

I had to make a few changes to the script @janfabry included for converting tags correctly. I'm running Git 2.4.5, so maybe things have changed in the last two years :)

git for-each-ref refs/remotes/origin/tags | cut -d / -f 5- |
while read ref
do
git tag -a "$ref" -m"Tag $ref" "refs/remotes/origin/tags/$ref"
done

How do you use GitHub if every time 'git svn dcommit' is called it overwrites SHA-1 hashes for all new commits?

There doesn't seem to be anything in this set of directions that addresses how I get access to my SVN repository. Just doing this command:
git svn clone --no-minimize-url -s -r529177 http://plugins.svn.wordpress.org/easy-digital-downloads/

Doesn't seem to quite cover it? Because then, when I go to commit changes to SVN, I get "No changes between 7cff1bf856e9b32a8c44d64675aae99b395ad2e6 and refs/remotes/origin/trunk"

What am I missing?

I'm seeing the same problem described above by @tolvstein and am unable to create tags. The exact error I'm seeing is:

Authorization failed: POST of '/!svn/me': authorization failed: Could not authenticate to server: rejected Basic challenge (https://plugins.svn.wordpress.org) at /Applications/Xcode.app/Contents/Developer/usr/libexec/git-core/git-svn line 1217.

The same issue is described here, here, and here, but none of these other threads have any resolution. :(

Any updates on this? I have the same problem and it's very annoying... :-(

The correct is:

git svn clone -r529177:HEAD -s --no-minimize-url http://plugins.svn.wordpress.org/easy-digital-downloads/

The solution described by @meitar that use svn cp it worked for me:

svn cp https://plugins.svn.wordpress.org/wp-agenda/trunk https://plugins.svn.wordpress.org/wp-agenda/tags/1.7 -m "creating tag 1.7"

It's not a real solution, but there's no way to authenticate using git svn tag.

use git svn rebase --all

You can use travis to deploy pluings and themes if you don't need to keep development history in SVN (keep it only in Git).
Here is a guide for it https://kuzmi.ch/articles/wordpress-plugin-and-theme-development-using-git-only/

Is it possible to additionally "upload/sync" my git tags to SVN? I have created several tags only locally. As I understand git svn tag 1.1 creates tag only for current state (latest commit in SVN), it does not create tag for that local commit with the same local tag.

Btw when I use git svn tag 1.1 I get:

Authentication failed: No more credentials or we tried too many times.
Authentication failed at ...Git/mingw64/libexec/git-core/git-svn line 1196.

but git svn dcommit works well