docutils migrations to github

docutils_to_gh.rst

The conversion from sourceforge was done using the github importer tool. (https://import.github.com/new)

A pristine copy of the old repository is available at https://github.com/docutils/docutils-old.

Restructuring repository

One of the issues with the git mirror at http://repo.or.cz/w/docutils.git and the vanilla import at https://github.com/docutils/docutils-old is:

1. docutils - docutils/* should preferably be on repository root
2. prest - better off broken into a separate repository
3. sandbox - obsolete in dvcs. On github the contributor has a workflow to fork and pull request
4. web

The size of the git repository is huge because it contains history for these 4 projects, a whopping 21.03MiB, which should be separated. Since rewriting history isn't ideal, the four projects will be split, with their own histories' preserved.

After prest, sandbox and web is split, docutils/* branch will have it's master overwritten by subtree split, preserving only the project's trunk.

Move web

$ git subtree split -P web -b web
$ git checkout web
$ git remote add web git@github.com:docutils/web.git
$ git push web web:master
$ git branch -D web

https://github.com/docutils/web now has the old web and all history preserved. Filter web from the docutils repository.

$ git filter-branch --tree-filter "rm -rf web" --prune-empty HEAD
$ git push --force

It's possible that web will be made obsolete by sphinx documentation in the future.

Move sandbox

$ git subtree split -P sandbox -b sandbox
$ git checkout web
$ git remote add sandbox git@github.com:docutils/sandbox.git
$ git push sandbox sandbox:master
$ git branch -D sandbox

https://github.com/docutils/sandbox now has the old sandbox and all history preserved.

Made a note that sandbox is obsolete in favor of forking https://github.com/docutils/docutils.

Filter web from the docutils repository.

$ git filter-branch --tree-filter "rm -rf sandbox" --prune-empty -f HEAD  # note the added -f
$ git push --force

Move prest

$ git subtree split -P prest -b prest
$ git checkout web
$ git remote add prest git@github.com:docutils/prest.git
$ git push prest prest:master
$ git branch -D prest

https://github.com/docutils/prest now holds the prest project and all history preserved.

The original import did have tags for prest-0.3.10 and prest-0.3.11. The commit log mentions version numbers, it is possible to go back and tag refs if desired.

Prest is a perl project. It may be nice to have a perl programmer modernize its' packaging / code / tests.

Filter out the project.

$ git filter-branch --tree-filter "rm -rf prest" --prune-empty -f HEAD  # note the added -f
$ git push --force

Move docutils

$ git subtree split -P docutils -b docutils
$ git checkout docutils
$ git push --force origin docutils:master  # loo carefully, overwriting the main repository
$ git checkout master
$ git reset --hard origin/master
$ git branch -D docutils

Welcome to your new home, docutils.

Other cleaning

Wipe stale branches.

$ git branch
address-rendering
docutils-0.4
index-bug
lossless-rst-writer
nesting
plugins
subdocs

I have made the decision to clean sweep up old branches, all of which are older than 5 years. Their history is still preserved in https://github.com/docutils/docutils-old.

$ git push --delete origin address-rendering && git branch -D address-rendering
# docutils-0.4 also has a tag with the same name, explicitly remove branch remotely
$ git push origin :refs/heads/docutils-0.4 && git branch -D docutils-0.4
$ git push --delete origin index-bug && git branch -D index-bug
$ git push --delete origin lossless-rst-writer && git branch -D lossless-rst-writer
$ git push --delete origin nesting && git branch -D nesting
$ git push --delete origin plugins && git branch -D plugins
$ git push --delete origin subdocs && git branch -D subdocs

Tags:

All tags from original import are to be dropped. They will be retagged by hand (below):

$ git push origin :refs/tags/prest-0.3.10 && git tag -d prest-0.3.10
$ git push origin :refs/tags/prest-0.3.11 && git tag -d prest-0.3.11
$ git push origin :refs/tags/start && git tag -d start
$ git push origin :refs/tags/merged_to_nesting && git tag -d merged_to_nesting
$ git push origin :refs/tags/initial && git tag -d initial
$ ... etc.

Garbage

The above did not correctly resize the repository. Possibly because of branches / tags not being filtered.

$ git filter-branch --tree-filter "rm -rf sandbox web prest" --prune-empty -f -- --all

Tags

Tags for docutils 0.1 and onward are tagged by hand. This expands the versions cataloged in svn for archaelogical purposes. New contributors trying to understand the inception of docutils will welcome being able to date back to 2002 and see the roots.

You can see the tags == new releases cut at https://github.com/docutils/docutils/releases, with their appropriate release dates.

$ git log --grep=version --grep=0.

The tags should match the versions available on pypi / sourceforge. Entering a bad command in pip $ pip install doc shows the versions available on pypi.

Site

GitHub Pages can use tools/buildhtml.py to host the website and documentation.

Relevant issue: https://github.com/docutils/docutils/issues/6 Repo: https://github.com/docutils/docutils.github.io Site: http://docutils.github.io/

Hurdles:

• gitattribute: $Revision -> $Id output to render correctly
• gitattribute: $Date of documentation buld to show

Documentation

Clarifying the documentation ecosystem

Docutils is capable of the task of building site without Sphinx.

There is a benefit that the community understand:

1. docutils and sphinx are different
2. docutils is capable of generating static websites by itself

The site should clarify to newcomers how docutils, sphinx and readthedocs are separate projects.

Supplementary docs in sphinx

That said, there is no harm in building sphinx documentation for the project alongside a static website. Here are the benefits that can be achieved with minimal effort by also having a sphinx documentation in parallel:

1. access to tools like sphinx.ext.autodoc to read into the machinery of docutils
2. multiple versions of documentation can be readily accessible
3. PDF, epub, etc. automatically built

docutils could fulfill 2, 3, and many other things with its own CI. That is a matter if there was sponsorship to do it (I am already using all my freetime outside of work)

Scenario:

A new docutils-docs repository is created, it can have a submodule / python module dependency of docutils. The sphinx documentation can then be updated, include the docs / autodoc modules from principal project, without interrupting the docutils repository.