Create a gist now

Instantly share code, notes, and snippets.

This script can be used to assist in documenting the Mahout Website source code on a developer's local machine.
######################################################################
#
# Author: Saleem Ansari <tuxdna@gmail.com>
# License: Apache License 2.0
#
# This script can be used to assist in documenting the Mahout Website
# source code on a developer's local machine.
#
#
# You need following dependencies to run this script:
#
# * Subversion
# * Python Markdown module
# * Perl YAML module
#
# For Ubuntu
# $ sudo apt-get install -y subversion libyaml-perl inotify-tools python-markdown
#
# For Fedora
# $ sudo yum install -y perl-YAML-LibYAML inotify-tools
#
#
# For the first time, bootstrap the site and cms build tool
#
# $ sh mahout-website.sh bootstrap
#
# This will checkout the site source and CMS build tool from SVN, and
# also setup various services/daemons required to auto-generate the
# website.
#
# From then on, you don't need to checkout any code from SVN, so just:
#
# $ sh mahout-website.sh
#
# The Mahout website can be accessed at http://localhost:8000/
#
# For example check this - http://localhost:8000/users/sparkbindings/home.html
#
######################################################################
## You may only need to tweak this variable, nothing else.
echo $HOME
if [ "x$HOME" == "x" ]
then
MAHOUT_WS=/tmp/mahout-website-workspace
else
MAHOUT_WS=$HOME/mahout-website-workspace
fi
echo "Workspace Directory: $MAHOUT_WS"
mkdir -p $MAHOUT_WS
SVN_REPO=http://svn.apache.org/repos/asf/mahout/site/mahout_cms/trunk
CMS_REPO=https://svn.apache.org/repos/infra/websites/cms/build/
SITE_TARGET_PATH=$MAHOUT_WS/mahout-website
SITE_SOURCE_PATH=$MAHOUT_WS/trunk
CMS_BUILDTOOL_PATH=$MAHOUT_WS/cms-build-tool
# Only do SVN checkout when bootstraping the sources for the first time
if [ "x$1" == "xbootstrap" ]
then
echo "Lets bootstrap ..."
echo "Checkout Mahout Website source: $SVN_REPO"
svn co $SVN_REPO $SITE_SOURCE_PATH
echo "Checkout CMS build tool: $CMS_REPO"
svn co $CMS_REPO $CMS_BUILDTOOL_PATH
echo "Bootstrap complete!"
fi
if [ ! -f "$SITE_SOURCE_PATH" ]
then
echo "Please run Bootstrap first!"
fi
WEB_ROOT=$SITE_TARGET_PATH/content
mkdir -p $WEB_ROOT
cd $WEB_ROOT
echo "Setup web-server..."
# kill existing server
pkill -f 'python -m SimpleHTTPServer'
# start a new server
python -m SimpleHTTPServer &
echo $PWD
echo "Setup markdownd daemon..."
# kill existing daemon
pkill -f 'python markdownd.py'
## Start the markdownd daemon
cd $CMS_BUILDTOOL_PATH
echo "CMS buildtool location: $CMS_BUILDTOOL_PATH"
export MARKDOWN_SOCKET=`pwd`/markdown.socket
export PYTHONPATH=`pwd`
python markdownd.py &
echo "Generate and watch files..."
## Build the site and keep watching for changes
perl build_site.pl --source-base $SITE_SOURCE_PATH --target-base $SITE_TARGET_PATH
firefox http://localhost:8000/ &
while inotifywait -e close_write -r $SITE_SOURCE_PATH
do
perl build_site.pl --source-base $SITE_SOURCE_PATH --target-base $SITE_TARGET_PATH
done
@tuxdna
Owner
tuxdna commented Apr 23, 2014

Based on the steps here

@tuxdna
Owner
tuxdna commented Mar 18, 2015

If this error is encountered:

Traceback (most recent call last):
  File "markdownd.py", line 43, in <module>
    dummy = markdown.markdown(x, EXTENSIONS)
  File "/usr/lib/python2.7/dist-packages/markdown/__init__.py", line 464, in markdown
    md = Markdown(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/markdown/__init__.py", line 154, in __init__
    configs=kwargs.get('extension_configs', {}))
  File "/usr/lib/python2.7/dist-packages/markdown/__init__.py", line 180, in registerExtensions
    ext = self.build_extension(ext, configs.get(ext, {}))
  File "/usr/lib/python2.7/dist-packages/markdown/__init__.py", line 265, in build_extension
    return module.makeExtension(**configs)
  File "/home/tuxdna/mahout-website-workspace/cms-build-tool/mdx_elementid.py", line 150, in makeExtension
    return IdExtension(configs=configs)
  File "/usr/lib/python2.7/dist-packages/markdown/extensions/__init__.py", line 36, in __init__
self.setConfigs(kwargs.pop('configs', {}))
  File "/usr/lib/python2.7/dist-packages/markdown/extensions/__init__.py", line 75, in setConfigs
    for key, value in items:
TypeError: 'NoneType' object is not iterable

The apply the following patch:

$ svn diff cms-build-tool/mdx_elementid.py
Index: cms-build-tool/mdx_elementid.py
===================================================================
--- cms-build-tool/mdx_elementid.py (revision 944228)
+++ cms-build-tool/mdx_elementid.py (working copy)
@@ -146,7 +146,7 @@
         md.treeprocessors.add("elid", idext, "_begin")


-def makeExtension(configs=None):
+def makeExtension(configs={}):
     return IdExtension(configs=configs)

 if __name__ == "__main__":

Reference: waylan/Python-Markdown#357

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment