Setting up a developers area for new python package structure in the GEM online software environment

dev-setup.md

Instructions to set up a working developers area

• Clone the repository you'll be doing developments on
  • For most repositories other than cmsgemos, you should do the following to bring in the submodules
    • git submodule init
    • git submodule update
  • Development will, in general, not be supported on non GEM DAQ machines, so if you choose to do so, support will only be provided on a best effort basis
• If you will be doing developments that depend on external packages, find the appropriate version to test against
  • gemplotting
  • vfatqc
  • cmsgemos
• Take the appropriate tar.gz file for the package
  • gempython_vfatqc-2.0.0.tar.gz
  • gempython_gemplotting-1.0.0.tar.gz
  • cmsgemos
• Set up a python virtual environment somewhere:

python -m virtualenv -p <python version> --system-site-packages /path/to/your/venv
. /path/to/your/venv/bin/activate
python -m pip install -r requirements-dev.txt
# equivalent to the following
# python -m pip install -U importlib setuptools pip
# with a python2.6 virtualenv on slc6 default, uncomment the following line
# python -m pip install -I importlib 'setuptools<39.2.0'  'pip<10.1.0'
# install the necessary, external, unmodified dependencies
python -m pip install <gempython package>.tar.gz --no-deps

# set up the environment
export PATH=$VIRTUAL_ENV/lib/python*/site-packages/gempython/scripts:$PATH
export PATH=$VIRTUAL_ENV/lib/python*/site-packages/gempython/gemplotting/macros:$PATH

# create the mapping files
find <path/to/venv/lib/../site-packages/gempython> -type f -name buildMapFiles.py -exec python {} \;

• You can do your development inside a virtualenv with no problem
  • Due to the tree structure of the virtualenv, trying to make rpm a package from inside a virtualenv will not currently work
  • A solution for this will be investigated, but recommend to not try to build from inside the virtualenv, rather use it for testing the built package
  • A currently untested workaround is a new make pip target, which will only build the bdist/sdist tarballs
• When you are ready to test your changes to the package you are working on
  • Compile the changes and create a new package

make && make pip

• Install the new package into your virtualenv

pip install -I rpm/<package_updated>.tar.gz

Same in zsh:
% source setup_gemdaq.sh -p dorney_script_venv
VENV_ROOT dorney_script_venv
ELOG_PATH not set, please set ELOG_PATH to a directory where plots created by analysis applications will be written
(export ELOG_PATH=///) and then rerun this script

@lmoureaux This is working as intended, you need to set $ELOG_PATH as it is stating. For the bash issue I suspect it was a lack of OPTIND being correctly set so that getopts would work correctly when sourced; please try again.

...for Python 2.6

Whether 2.6 is supported depends on when the p5 machines will be migrated to cc7. If the time scale is long then okay we can think of adding support; but otherwise I think this is a moot point. You can use the cc7 virtual machine at P5 which defaults to python2.7.

Network isn't accessible, so packages have to be downloaded by hand

That's not an issue with the script. You need to either setup network configuration to have port forwarding https or download the packages by hand (as is done when using setup_ctp7.sh).