Skip to content

Instantly share code, notes, and snippets.

View berghaus's full-sized avatar

Frank Berghaus berghaus

16 followers · 6 following
View GitHub Profile
@berghaus
berghaus / csnode_config.pp
Last active December 24, 2015 03:09
Cloud Scheduler puppet manifest.
class { 'gce_node':
head => 'to.be.contextualized.by.cloud.scheduler',
role => 'csnode',
use_cvmfs => true,
condor_pool_password => undef,
condor_use_gsi => true,
condor_slots => $processorcount,
use_xrootd => false,
condor_vmtype => 'cernvm-batch-node-2.7.2-x86_64',
condor_homedir => '/scratch',
@berghaus
berghaus / belle_csnode.pp
Created October 24, 2013 23:18
Cloud scheduler VM contextualization for Belle2 experiment using the atlasgce_module's belle branch.
class {'gce_node':
head => 'to.be.contextualized.by.cloud.scheduler',
role => 'csnode',
use_cvmfs => true,
condor_use_gsi => true,
condor_slots => 32,
use_xrootd => false,
cvmfs_domain_servers => "http://cvmfs-stratum1-kit.gridka.de/cvmfs/belle;http://cvmfs-stratum-one.cern.ch/opt/belle",
use_apf => false,
condor_vmtype => 'cernvm-batch-node-2.7.2-x86_64',
@berghaus
berghaus / puppet.conf-atlas
Created November 20, 2013 23:49
Puppet agent configuration for ATLAS batch nodes
[main]
# The Puppet log directory.
# The default value is '$vardir/log'.
logdir = /var/log/puppet
# Where Puppet PID files are kept.
# The default value is '$vardir/run'.
rundir = /var/run/puppet
# Where SSL certificates are kept.
@berghaus
berghaus / consistency_check.sh
Last active August 29, 2015 13:59
Bash script to check for consistency between instances registered in nova, managed by cloud scheduler, and advertising to HTCondor.
#!/bin/bash
#
# Check whether servers booted on gridpp-imperial match between nova, cloud scheduler, and condor
# lists any inconsistencies.
#
LOG_FILE=/var/log/cloud_cleanup.log
E_BADARGS=85
if [ ! -n "$1" ]
@berghaus
berghaus / LocalComputingElement.py
Last active August 29, 2015 14:08
Allow DIRAC to handle large lists of jobs by breaking the list into manageable chunks.
def _getJobStatus( self, jobIDList ):
""" Get the status information for the given list of jobs
"""
resultDict = {}
jobDict = {}
#for job in jobIDList:
stdout = ''
stderr = ''
from DIRAC.Core.Utilities.List import breakListIntoChunks
@berghaus
berghaus / killAmazonInstances.sh
Created October 29, 2014 23:12
Script to kill stopped instances in all amazon EC2 regions
#!/bin/bash
export EC2_URL=https://ec2.us-east-1.amazonaws.com
export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64
export EC2_HOME=/usr/local/ec2/ec2-api-tools-1.7.1.0
export PATH=$PATH:$EC2_HOME/bin
export AWS_ACCESS_KEY=
export AWS_SECRET_KEY=
for endpoint in $(ec2-describe-regions | awk '{print $3}');
@berghaus
berghaus / keybase.md
Created February 16, 2015 13:49
keybase identification

Keybase proof

I hereby claim:

  • I am berghaus on github.
  • I am berghaus (https://keybase.io/berghaus) on keybase.
  • I have a public key whose fingerprint is DF21 F80F C7DE CC9E 331B E8A4 9CD8 F59F 0FA8 2BFC

To claim this, I am signing this object:

@berghaus
berghaus / query-inspire.py
Created February 9, 2016 10:40
A script to query the number of publications per year by the LHC collaborations from the inspire records. Written with hints from Sunje, Annette, Patricia, and Anxhela :-)
import json
import urllib2
from prettytable import PrettyTable
experiments = [ 'cern-lhc-alice', 'cern-lhc-atlas', 'cern-lhc-cms', 'cern-lhc-lhcb']
years = range(2008,2016)
publications = PrettyTable( ["Year"] + experiments )
for year in years:
@berghaus
berghaus / csv2json.py
Last active February 29, 2016 12:01
A script to convert CSV data into a list of JSON dictionaries.
# script to convert accelerators csv to json
import pandas
import json
import sys
def main(argv):
in_file = argv[0]
out_file = argv[1]
@berghaus
berghaus / ATLAS_CAP_Metadata.md
Created March 15, 2016 16:10
a list of stuff from CMS/LHCb with some things I think may be relevant for ATLAS.
  • Basic Information:
    • Analysis number: On CMS' DAS as a catalogue of analyses. Do we have a similar register of ATLAS analyses?
  • Dataset: An array of files following the software provenance of the xAOD (level-3) used by the analysis back to the original RAW (level-4) data. This would be on AMI right now. Each data set here should contain:
    • Title
    • Description
    • type (RAW, ESD, AOD, D3PD)
    • Software version that produced the data
    • Dataset ID
    • Number of events
  • Integrated luminosity