Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Bash script for aggregating code-climate coverage stats across multiple parallel job runs on SemaphoreCI (e.g. with boosters)
#!/bin/bash
# Configuration via Environment Variables
# COVERAGE_BUCKET name of s3 bucket where coverage files will be sent.
# AWS_ACCESS_KEY_ID
# AWS_SECRET_ACCESS_KEY credentials for an IAM user with
# read/write permissions on the s3 bucket
# NON_COVERAGE_JOB_COUNT Number of semaphore tasks that do not generate coverage
# Defaults to 0
# Download the codeclimate reporter tool, make it executable.
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > /home/runner/.rbenv/bin/cc-test-reporter
chmod +x /home/runner/.rbenv/bin/cc-test-reporter
# Where the remote coverage files will be stored. Ensures that coverage files for
# different projects, branches, and builds do not clobber one another.
REMOTE="s3://$COVERAGE_BUCKET/$SEMAPHORE_PROJECT_NAME/coverage/branch/$BRANCH_NAME/build/$SEMAPHORE_BUILD_NUMBER"
# Don't try and send coverage if none exists.
if [ -d coverage ]; then
cc-test-reporter format-coverage \
--output "coverage/codeclimate.$SEMAPHORE_CURRENT_JOB.json"
# Push coverage stats from this job to central location
aws s3 sync coverage/ $REMOTE
# Remove the coverage dir after upload
rm -rf coverage
fi
# The sum-coverage tasks needs to know how many different
# jobs will be contributing coverage stats. We can start
# with the assumption that the number of semaphore threads
# defined by SEMAPHORE_THREAD_COUNT will be the number of
# jobs, but if there are some semaphore tasks that do not
# generate coverage stats, we can subtract those from the
# expected count.
NON_COVERAGE_JOB_COUNT=${NON_COVERAGE_JOB_COUNT:-0}
PARTS=$((SEMAPHORE_THREAD_COUNT-NON_COVERAGE_JOB_COUNT))
# Pull down all the coverages stats uploaded to s3 so far.
aws s3 sync $REMOTE coverage/
# Count the number of jobs that have been aggregated so far.
COMPLETED_PARTS=$(ls -l coverage/codeclimate.*.json | wc -l)
# If all the jobs have contributed their stats, then run the
# aggregation task and upload the coverage to code climate.
if [ $COMPLETED_PARTS -eq $PARTS ]; then
cc-test-reporter sum-coverage --output - --parts $PARTS \
coverage/codeclimate.*.json | \
cc-test-reporter upload-coverage --input -
fi
@skylarmb

This comment has been minimized.

Copy link

commented Sep 6, 2018

This is awesome! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.