Since I have wasted a lot of time trying to make this work, I just wanted to post this gist to help anyone having issues to setup the plugin.
- An AWS account
- A running cluster on EMR with Ganglia installed.
- An elasticsaerch cluster running.
def explode_from_fields(df, target_columns: list, separator: str): | |
""" df = dataframe to split, | |
target_columns = list of the columns containing the values to split, | |
if the elements returned from the split are not the | |
same for each columns, the shorter ones are extended | |
to the longest. | |
separator = the symbol used to perform the split | |
returns: A dataframe with each entry for the target column separated, | |
with each element moved into a new row. | |
The values in the other columns are duplicated across the newly divided rows.""" |
#!/bin/bash | |
SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" | |
COMMITS_VERSIONS=$(git log --follow -- "${SCRIPTDIR}/../VERSION" | grep -E '^commit' | sed 's#commit ##g') | |
NEXT=$(echo -e "$COMMITS_VERSIONS" | head -1) | |
COMMITS_VERSIONS=$(echo -e "$COMMITS_VERSIONS" | tail -n+2) | |
for commit in $COMMITS_VERSIONS; do | |
VERSION=$(git show ${NEXT}:../pycuebiq/VERSION) |
#!/bin/bash | |
commit_hash=$(git rev-parse HEAD | head -c8) | |
text="*[DEPLOY] - ($commit_hash) I have made a new deploy for Project! :bomb:*" | |
token="YOUR_TOKEN" | |
slackhost="YOUR_SLACK_HOST" | |
channel="CHANNEL_NAME" | |
escapedText=$(echo "$text" | sed 's/"/\"/g' | sed "s/'/\'/g" ) | |
json="{\"channel\": \"#$channel\", \"text\": \"$escapedText\",\"icon_emoji\":\":apple:\",\"username\":\"My Bot\"}" | |
# Example of complex data |
#! /bin/bash | |
# Note: run from /var/lib/ganglia directory, no need for subdir | |
# Run with: find /var/lib/ganglia/rrds/ -type d -exec sh -c 'cd "{}" ; /var/lib/ganglia/rrd_exporter.sh ;' \; | |
mkdir -p /tmp/ganglia-export/xml/ | |
RUNDATE="$(date --date='now' +%F-%R)" | |
BACKUPFILE="/tmp/ganglia-export/rrds-${RUNDATE}.tar" | |
EXPORTFILE="/tmp/ganglia-export/xml-export-${RUNDATE}.tar" | |
# backup current rrd files -- tar all and place in /tmp |
Since I have wasted a lot of time trying to make this work, I just wanted to post this gist to help anyone having issues to setup the plugin.
preBootstrapCommands: &preboot | |
- "export DOCKER_SECRET='my-docker-secret'" | |
- "export DISK_WITH_RAID=''" | |
- "export DISK_MOUNT=''" | |
- | | |
#!/bin/bash -x | |
# Writing Docker Secret | |
if [[ ! -z "${DOCKER_SECRET}" ]]; then | |
secret=$(aws --region $AWS_REGION secretsmanager get-secret-value --secret-id ${DOCKER_SECRET} --query SecretString --output text) |