You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Bash configuration and other to make cli usage better
Some helpers for bashrc
Enter clean (umodified environment) with original bash
I use it for MacOS for testing script which should work on a clean MacOS system (which has bash 3)
In .bashrc:
# First line in file, store original envexport CLEAN_ENV="$(env | xargs)"# [...] Do whatever customzations that you do [...]# I need sometimes to get rid of any customizations (GNU coreutils especially)# for testing stuff that needs to be portable (usually CI automation).# Also use the build-in old-as-hell bash :/functionclean-env() {
env -i $CLEAN_ENV'PS1=\[\e[0;31m\](clean)\[\e[0m\] \[\e[0;33m\]\u\[\e[0;32m\]@\h\[\e[0m\] \[\e[0;34m\]\W\[\e[0m\] $ ' /bin/bash --norc --noprofile
}
Especially useful with complicated opts (overlayfs! docker!).
mount | sed -e 's/$/\n/g'| sed -e 's/(\([^)]\+\))/(\n \1\n)/g'| sed -e 's/,/,\n /g'| sed -e 's/:/:\n /g'| sed -e 's/=/=\n /g'
The same but in a one-line function that allows grepping...
functionmntgrep() { mount | grep $@| sed -e 's/$/\n/g'| sed -e 's/(\([^)]\+\))/(\n \1\n)/g'| sed -e 's/,/,\n /g'| sed -e 's/:/:\n /g'| sed -e 's/=/=\n /g'; }
Example output
overlay on /var/lib/docker/overlay2/3437d9dbad97a86246dd19286d642ea99150625bb636ba699ff8780bdfa83621/merged type overlay (
rw,
relatime,
lowerdir=
/var/lib/docker/overlay2/l/AU2SFSYBUK7MAXSPLMTE7WDPCL:
/var/lib/docker/overlay2/l/WYLIWNY3O6ORPQRGRW2WV5N45S:
/var/lib/docker/overlay2/l/OSTUXN4WFDPDMSCR2WAJIUWRHL:
/var/lib/docker/overlay2/l/KAX5BB3XQFMRKHSYOQT6NMYYBZ:
/var/lib/docker/overlay2/l/OTUEIXT3M7PINGAD5KNCCWLRW7:
/var/lib/docker/overlay2/l/GGANPDIO7LWJAMPXFV65PI7CBB:
/var/lib/docker/overlay2/l/EZLAO3H6BUP66KALK77SFP6GMG:
/var/lib/docker/overlay2/l/R4TPZEFLP24Q2FCSSTFZBMZIKC,
upperdir=
/var/lib/docker/overlay2/3437d9dbad97a86246dd19286d642ea99150625bb636ba699ff8780bdfa83621/diff,
workdir=
/var/lib/docker/overlay2/3437d9dbad97a86246dd19286d642ea99150625bb636ba699ff8780bdfa83621/work
)
shm on /var/lib/docker/containers/9f35262ee7bc2e0781e55c1bf8755cbca9190d92675073305416f20d4cf8be91/mounts/shm type tmpfs (
rw,
nosuid,
nodev,
noexec,
relatime,
size=
65536k
)
nsfs on /run/desktop/docker/netns/d2cbd479cc09 type nsfs (
rw
)
This will take a looong time - go get a coffee or sth!
(Divided into two steps, this way you can see progress and not get bored.)
echo'This will take a long time, be patient...' \
&& (echo -e '\n--- Counting logs ---\n'; du -sh -t 5M --time /var/log/*| tee -a /tmp/var_log_sizes) \
&& (echo -e '\n--- Counting workspaces ---\n'; du -sh -t 100M --time /var/lib/jenkins/workspace/*| tee -a /tmp/jenkins_workspace_sizes) \
&& (echo -e '\n--- Counting jobs ---\n'; du -sh -t 20M --time /var/lib/jenkins/jobs/*| tee -a /tmp/jenkins_job_sizes) \
&& (echo -e '\n--- Biggest system logs ---\n'; cat /tmp/var_log_sizes | sort -hrs ) \
&& (echo -e '\n--- Biggest workspaces ---\n'; cat /tmp/jenkins_workspace_sizes | sort -hrs ) \
&& (echo -e '\n--- Biggest job archives ---\n'; cat /tmp/jenkins_job_sizes | sort -hrs )
Note: You can safely remove workspaces in most cases (unless a job is running!) because it will be rebuilt
next time job is ran (just it will take a while longer). But YMMV so execute caution.
I recommend removing workspace which were last used long time ago (hence the --time switch). They probably
won't be used anytime soon.
Reclaim space used by docker
CAUTION! This will remove all docker data. Generally all images
will be fetched automatically the next day from repos, so rather
no biggie.
docker system prune -a -f && docker volume prune -f
Reclaim space used by archived builds of some type (build-app-* as example)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
echo "Will attempt to show live stats using $USER@$HOST from log $LOGFILE!"
echo "---"
echo "Report: $REPORTPATH | $REPORTURL"
echo "Websocket: localhost:$WSPORT"
echo "---"
echo "*** Press CTRL+C (Twice!) to stop ***"
echo "---"
echo "Setup temps..."
mkdir -p "$TMPDIR"
echo "<html><head><meta http-equiv="refresh" content="2"></head><center><h1>Loading report for <br/><em>$(title)</em></h1> <br/><br/>Refresh in a moment...<center></html>" > $REPORTPATH
echo "Starting PHP webserver..."
php -S "127.0.0.1:$HTTPPORT" -t "$TMPDIR" &
echo "Open report in browser..."
open "$REPORTURL"
echo "Starting goaccess parser with logs tailed live via SSH..."
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
console.log('Posted to slack: ' + JSON.stringify(payload));
resolve(res);
});
req.write(payload);
req.end();
});
}
function parseGroup(group) {
var elements = group.split('/');
if (elements.length < 4) {
return {
project: 'unknown',
env: 'unknown'
}
}
return {
project: elements[1],
env: elements[2],
};
}
function handleLogItem(data, project, env, link) {
var message = data.message;
if (message.length > 600) {
message = message.substring(0, 600) + '\n[...]';
}
return postToSlack({
"text": `:skull_and_crossbones: CRITICAL error on *${project}*-_${env}_: <${link}|See details here>`,
"attachments": [
{
"color": "#D00000",
"fields":[
{
"title": "Exception",
"value": message,
"short": false
}
]
}
]
});
}
function handleData(data) {
var handled = [],
group = data.logGroup,
stream = data.logStream,
groupData = parseGroup(group),
project = groupData.project,
env = groupData.env,
region = process.env.AWS_DEFAULT_REGION,
link = `https://${region}.console.aws.amazon.com/cloudwatch/home?region=${region}#logEventViewer:group=${encodeURIComponent(group)};stream=${encodeURIComponent(stream)}`
data.logEvents.forEach((item) => {
handled.push(
handleLogItem(
item,
project,
env,
link
)
);
})
return Promise.all(handled);
}
exports.handler = (event, context, done) => {
var payload = new Buffer(event.awslogs.data, 'base64');
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters