Skip to content

Instantly share code, notes, and snippets.

@marcbachmann
Last active September 21, 2020 14:08
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save marcbachmann/16574ba8c614bb3b78614a351f324b86 to your computer and use it in GitHub Desktop.
Save marcbachmann/16574ba8c614bb3b78614a351f324b86 to your computer and use it in GitHub Desktop.
Several bash scripts for TravisCI. Scripts to update docker, run a http proxy, and pull most recent docker image from docker hub
docker exec squid cat /var/log/squid3/access.log
WHITELIST=$(cat ./.HTTP_PROXY_WHITELIST &> /dev/null || echo $HTTP_PROXY_WHITELIST | tr " " "\n")
>&2 echo Using whitelist: && >&2 echo $WHITELIST
echo $WHITELIST > ./whitelist
if [ -z "$WHITELIST" ]; then
$whitelist_config='
http_access allow all
http_access allow CONNECT all
'
else
$whitelist_config='
acl whitelist dstdomain "/etc/squid3/whitelist"
http_access allow whitelist
http_access allow CONNECT whitelist
http_access deny all
'
fi
echo "
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.0.0/16
acl localnet src fc00::/7
acl localnet src fe80::/10
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_port 3128
coredump_dir /var/spool/squid3
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
refresh_pattern . 0 20% 4320
http_access deny !Safe_ports
acl SSL_ports port 443
acl SSL_ports port 80
http_access deny CONNECT !SSL_ports
$whitelist_config
cache deny all
cache_dir null /tmp
" > ./squid.conf
CONTAINER=$(docker run -d --name squid --rm -v $PWD/squid.conf:/etc/squid3/squid.conf -v $PWD/whitelist:/etc/squid3/whitelist --publish 3128:3128 sameersbn/squid:3.3.8-23 -d1)
IP=$(docker inspect --format '{{.NetworkSettings.IPAddress }}' $CONTAINER)
>&2 echo ">> Proxy started on $IP:3128"
echo $IP:3128
#!/bin/bash
trap "exit" INT TERM
trap 'if [ "$pid" != "" ]; then kill $pid; fi' EXIT
for IMAGE_TAG in "$@";
do
bash -c ">&2 printf '> Try to fetch prebuilt image $IMAGE_TAG';while :;do >&2 printf .;sleep 1;done;" &
pid=$!
IMAGE_ID=$(docker pull $IMAGE_TAG &> /dev/null && docker images $IMAGE_TAG -q | head -1)
kill $pid
pid=
>&2 echo ""
if [ "$IMAGE_ID" != '' ]; then
>&2 echo ">> Successfully fetched image $IMAGE_TAG $IMAGE_ID"
echo $IMAGE_ID
exit 0
else
>&2 echo "Failed to fetch prebuilt image $IMAGE_TAG"
fi
done
>&2 echo '>> No prebuilt docker image found'
# We still exit with 0 because we're handling a missing docker
# image in the build step. We don't want to cancel the tests here
exit 0

Travis Docker Scripts

Usage

Define the GIST url and pipe some script to bash/sh

GIST=https://gist.githubusercontent.com/marcbachmann/16574ba8c614bb3b78614a351f324b86/raw/a7e17acc57938a047c093f560858e3b0a8e2c741
curl -s $GIST/script-name | bash

Upgrade docker

Upgrade to the latest docker and docker-compose version

install:
  - GIST=https://gist.githubusercontent.com/marcbachmann/16574ba8c614bb3b78614a351f324b86/raw/a7e17acc57938a047c093f560858e3b0a8e2c741
  - curl -s $GIST/update-docker-engine.sh | bash

Cache your images

Download your most recent build from the docker registry and use it as base image

install:
  # preload existing container for caching
  - IMAGE_TAG=livingdocs/editor:${TRAVIS_TAG:-$TRAVIS_BRANCH}
  - docker login -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD" 1>/dev/null
  - CACHED_IMAGE=$(curl -s $GIST/pull-docker-image.sh | bash -s $IMAGE_TAG livingdocs/editor:latest)

script:
  - docker build --cache-from ${CACHED_IMAGE:-false} -t editor .
  - docker tag editor livingdocs/editor:latest
  - docker push livingdocs/editor

http proxy (squid)

Setup

ip=$(curl -s $GIST/http-proxy-setup.sh | bash)
export HTTP_PROXY=http://$ip
export HTTPS_PROXY=https://$ip

Check logs

curl -s $GIST/http-proxy-requests.sh | bash
#!/bin/bash
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -o Dpkg::Options::="--force-confold" --force-yes -y docker-ce
sudo rm /usr/local/bin/docker-compose
GHRELEASES=`curl -w "%{url_effective}\n" -I -L -s -S https://github.com/docker/compose/releases/latest -o /dev/null | sed 's/\/tag\//\/download\//'`
curl -L $GHRELEASES/docker-compose-`uname -s`-`uname -m` > docker-compose
chmod +x docker-compose
sudo mv docker-compose /usr/local/bin
echo ----------------------------------
docker --version
docker-compose --version
echo ----------------------------------
echo Docker Info:
docker info
echo ----------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment