Skip to content

Instantly share code, notes, and snippets.

@romaricpascal
Created March 22, 2017 11:54
Show Gist options
  • Save romaricpascal/1cbbc162d2b129b2910701b61d903382 to your computer and use it in GitHub Desktop.
Save romaricpascal/1cbbc162d2b129b2910701b61d903382 to your computer and use it in GitHub Desktop.
Import OVH shared logs into Piwik
#! /usr/bin/env bash
cd "$(dirname "${BASH_SOURCE[0]}")"
./ovh-download-logs.sh `date --date=yesterday +"%Y-%m-%d"` | zcat | ./piwik-import.sh
#! /usr/bin/env bash
: ${1?"Please provide the target date for logs"}
DATE=$1
DOMAIN=<fill me>
FOLDER=`date --date=$DATE +"logs-%m-%Y"`;
FILE="$DOMAIN-$(date --date=$DATE +'%d-%m-%Y').log.gz";
URL="https://logs.cluster020.hosting.ovh.net/$DOMAIN/$FOLDER/$FILE";
curl `cat .curlauth` $URL
#! /usr/bin/env bash
PIWIK_URL=<fill me>
PIWIK_PATH=<fill me>
python "$PIWIK_PATH/misc/log-analytics/import_logs.py" \
--enable-static \
--enable-http-errors \
--url=$PIWIK_URL \
`cat .piwikauth` \
--add-sites-new-hosts \
--log-format-regex='(?P<ip>\S+)\s+(?P<host>[\w\-\.]*)(?::\d+)?\s+(?P<userid>\S+)\s+\[(?P<date>.*?)\s+(?P<timezone>.*?)\]\s+"\S+\s+(?P<path>.*?)\s+\S+"\s+(?P<status>\S+)\s+(?P<length>\S+)\s+"(?P<referrer>.*?)"\s+"(?P<user_agent>.*?)"' \
- < /dev/stdin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment