Skip to content

Instantly share code, notes, and snippets.

@atenni

atenni/Archive Moodle Logs.md

Last active June 7, 2017 08:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save atenni/6182574 to your computer and use it in GitHub Desktop.
Save atenni/6182574 to your computer and use it in GitHub Desktop.
Download ZIP
Script to archive moodle logs to a separate stats server for future long time series/big data analysis.
Raw
Archive Moodle Logs.md

Archiving mdl_log, daily

(From http://www.ualberta.ca/~dostatni/moodlemoot2013/mdl_log.html)

The following script is executed daily. It generates a CSV file with all the moodle activity for previous day. That file is saved together with apache access and error logs in a log server. This data is used to verify student challenges ( eg. "I couldn't submit my assignment because there was a problem with the system"), identify trends and trace the origin of errors (to a specific code deployment).

This level of archiving allows us to remove the mdl_log entries from the active database. Currently the logs take up about half of the size of the database on disk.

dump_yesterdays_logs.sql

copy ( select * from mdl_log where time >= extract ( 'epoch' from (current_date - interval '1 day')) and time < extract ( 'epoch' from current_date ) ) to '/backups/dumps/mdl_log.csv' with CSV HEADER FORCE QUOTE url;

crontab:

/usr/bin/psql -f /var/lib/postgresql/scripts/dump_yesterdays_logs.sql moodle >> /var/log/postgresql/mdl_log.log 2>&1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment