Skip to content

Instantly share code, notes, and snippets.

@lpar
Created June 17, 2011 20:23
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lpar/1032241 to your computer and use it in GitHub Desktop.
Save lpar/1032241 to your computer and use it in GitHub Desktop.
Automatic MD5 checksum generating daemon for use with vsftpd
#!/usr/bin/env ruby
# encoding: UTF-8
'di '
'ig00 '
# VSFTPD auto-MD5 program
#
# Run man -l <this file> for documentation
require 'rb-inotify'
require 'elif'
require 'digest/md5'
require 'logger'
require 'fileutils'
require 'yaml'
conffile = ARGV[0] || '/etc/vsftpd/md5daemon.yaml'
config = YAML.load_file(conffile)
FTPUID = config['ftp_uid'] || 'ftp'
FTPGID = config['ftp_gid'] || 'ftp'
PIDFILE = config['pid_file'] || '/var/run/md5daemon.pid'
MYLOG = config['md5daemon_log'] || '/var/log/vsftpd-md5'
HOMEDIR = config['ftp_incoming'] || '/home/ftp/incoming'
XFERLOG = config['vsftpd_xferlog'] || '/var/log/vsftpd'
BUFFERSIZE = config['buffer_size'] || 10*1024*1024
LOG = Logger.new(MYLOG)
def md5sum(infilename, outfilename)
LOG.info("Checksumming #{infilename}")
infile = File.open(infilename, 'r')
fnam = infilename.sub(/.*\//, '')
digest = Digest::MD5.new()
while data = infile.read(BUFFERSIZE)
digest << data
end
outfile = File.open(outfilename, 'w')
outfile.puts("#{digest.hexdigest} #{fnam}")
outfile.close
FileUtils.chown(FTPUID, FTPGID, outfilename)
FileUtils.chmod(0644, outfilename)
FileUtils.chmod(0, infilename)
infile.close
LOG.info("Checksummed #{infilename}")
end
def processfile(user, path, bytes)
filename = "#{HOMEDIR}/#{user}#{path}"
md5name = "#{filename}.md5"
# If we hit an MD5 we already generated, tell caller we're done
if File.exists?(md5name) and File.size(md5name) >= 32
LOG.info("#{md5name} already exists, we're done.")
return false
end
if !File.exists?(filename)
LOG.warn("#{filename} not found.")
return true
end
fsize = File.size(filename)
if fsize != bytes
LOG.warn("#{filename} is #{fsize} bytes, expected #{bytes}.")
return true
end
# OK, let's MD5
md5sum(filename, md5name)
return true
end
def processlog
LOG.info("Log file changed, processing.")
log = Elif.open(XFERLOG, "r")
while line = log.gets.untaint
line.chop!
m = line.match(/\[pid \d+\] \[([^\]]+)\] OK UPLOAD: Client "([0-9\.:]*)", "(.*)", (\d+) bytes/)
if m
LOG.info("Upload detected.")
user = m[1]
path = m[3]
bytes = m[4].to_i
if !processfile(user, path, bytes)
break
end
else
LOG.info("Ignoring line {#{line}}")
end
end
log.close
LOG.info("Done processing log file.")
end
# Now for the code to run on startup...
if File.exists?(PIDFILE)
pid = File.open(PIDFILE).gets.chop.to_i
begin
Process.getpgid(pid)
STDERR.puts("Already running according to #{PIDFILE}")
exit 0
rescue Errno::ESRCH
STDERR.puts("Cleaning up old #{PIDFILE}")
File.unlink(PIDFILE)
end
end
mypid = Process.fork
if mypid
# I am not the child
exit 0
end
begin
mypid = Process.pid
pidfile = File.open(PIDFILE, "w")
pidfile.puts(mypid)
pidfile.close
# Run once on startup to catch any leftovers
processlog
# Then run whenever the VSFTPD log file changes
notifier = INotify::Notifier.new
notifier.watch(XFERLOG, :modify) { processlog }
notifier.run
ensure
File.erase(PIDFILE)
end
__END__
.00
'di
.TH md5daemon 8 "17 Jun 2011"
.SH NAME
md5daemon \- MD5 checksumming daemon for vsftpd
.SH SYNOPSIS
.B md5daemon
.SH DESCRIPTION
.PP
This is a daemon designed to be run on a Linux-based FTP server using
vsftpd. It monitors the vsftpd transfer log using inotify, and creates .md5
files containing checksums for all files uploaded. This provides users with an
easy way to check that their files were not corrupted during upload.
.PP
Two Ruby Gems are required: \fBrb-inotify\fR and \fBelif\fR. Both are
available from rubygems.org. The MD5 calculation is performed using Ruby's
standard library, which is a wrapper around native C code.
.PP
The code expects the transfer log to be in vsftpd's native format, rather than
legacy wuftpd format. Therefore the vsftpd server should be configured with
\fBxferlog_enable=YES\fR and \fBxferlog_std_format=NO\fR
.PP
This program has been used for some months on a server that regularly sees
uploads of 10GB or more.
.PP
Configuration is in YAML. The path to a configuration file can be provided on
the command line; otherwise, \fB/etc/vsftpd/md5daemon.yaml\fR is used.
.SH OPTIONS
.PP
The following configuration parameters can be set:
.TP
\fBbuffer_size\fR
buffer size to use when checksumming, default 10MiB
.TP
\fBftp_uid\fR
the user to chown the MD5 files to, default \fBftp\fR
.TP
\fBftp_gid\fR
the group to chown the MD5 files to, default \fBftp\fR
.TP
\fBftp_incoming\fR
the FTP home directory under which uploads are found,
default \fB/home/ftp/incoming\fR
.TP
\fBvsftpd_xferlog\fR
the location of vsftpd's transfer log, default \fB/var/log/vsftpd\fR
.TP
\fBmd5daemon_log\fR
where to log md5daemon's activity, default \fB/var/log/vsftpd-md5\fR
.TP
\fBpid_file\fR
the pid file to check/create, default \fB/var/run/md5daemon.pid\fR
.SH EXAMPLES
.nf
# Example configuration file
vsftpd_xferlog: /var/log/vsftpd
ftp_uid: vsftpd
ftp_gid: vsftpd
ftp_incoming: /srv/ftp/incoming
.fi
.SH "SEE ALSO"
md5sum(1)
.SH AUTHOR
mathew <meta@pobox.com>
.SH COPYRIGHT
Copyright (c) 2010-2011. Licensed under the same terms as vsftpd.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment