Skip to content

Instantly share code, notes, and snippets.

@cristianp6
Last active April 29, 2020 10:26
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save cristianp6/c063e36826a6b1623f3d to your computer and use it in GitHub Desktop.
Save cristianp6/c063e36826a6b1623f3d to your computer and use it in GitHub Desktop.
Write retrieved emails on a log file (Ubuntu/Debian)

Write retrieved emails in a file

As root, install fetchmail

$ apt-get install fetchmail

Check if fetchmail has SSL support: if you see something like "libssl.so.0" then yours has it

$ ldd /usr/bin/fetchmail

Test Gmail SSL certificates

$ openssl s_client -connect pop.gmail.com:995

You should see something like "+OK Gpop ready for requests from ..."

Setup fetchmail

$ wget -O ~/.fetchmailrc https://gist.github.com/cristianp6/c063e36826a6b1623f3d/raw/f429c4633992c9f8d95e2263c2876b7d38972912/fetchmailrc
$ chmod 0700 ~/.fetchmailrc

Edit the file with your settings

$ nano ~/.fetchmailrc

As root, create the "fetchmail handler"

$ wget -O /sbin/fetchmailhandler https://gist.githubusercontent.com/cristianp6/c063e36826a6b1623f3d/raw/06537116cfb81041932149257d68d2ffcc6f363a/fetchmailhandler
$ chmod +x /sbin/fetchmailhandler
  • Change the ownername as the username who have to read the file
  • Change the filepath where you want to store the emails
$ nano /sbin/fetchmailhandler

Test fetchmail

$ fetchmail -d0 -vk -f .fetchmailrc

Kill eventual fetchmail daemon process with:

$ fetchmail -q

Set a cronjob for fetchmail

Find fetchmail path

$ whereis fetchmail

The path should be something like "fetchmail: /usr/bin/fetchmail ..."

$ crontab -e

# Fetch emails every 5 minutes max
*/5 * * * * /usr/bin/fetchmail > /dev/null 2>&1

Or start fetchmail daemon

$ fetchmail

Now you can parse the file "/path/to/the/file/[unixtimeInMicroseconds].eml"

Source

#!/bin/bash
owner="YourOSuserHere"
filepath="/path/to/the/file/"
filename="$(date +%s%N).eml"
mkdir -p $filepath && cd $filepath
touch $filename && chown $owner $filename
while read line
do
echo "$line" >> $filename
done < "${1:-/proc/${$}/fd/0}"
#set daemon 600
set postmaster "YourOSuserHere"
poll pop.gmail.com with proto POP3 and options no dns
user 'account@domain.com' there with password 'YourPasswordHere' is YourOSuserHere here options ssl
mda /sbin/fetchmailhandler
@davidtmerritt
Copy link

Nice work: this got me further than other tutorials did. There are a couple problems, still:

  1. your wget command to fetch the .fetchmailrc file points to a fetchmailhandler file
  2. when running the fetchmail test command (with switches), I get an error: /sbin/fetchmailhandler: line 7: syntax error near unexpected token done' /sbin/fetchmailhandler: line 7:done < "${1:-/proc/${$}/fd/0}"'

@cristianp6
Copy link
Author

Thanks for the hint!
I've updated the README file with the right wget link to fetchmailrc and ASP I'll check the second error you've reported.

@bmihaescu
Copy link

Hello,

After "while read line" you should put ; otherwise you get syntax error at line 7.

Copy link

ghost commented Aug 2, 2016

Thank you - all up after adding the ; on line 7

@ulfrenman
Copy link

Nice work, but I can't get it working and I don't understand all of it. Questions:

  1. Whats with the "${1:-/proc/${$}/fd/0}", why not just /proc/${$}/fd/0? What does that construction give us?
  2. Why cant I use something like tee or cat as mda? Ex: mda "tee /path/to/the/file/mails.log > /dev/null"?
  3. This suggest cat could be used as mda.

@Jonyx4
Copy link

Jonyx4 commented Apr 2, 2017

Any idea why in gmail it keeps processing all the same mails all the time?

@hlev
Copy link

hlev commented Feb 18, 2019

Any idea why in gmail it keeps processing all the same mails all the time?

In case someone wanders here: because fetchmail with POP3 and the above configuration does not track what it has already fetched and what it hasn't.

You'd need to add uidl to the poll ... line in your .fetchmailrc and a location for the idfile (optional)

set idfile <path relative to FETCHMAILHOME which is by default HOME. If you don't set either it would be ~/.fetchids>

poll .... uidl

fetchmail will track the message IDs it has already downloaded and won't fetch duplicates.

Also, with the above configuration a POP3 client would flag the messages for deletion on the server, which means that even without accounting for duplicates, you should not receive the same email twice, but this seems to be overrideable in Gmail in the "POP/IMAP" settings for the inbox and Gmail may keep the message anyway.

You may want to add keep to the user config line to be sure your client won't ask the server to purge the messages.

user ... keep

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment