curl commands to query imap servers

curl_imap_query_commands.md

curl commands to query imap servers

Based on https://busylog.net/telnet-imap-commands-note/

curl options

• -k -- don't verify certificate (optional)
• -n -- use .netrc for username and password (optional)
• -X -- request to send to server

curl -n imaps://mail.server.tld -X 'imap command'

.netrc format

machine mail.server.tld login username password supersecretpw

machine mail.server.tld
    login username
    password supersecretpw

url format

• tls -- imaps://
• plain -- imap://

Note: enclose url in quotes if your user id or password have shell characters

Examples

get count of message in mailbox INBOX

curl -n imaps://mail.server.tld -X 'STATUS INBOX (MESSAGES)'
* STATUS INBOX (MESSAGES 36)

get count of unseen (unread) message in mailbox INBOX

curl -n imaps://mail.server.tld -X 'STATUS INBOX (UNSEEN)' 
* STATUS INBOX (UNSEEN 20)

get count of unseen (unread) message in mailbox sub-mailbox

curl -n imaps://mail.server.tld -X 'STATUS Archive.2016 (MESSAGES)' 
* STATUS Archive.2016 (MESSAGES 3069)

get count of recent messages in mailbox INBOX (received but not listed)

curl -n imaps://mail.server.tld -X 'STATUS INBOX (RECENT)'   
* STATUS INBOX (RECENT 0)

list all mailboxes/folders

curl -n imaps://mail.server.tld -X 'LIST "" "*"'                     
* LIST (\HasNoChildren \Drafts) "." Drafts
* LIST (\HasChildren) "." Archive
* LIST (\HasNoChildren) "." Archive.2013
* LIST (\HasNoChildren) "." Archive.2014
* LIST (\HasNoChildren) "." Archive.2015
* LIST (\HasNoChildren) "." Archive.2016
* LIST (\HasChildren) "." Archive.old_account
* LIST (\HasNoChildren) "." Archive.old_account.sent
* LIST (\HasNoChildren \Junk) "." Junk
* LIST (\HasNoChildren \Sent) "." "Sent Messages"
* LIST (\HasNoChildren \Trash) "." Trash
* LIST (\HasNoChildren) "." INBOX

fetch mail

curl -n 'imaps://mail.server.tld/INBOX;UID=1'
# fetch gmail Drafts mailbox
curl -n 'imaps://imap.gmail.com:993/%5BGmail%5D/Drafts;UID=1'
# fetch gmail Sent Mail mailbox
curl -n 'imaps://imap.gmail.com:993/%5BGmail%5D/Sent%20Mail;UID=1'

Alternative to -n/--netrc: -u USER:PASSWORD. More curl examples: http://server1.sharewiz.net/doku.php?id=curl:perform_imap_queries_using_curl

Nice write up. Here are some additions:

Fill array with mail ids:

IFS=" " read -r -a mail_ids <<< "$(curl --silent --show-error --user $imap_login:$imap_pass -X "SEARCH ALL" --url "imaps://$imap_host:$imap_port/INBOX" | grep -oP "(?<=SEARCH)[ 0-9]+")"

Download mail to file:

curl --silent --show-error --user $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/INBOX;UID=$mail_id" > "$mail_file"

Obtain boundary (a little bit dirty because of -A1, but it works):

# obtain boundary (https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html)
grep -A1 '^Content-Type: multipart/mixed' "$mail_file"
mail_boundary=$(grep -A1 '^Content-Type: multipart/mixed' "$mail_file" | grep -oP '(?<=boundary=")[^"]+')

Now use dos2unix to remove windows line breaks (\r) and split mail parts with boundary

csplit --silent --prefix="$mail_file." --suffix-format='%03d.part' "$mail_file" "/^--$mail_boundary/" "{*}"

Now while looping through the mail parts to find attachments. Obtain filename:

mail_attachment_file=$(grep -oP '(?<=Content-Disposition: attachment; filename=")[^"]+' "$mail_part_file")

Extract attachment file content:

grep -Pzo "(?s)(?<=Content-Transfer-Encoding: base64\n\n).*?(?=\n\n)" "$mail_part_file" | tr -d '\n' | base64 --decode > "$mail_file-$mail_attachment_file"

And some more:

 # copy mail to trash
curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/INBOX" -X "COPY $mail_id \"/Gel&APY-schte Elemente\""

# flag mail for deletion
curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/INBOX" -X "STORE $mail_id +FLAGS (\\Deleted)" >/dev/null

# delete flagged mails (must be outside of a loop as the IMAP server re-assigns mail ids on every deletion!)
curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/INBOX" -X "EXPUNGE" 

Another interesting fact:
--login-options AUTH=NTLM is needed in recent versions of curl as the default for "user@server" seem to became AUTH=GSSAPI which returns "gss_init_sec_context() failed: No Kerberos credentials available" if you don't use Kerberos authentification.

This one wasn't easy. I wanted to remove all mails which are older than 90 days from the trash bin. After several tries I found out that the folder name returned bei LIST is "Gel&APY-schte Elemente" and to use it in --url it must be urlencoded as follows:

Gel%26APY-schte%20Elemente

This is the final code:

# clean up trash mails
IFS=" " read -r -a mail_ids <<< "$(curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/Gel%26APY-schte%20Elemente" -X "SEARCH BEFORE $(date +'%d-%b-%Y' --date='90 days ago')"  | grep -oP "(?<=SEARCH)[ 0-9]+")"
for (( i=0; i<${#mail_ids[@]}; i++ )); do
  mail_id="${mail_ids[$i]}"
  curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/Gel%26APY-schte%20Elemente" -X "STORE $mail_id +FLAGS (\\Deleted)" >/dev/null
done
[[ ${#mail_ids[@]} -gt 0 ]] && curl -sS --login-options AUTH=NTLM -u $imap_login:$imap_pass --url "imaps://$imap_host:$imap_port/Gel%26APY-schte%20Elemente" -X "EXPUNGE" >/dev/null

Attention: %b of the date command could return a localized string. To force english output you should add export LC_ALL=C to the top of your script.