martycagas/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Better dovecot mail download over IMAP

I have certainly violated this xkcd at this point...
A wrapper script around the Dovecot's doveadm backup utility, that interactively downloads emails from another IMAP server.
Useful when the other server does not support master users. If it does, there's a much easier way - it's not needed to download mailboxes per-user and have them enter the password manually. But some really old IMAP servers have no idea of master users and that's why this script exists.
Dependencies


The Dovecot IMAP server (provides the doveadm utility)
GNU bash, version >= 5.1.8 (this is the version I wrote and used this script on, the real requirement is most likely lower)
A prepared migration configuration file

The migration configuration file

Dovecot's official documentation on mailbox migration mentions the following to be included in your migration config:

Mail user, home and location settings
Namespace definitions, including public and shared namespaces
ACL settings
Quota without rules (to make sure quota gets calculated, but not enforced)
Mail caching settings
Attachment detection settings
Mailbox attribute settings
Compression and encryption settings
NFS related settings

The items in bold text (emphasis mine), are what usually needs to be set up as a baseline. The rest can probably be ignored and is more of a matter of compatibility with your current setup. The easiest way to get a configuration file identical to your production config is redirecting the output of doveconf -n to a file, then combing through it to remove the unnecessary settings.
The script assumes a migration configuration file in /etc/dovecot/dovecot-migration.conf
In addition, some additional configuration for the Dovecot's IMAP client is needed. This configuration won't be a part of the doveconf -n output, as the IMAP client is not usually used in production environments[citation needed].
ssl = required

imapc_features = rfc822.size fetch-headers

imapc_ssl = imaps
imapc_port = 993

# These are much preferable to setting verify to no,
# but might be troublesome to set up
#ssl_client_ca_dir = /etc/ssl/certs
# or
#ssl_client_ca_file = /etc/ssl/ca-certificates.pem

# Unsafe, but possible
# Another reason why we want a secondary configuration file for migration
ssl_client_require_valid_cert = no

In addition, the imapc_host, imapc_user and imapc_password are entered using the -o flags as part of the command.
Usage

Make sure the script has the executable permission:
# chmod u=rwx,go= ./mailbox-migration-utils.sh
There's no need to consider group and world permissions, because the doveadm utility needs to be invoked as a root.
If at any point you're unsure about the usage, run the script with --help as the first argument.
# ./mailbox-migration-utils.sh --help
Available commands:
    ./mailbox-migration-utils.sh download ... [-h|--help]
    ./mailbox-migration-utils.sh sync ... [-h|--help]
    ./mailbox-migration-utils.sh backup ... [-h|--help]
    ./mailbox-migration-utils.sh --version
    ./mailbox-migration-utils.sh --help
Both download and sync commands are implemented and have been tested in production to work reliably.
download

# ./mailbox-migration-utils.sh download -u user -r remotehost [--batch-mode] [--help]
This will attempt to download email of the user@remotehost and store it locally as user using the doveadm backup -R call. The download protocol (IMAP/IMAPS) has to be manually specified in the /etc/dovecot/dovecot-migration.conf file.
The command also accepts the optional --batch-mode argument, which suppresses the initial warning about potentially overwriting mailboxes.
sync

# ./mailbox-migration-utils.sh sync -u user -r remotehost [--help]
The sync command is very similar to the download command, but it instead uses the doveadm sync -R1 call. This means the remote mailbox will be one-way merged into the local mailbox. The call of the command's synopsis is the same as that of the download command. The download protocol (IMAP/IMAPS) has to be manually specified in the /etc/dovecot/dovecot-migration.conf file.
Security caveat

This script doesn't expose passwords directly to users, nor it stores them into files. The passwords won't show up in the shell history but may be logged depending on the system setting. During the download, the password is visible for any user with sufficient permissions to list the running process.
Extra useful tools


shfmt

Used to parse, syntactically check and format shell scripts.
This code was formatted using shfmt -i 4 -l -w ./mailbox-migration-utils.sh.
Can be simply installed with...

... on RHEL-based distributions: sudo dnf install shfmt.
... on Debian-based distributions: sudo apt install shfmt.


shellcheck

Used to statically analyse shell code.
This code was checked using shellcheck ./mailbox-migration-utils.sh (even though some warnings are present as they're intentional).
Can be simply installed with...

... on RHEL-based distributions: sudo dnf install ShellCheck.
... on Debian-based distributions: sudo apt install shellcheck.


I just really like those two tools which greatly helped me so far and wanted to give them a little bit of the appreciation they deserve.
License

MIT License
Copyright (c) 2022 Martin Cagas
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

  
## mailbox-migration-utils.sh
#!/usr/bin/env bash

# NAME
#   Better Mail Sync Wrapper for Dovecot
#
# SYNOPSIS
#   ./mailbox-migration-utils.sh download -u|--user user -r|--remote-host host [--batch-mode]
#   ./mailbox-migration-utils.sh download [-h|--help]
#   ./mailbox-migration-utils.sh sync -u|--user user -r|--remote-host host
#   ./mailbox-migration-utils.sh sync [-h|--help]
#   ./mailbox-migration-utils.sh --version
#   ./mailbox-migration-utils.sh --help
#
# DESCRIPTION
#   A wrapper script around the 'doveadm' utility intended for easy download and sync
#   of user emails from a remote IMAP mailbox to the local Dovecot mailbox and for
#   backing up local mail to a local destination.
#
# AUTHOR
#   Martin Cagas
#
# EXIT STATUS
#   0       Synchronisation completed successfully.
#   1       Error while parsing arguments.
#   2       File system access error.
#   10      User not found in the local Dovecot user database.
#   11      The person running this script chose to interrupt the synchronisation.
#
# CHANGELOG
#   version 1.2.0
#       - Added changelog :)
#       - Added the working sync command
#       - Improved and simplified the doveadm calls (now uses the imapc_host to find the remote)
#       - Removed the -c|--command flag again
#       - Additional bug fixes
#   version 1.1.3
#       - Added the initial implementation of the not-yet-working 'sync' command
#       - Added the -c|--command flag
#   version 1.1.2
#       - Added the version flag
#   version 1.1.1
#       - Now with a lot more safety precautions and checks
#
# COPYRIGHT
#   Copyright (c) 2022 Martin Cagas; MIT License <https://choosealicense.com/licenses/mit/>
#   This is a free software: you are free to change and redistribute it.
#   There is NO WARRANTY, to the extent permitted by law.

# We don't want to source this file as it would pollute alias definitions and shell options
# But it's a script - if you want to source it regardless, just delete these commands:
$(return &>/dev/null)
if (($? == 0)); then
    printf "Please don't source this script.\n"
    return 1
fi

# Declare the version
declare -r version="1.2.0"

# Some color text definitions
declare -r red=$'\e[1;31m'
declare -r green=$'\e[1;32m'
declare -r nc=$'\e[0m'

# Function to query yes/no
# 'No' being the default (is matched on empty string)
function prompt_confirmation() {
    while true; do
        read -r -p "${*} [y/N]: " yn
        case $yn in
        [Yy]*) return 0 ;;
        [Nn]* | '') return 1 ;;
        esac
    done
}

# Download a user's mailbox from our old IMAP server using 'doveadm backup -R',
# overwriting the local user's mailbox in the process.
function download() {
    # Print the download command's usage
    function print_usage() {
        printf "Usage:\n"
        printf './mailbox-migration-utils.sh \\\n'
        printf "\tdownload -u|--user user -r|--remote-host host [--batch-mode]\n"
        printf "\tdownload [-h|--help]\n"
    }

    # A function to safely delete the failure message file
    function delete_failure_txt() {
        # The :? here serves as a check for undefined variable
        # If the variable is undefined, fail
        local -r l_file="${log_directory:?}/failure.txt"
        # Checks if the file actually exists before attempting to delete it
        if [[ -f "${l_file}" ]]; then
            rm -f "${l_file}"
            return
        else
            printf "Attempted delete of %s, but the file does not exist!\n" "${l_file}" >&2
            return 1
        fi
    }

    # A function to safely delete the user mailbox file
    function delete_user_mailbox() {
        # The :? here serve as a check for undefined variables
        # If any variable is undefined, fail
        local -r l_user="${user:?}"
        local -r l_maildir="${mail_directory:?}"
        # Check if the user actually exists
        if doveadm user "${l_user}" &>/dev/null; then
            # Check if the mail directory actually exists
            if [[ -d "${l_maildir}" ]]; then
                rm -rf "${l_maildir}"
                return
            else
                printf "Attempted delete of %s, but the directory does not exist!\n" "${l_maildir}" >&2
                return 1
            fi
        else
            printf "Attempted delete of %s's mailbox, but the user does not exist!\n" "${l_user}" >&2
            return 1
        fi
    }

    function download_worker() {
        # Output additional info
        # Since the output of the entire download_worker() is later being redirected into a file,
        # it'll serve as annotations for the logs
        printf "Beginning download for user %s on " "${user}"
        date
        printf "\n"

        local -i exit_status=0

        # Measure the time it takes to sync the mailbox
        local TIMEFORMAT="Time spent processing: %R seconds."
        time {
            doveadm -c /etc/dovecot/dovecot-migration.conf \
                -o imapc_host="${remote_host:?}" \
                -o imapc_user="${user:?}" \
                -o imapc_password="${password:?}" \
                backup -Ru "${user:?}" imapc:

            # Store the 'doveadm' exit status before 'time' overwrites it
            exit_status=${?}

            # Based on 'dsync' exit statuses
            # See 'man dsync' for more info
            if ((exit_status == 0)); then
                printf "Download was done perfectly.\n"
            elif ((exit_status == 2)); then
                printf "Download was done without errors, but some changes couldn't be done, so the mailboxes aren't perfectly synchronised.\n"
                printf "This occurs if one of the mailboxes changes during the syncing.\n"
            elif ((exit_status == 1 || exit_status > 2)); then
                printf "Download failed.\n"
                echo -n "${user:?}" >"${log_directory:?}/failure.txt"
            fi
        }

        # The return the original exit status
        return "${exit_status}"
    }

    local user
    local remote_host
    local batch_mode_flag="false"
    local help_flag="false"

    # Parse the arguments using getopt (enchanced)
    # See 'man 1 getopt' and '/usr/share/doc/util-linux/getopt-example.bash'
    local temp
    temp=$(getopt -o 'u:r:h' --long 'user:,remote-host:,batch-mode,help' -n 'Mailbox Migration Utils' -- "$@")

    if (($? != 0)); then
        printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
        exit 1
    fi

    eval set -- "$temp"
    unset temp

    while true; do
        case "${1}" in
        "-u" | "--user")
            user="${2}"
            shift 2
            continue
            ;;
        "-r" | "--remote-host")
            remote_host="${2}"
            shift 2
            continue
            ;;
        "--batch-mode")
            batch_mode_flag="true"
            shift
            continue
            ;;
        "-h" | "--help")
            help_flag="true"
            shift
            continue
            ;;
        "--")
            shift
            break
            ;;
        *)
            printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
            exit 1
            ;;
        esac
    done

    # Declare the parsed variables read-only
    # This ensures they're never accidentally changed or unset
    declare -r user
    declare -r remote_host

    # If help was set, print usage and exit cleanly
    if [[ "${help_flag}" == "true" ]]; then
        print_usage
        exit 0
    fi

    # If either user or remote host are unspecified, print usage and exit
    if [[ -z "${user}" || -z "${remote_host}" ]]; then
        printf "%s\n" "${red}Missing arguments!${nc}"
        print_usage
        exit 1
    fi

    # If batch mode is set, don't prompt user for confirmation
    if [[ "${batch_mode_flag}" == "false" ]]; then
        printf "%s\n" "${red}IMPORTANT: This command copies the remote mailbox to local, overwriting the ${user}'s local mail in the process!${nc}"
        printf "Did you mean 'sync' or 'backup' instead?\n"
        prompt_confirmation "Please confirm you want to continue with 'download'" || exit 11
    fi

    # Check if the migration configuration file exists
    if [[ ! -f "/etc/dovecot/dovecot-migration.conf" ]]; then
        printf "%s\n" "${red}The migration configuration file does not exist!${nc}" >&2
        printf "Aborting...\n" >&2
        exit 2
    fi

    # Check if doveadm can actually find the local user before going further
    if ! doveadm user "${user}" &>/dev/null; then
        printf "%s\n" "${red}User not found in the local Dovecot user database.${nc}" >&2
        printf "Aborting...\n" >&2
        exit 10
    fi

    # Set to the path to the mailbox and the log directory
    # Make both values read-only
    local -r mail_directory="/var/vmail/${user:?}"
    local -r log_directory="/tmp/mailbox-migration"

    if [[ ! -d "${log_directory}" ]]; then
        printf "The log directory %s does not exist!\n" "${log_directory}"
        printf "Attempting to create it...\n"
        if mkdir "${log_directory}"; then
            printf "Directory created succesfully.\n"
        else
            printf "%s\n" "${red}Could not create the directory.${nc}"
            printf "Aborting...\n"
            exit 2
        fi
    fi

    # Check if past sync failed
    # Warn the user about overwriting logs, if they wish to continue
    if [[ -f "${log_directory}/failure.txt" ]]; then
        printf "%s\n" "${red}The last download failed!${nc}"
        if prompt_confirmation "Are you sure you want to continue? This can permanently overwrite past logs!"; then
            delete_failure_txt
        else
            exit 11
        fi
    fi

    while true; do
        local password

        # Read the password without printfing it back to the shell
        read -rs -p "Password for ${user}: " password
        printf "\n" # Extra newline is required here because read relies on the newline from enter - but echo is disabled

        # Spawn the download_worker function as a background job and immediately disown it
        # This will cause the job to not be terminated in case the current shell gets closed for any reason
        download_worker &>"${log_directory}/${user}.log" &
        disown

        # Wait for the disowned process to finish
        # Play an animation of changing dots to signal the script hasn't frozen
        local -i pid=${!}
        local -i step=1
        local status_string="Copying the mailbox for ${user}"

        while kill -0 "${pid}" &>/dev/null; do
            # Move cursor at the start of the line and delete it
            tput hpa 0 && tput el

            # Print the base string
            printf "%s" "${status_string}"

            # Print the required number of dots
            # This is true, cursed bash black magic... please don't learn from this...
            printf ".%.0s" $(eval "echo {1..${step}}")

            # Increment the counter
            ((step++))

            # The max number of dots can be adjusted by the upper limit of the step variable
            if ((step > 3)); then
                step=1
            fi

            # Sleep for half a second
            sleep 0.5
        done

        # Change the prompt one last time before writing the final status
        tput hpa 0 && tput el

        # Check if the process failed and attempt it again, if so
        if [[ -f "${log_directory}/failure.txt" && $(cat ${log_directory}/failure.txt) == "${user}" ]]; then
            printf "%s\n" "${red}${status_string}... failed!${nc}"
            printf "Possibly an authentication error?\n"

            if prompt_confirmation "Retry?"; then
                # All deletes are done with a plethora of checks
                # Better safe than sorry
                delete_user_mailbox
                delete_failure_txt
            else
                exit 11
            fi
        else
            printf "%s\n" "${green}${status_string}... done!${nc}"
            break
        fi
    done

    # Wrap up with a summary (and an advice how to check the copied mails
    printf "Finished copying the mailbox for %s. Total size: %s\n" "${user}" "$(du -c --si "${mail_directory}" | tail -1 | awk '{print $1}')"
    printf "Please verify the operation with %s.\n" "${green}'du -c --si /var/vmail/${user}'${nc}"

    exit 0
}

## My little programmer's defense of what you'll see ahead:
## Most of you will see the following code is identical and say I should have extracted it into a single code somehow.
## While there are parts that should (and will) be a function - like the wait cycle,
## the rest of the code isn't "logically connected" - i.e. I would say the fact the following code is so similar is due to a coincidence,
## not a logical outcome - like two procedures using the same mathematical function.
## Also it's 10 pm, I'm tired and it's got to be working tomorrow morning.

# Download a user's mailbox from our old IMAP server using 'doveadm sync -R1',
# merging the remote mailbox into the local mailbox
function sync() {
    # Print the download command's usage
    function print_usage() {
        printf "Usage:\n"
        printf './mailbox-migration-utils.sh \\\n'
        printf "\tsync -u|--user user -r|--remote-host host\n"
        printf "\tsync [-h|--help]\n"
    }

    # A function to safely delete the failure message file
    function delete_failure_txt() {
        # The :? here serves as a check for undefined variable
        # If the variable is undefined, fail
        local -r l_file="${log_directory:?}/failure.txt"
        # Checks if the file actually exists before attempting to delete it
        if [[ -f "${l_file}" ]]; then
            rm -f "${l_file}"
            return
        else
            printf "Attempted delete of %s, but the file does not exist!\n" "${l_file}" >&2
            return 1
        fi
    }

    function sync_worker() {
        # Output additional info
        # Since the output of the entire sync_worker() is later being redirected into a file,
        # it'll serve as annotations for the logs
        printf "Beginning sync for user %s on " "${user}"
        date
        printf "\n"

        local -i exit_status=0

        # Measure the time it takes to sync the mailbox
        local TIMEFORMAT="Time spent processing: %R seconds."
        time {
            doveadm -c /etc/dovecot/dovecot-migration.conf \
                -o imapc_host="${remote_host:?}" \
                -o imapc_user="${user:?}" \
                -o imapc_password="${password:?}" \
                sync -R1u "${user:?}" imapc:

            # Store the 'doveadm' exit status before 'time' overwrites it
            exit_status=${?}

            # Based on 'dsync' exit statuses
            # See 'man dsync' for more info
            if ((exit_status == 0)); then
                printf "Synchronisation was done perfectly.\n"
            elif ((exit_status == 2)); then
                printf "Synchronisation was done without errors, but some changes couldn't be done, so the mailboxes aren't perfectly synchronised.\n"
                printf "This occurs if one of the mailboxes changes during the syncing.\n"
                printf "To solve this, run the 'sync' command for the same user again.\n"
            elif ((exit_status == 1 || exit_status > 2)); then
                printf "Synchronisation failed.\n"
                echo -n "${user:?}" >"${log_directory:?}/failure.txt"
            fi
        }

        # The return the original exit status
        return "${exit_status}"
    }

    local user
    local remote_host
    local help_flag="false"

    # Parse the arguments using getopt (enchanced)
    # See 'man 1 getopt' and '/usr/share/doc/util-linux/getopt-example.bash'
    local temp
    temp=$(getopt -o 'u:r:h' --long 'user:,remote-host:,help' -n 'Mailbox Migration Utils' -- "$@")

    if (($? != 0)); then
        printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
        exit 1
    fi

    eval set -- "$temp"
    unset temp

    while true; do
        case "${1}" in
        "-u" | "--user")
            user="${2}"
            shift 2
            continue
            ;;
        "-r" | "--remote-host")
            remote_host="${2}"
            shift 2
            continue
            ;;
        "-h" | "--help")
            help_flag="true"
            shift
            continue
            ;;
        "--")
            shift
            break
            ;;
        *)
            printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
            exit 1
            ;;
        esac
    done

    # Declare the parsed variables read-only
    # This ensures they're never accidentally changed or unset
    declare -r user
    declare -r remote_host

    # If help was set, print usage and exit cleanly
    if [[ "${help_flag}" == "true" ]]; then
        print_usage
        exit 0
    fi

    # If either user or remote_host are unspecified, print usage and exit
    if [[ -z "${user}" || -z "${remote_host}" ]]; then
        printf "%s\n" "${red}Missing arguments!${nc}"
        print_usage
        exit 1
    fi

    # Check if the migration configuration file exists
    if [[ ! -f "/etc/dovecot/dovecot-migration.conf" ]]; then
        printf "%s\n" "${red}The migration configuration file does not exist!${nc}" >&2
        printf "Aborting...\n" >&2
        exit 2
    fi

    # Check if doveadm can actually find the local user before going further
    if ! doveadm user "${user}" &>/dev/null; then
        printf "%s\n" "${red}User not found in the local Dovecot user database.${nc}" >&2
        printf "Aborting...\n" >&2
        exit 10
    fi

    # Set to the path to the mailbox and the log directory
    # Make both values read-only
    local -r mail_directory="/var/vmail/${user:?}"
    local -r log_directory="/tmp/mailbox-migration"

    if [[ ! -d "${log_directory}" ]]; then
        printf "The log directory %s does not exist!\n" "${log_directory}"
        printf "Attempting to create it...\n"
        if mkdir "${log_directory}"; then
            printf "Directory created succesfully.\n"
        else
            printf "%s\n" "${red}Could not create the directory.${nc}"
            printf "Aborting...\n"
            exit 2
        fi
    fi

    # Check if past sync failed
    # Warn the user about overwriting logs, if they wish to continue
    if [[ -f "${log_directory}/failure.txt" ]]; then
        printf "%s\n" "${red}The last synchronisation failed!${nc}"
        if prompt_confirmation "Are you sure you want to continue? This can permanently overwrite past logs!"; then
            delete_failure_txt
        else
            exit 11
        fi
    fi

    while true; do
        local password

        # Read the password without printfing it back to the shell
        read -rs -p "Password for ${user}: " password
        printf "\n" # Extra newline is required here because read relies on the newline from enter - but echo is disabled

        # Spawn the syncworker function as a background job and immediately disown it
        # This will cause the job to not be terminated in case the current shell gets closed for any reason
        sync_worker &>"${log_directory}/${user}.log" &
        disown

        # Wait for the disowned process to finish
        # Play an animation of changing dots to signal the script hasn't frozen
        local -i pid=${!}
        local -i step=1
        local status_string="Syncing mailbox for ${user}"

        while kill -0 "${pid}" &>/dev/null; do
            # Move cursor at the start of the line and delete it
            tput hpa 0 && tput el

            # Print the base string
            printf "%s" "${status_string}"

            # Print the required number of dots
            # This is true, cursed bash black magic... please don't learn from this...
            printf ".%.0s" $(eval "echo {1..${step}}")

            # Increment the counter
            ((step++))

            # The max number of dots can be adjusted by the upper limit of the step variable
            if ((step > 3)); then
                step=1
            fi

            # Sleep for half a second
            sleep 0.5
        done

        # Change the prompt one last time before writing the final status
        tput hpa 0 && tput el

        # Check if the process failed and attempt it again, if so
        if [[ -f "${log_directory}/failure.txt" && $(cat ${log_directory}/failure.txt) == "${user}" ]]; then
            printf "%s\n" "${red}${status_string}... failed!${nc}"
            printf "Possibly an authentication error?\n"

            if prompt_confirmation "Retry?"; then
                # All deletes are done with a plethora of checks
                # Better safe than sorry
                delete_failure_txt
            else
                exit 11
            fi
        else
            printf "%s\n" "${green}${status_string}... done!${nc}"
            break
        fi
    done

    # Wrap up with a summary (and an advice how to check the copied mails
    printf "Finished syncing the mailbox for %s. New size: %s\n" "${user}" "$(du -c --si "${mail_directory}" | tail -1 | awk '{print $1}')"

    exit 0
}

# Backs up a local mailbox to another place
function backup() {
    # Print the backup command's usage
    function print_usage() {
        printf "Usage:\n"
        printf './mailbox-migration-utils.sh \\\n'
        printf "\tbackup %s\n" "${red}Not yet implemented!${nc}"
    }

    exit 1
}

# Prints the top script usage
function print_usage() {
    printf "Available commands:\n"
    printf "\t./mailbox-migration-utils.sh download ... [-h|--help]\n"
    printf "\t./mailbox-migration-utils.sh sync ... [-h|--help]\n"
    printf "\t./mailbox-migration-utils.sh backup ... [-h|--help]\n"
    printf "\t./mailbox-migration-utils.sh --version\n"
    printf "\t./mailbox-migration-utils.sh --help\n"
}

# Prints the program's version
function print_version() {
    printf "Better Mail Sync Wrapper for Dovecot, v%s\n" "${version}"
}

# Declare the command variable read-only
# Populate it with the first argument... this is notoriously unsafe (should use something like getopt),
# but until I figure out how getopt implements subparsers, this will have to do.
declare -r command="${1}"
# Shift away the first argument (so the rest could be passed to the local functions)
shift
# Call the appropriate command
# The 'version' command is hard-coded for now as getopt is struggling with
# subparsers for now (or maybe I'm just doing it wrong)
case "${command}" in
"download")
    download "${@}"
    ;;
"sync")
    sync "${@}"
    ;;
"backup")
    backup "${@}"
    ;;
"version" | "--version")
    print_version
    exit 0
    ;;
"--help")
    print_usage
    exit 0
    ;;
*)
    printf "%s\n" "${red}Unknown command!${nc}"
    print_usage
    exit 1
    ;;
esac
	#!/usr/bin/env bash

	# NAME
	# Better Mail Sync Wrapper for Dovecot
	#
	# SYNOPSIS
	# ./mailbox-migration-utils.sh download -u\|--user user -r\|--remote-host host [--batch-mode]
	# ./mailbox-migration-utils.sh download [-h\|--help]
	# ./mailbox-migration-utils.sh sync -u\|--user user -r\|--remote-host host
	# ./mailbox-migration-utils.sh sync [-h\|--help]
	# ./mailbox-migration-utils.sh --version
	# ./mailbox-migration-utils.sh --help
	#
	# DESCRIPTION
	# A wrapper script around the 'doveadm' utility intended for easy download and sync
	# of user emails from a remote IMAP mailbox to the local Dovecot mailbox and for
	# backing up local mail to a local destination.
	#
	# AUTHOR
	# Martin Cagas
	#
	# EXIT STATUS
	# 0 Synchronisation completed successfully.
	# 1 Error while parsing arguments.
	# 2 File system access error.
	# 10 User not found in the local Dovecot user database.
	# 11 The person running this script chose to interrupt the synchronisation.
	#
	# CHANGELOG
	# version 1.2.0
	# - Added changelog :)
	# - Added the working sync command
	# - Improved and simplified the doveadm calls (now uses the imapc_host to find the remote)
	# - Removed the -c\|--command flag again
	# - Additional bug fixes
	# version 1.1.3
	# - Added the initial implementation of the not-yet-working 'sync' command
	# - Added the -c\|--command flag
	# version 1.1.2
	# - Added the version flag
	# version 1.1.1
	# - Now with a lot more safety precautions and checks
	#
	# COPYRIGHT
	# Copyright (c) 2022 Martin Cagas; MIT License <https://choosealicense.com/licenses/mit/>
	# This is a free software: you are free to change and redistribute it.
	# There is NO WARRANTY, to the extent permitted by law.

	# We don't want to source this file as it would pollute alias definitions and shell options
	# But it's a script - if you want to source it regardless, just delete these commands:
	$(return &>/dev/null)
	if (($? == 0)); then
	printf "Please don't source this script.\n"
	return 1
	fi

	# Declare the version
	declare -r version="1.2.0"

	# Some color text definitions
	declare -r red=$'\e[1;31m'
	declare -r green=$'\e[1;32m'
	declare -r nc=$'\e[0m'

	# Function to query yes/no
	# 'No' being the default (is matched on empty string)
	function prompt_confirmation() {
	while true; do
	read -r -p "${*} [y/N]: " yn
	case $yn in
	[Yy]*) return 0 ;;
	[Nn]* \| '') return 1 ;;
	esac
	done
	}

	# Download a user's mailbox from our old IMAP server using 'doveadm backup -R',
	# overwriting the local user's mailbox in the process.
	function download() {
	# Print the download command's usage
	function print_usage() {
	printf "Usage:\n"
	printf './mailbox-migration-utils.sh \\\n'
	printf "\tdownload -u\|--user user -r\|--remote-host host [--batch-mode]\n"
	printf "\tdownload [-h\|--help]\n"
	}

	# A function to safely delete the failure message file
	function delete_failure_txt() {
	# The :? here serves as a check for undefined variable
	# If the variable is undefined, fail
	local -r l_file="${log_directory:?}/failure.txt"
	# Checks if the file actually exists before attempting to delete it
	if [[ -f "${l_file}" ]]; then
	rm -f "${l_file}"
	return
	else
	printf "Attempted delete of %s, but the file does not exist!\n" "${l_file}" >&2
	return 1
	fi
	}

	# A function to safely delete the user mailbox file
	function delete_user_mailbox() {
	# The :? here serve as a check for undefined variables
	# If any variable is undefined, fail
	local -r l_user="${user:?}"
	local -r l_maildir="${mail_directory:?}"
	# Check if the user actually exists
	if doveadm user "${l_user}" &>/dev/null; then
	# Check if the mail directory actually exists
	if [[ -d "${l_maildir}" ]]; then
	rm -rf "${l_maildir}"
	return
	else
	printf "Attempted delete of %s, but the directory does not exist!\n" "${l_maildir}" >&2
	return 1
	fi
	else
	printf "Attempted delete of %s's mailbox, but the user does not exist!\n" "${l_user}" >&2
	return 1
	fi
	}

	function download_worker() {
	# Output additional info
	# Since the output of the entire download_worker() is later being redirected into a file,
	# it'll serve as annotations for the logs
	printf "Beginning download for user %s on " "${user}"
	date
	printf "\n"

	local -i exit_status=0

	# Measure the time it takes to sync the mailbox
	local TIMEFORMAT="Time spent processing: %R seconds."
	time {
	doveadm -c /etc/dovecot/dovecot-migration.conf \
	-o imapc_host="${remote_host:?}" \
	-o imapc_user="${user:?}" \
	-o imapc_password="${password:?}" \
	backup -Ru "${user:?}" imapc:

	# Store the 'doveadm' exit status before 'time' overwrites it
	exit_status=${?}

	# Based on 'dsync' exit statuses
	# See 'man dsync' for more info
	if ((exit_status == 0)); then
	printf "Download was done perfectly.\n"
	elif ((exit_status == 2)); then
	printf "Download was done without errors, but some changes couldn't be done, so the mailboxes aren't perfectly synchronised.\n"
	printf "This occurs if one of the mailboxes changes during the syncing.\n"
	elif ((exit_status == 1 \|\| exit_status > 2)); then
	printf "Download failed.\n"
	echo -n "${user:?}" >"${log_directory:?}/failure.txt"
	fi
	}

	# The return the original exit status
	return "${exit_status}"
	}

	local user
	local remote_host
	local batch_mode_flag="false"
	local help_flag="false"

	# Parse the arguments using getopt (enchanced)
	# See 'man 1 getopt' and '/usr/share/doc/util-linux/getopt-example.bash'
	local temp
	temp=$(getopt -o 'u:r:h' --long 'user:,remote-host:,batch-mode,help' -n 'Mailbox Migration Utils' -- "$@")

	if (($? != 0)); then
	printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
	exit 1
	fi

	eval set -- "$temp"
	unset temp

	while true; do
	case "${1}" in
	"-u" \| "--user")
	user="${2}"
	shift 2
	continue
	;;
	"-r" \| "--remote-host")
	remote_host="${2}"
	shift 2
	continue
	;;
	"--batch-mode")
	batch_mode_flag="true"
	shift
	continue
	;;
	"-h" \| "--help")
	help_flag="true"
	shift
	continue
	;;
	"--")
	shift
	break
	;;
	*)
	printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
	exit 1
	;;
	esac
	done

	# Declare the parsed variables read-only
	# This ensures they're never accidentally changed or unset
	declare -r user
	declare -r remote_host

	# If help was set, print usage and exit cleanly
	if [[ "${help_flag}" == "true" ]]; then
	print_usage
	exit 0
	fi

	# If either user or remote host are unspecified, print usage and exit
	if [[ -z "${user}" \|\| -z "${remote_host}" ]]; then
	printf "%s\n" "${red}Missing arguments!${nc}"
	print_usage
	exit 1
	fi

	# If batch mode is set, don't prompt user for confirmation
	if [[ "${batch_mode_flag}" == "false" ]]; then
	printf "%s\n" "${red}IMPORTANT: This command copies the remote mailbox to local, overwriting the ${user}'s local mail in the process!${nc}"
	printf "Did you mean 'sync' or 'backup' instead?\n"
	prompt_confirmation "Please confirm you want to continue with 'download'" \|\| exit 11
	fi

	# Check if the migration configuration file exists
	if [[ ! -f "/etc/dovecot/dovecot-migration.conf" ]]; then
	printf "%s\n" "${red}The migration configuration file does not exist!${nc}" >&2
	printf "Aborting...\n" >&2
	exit 2
	fi

	# Check if doveadm can actually find the local user before going further
	if ! doveadm user "${user}" &>/dev/null; then
	printf "%s\n" "${red}User not found in the local Dovecot user database.${nc}" >&2
	printf "Aborting...\n" >&2
	exit 10
	fi

	# Set to the path to the mailbox and the log directory
	# Make both values read-only
	local -r mail_directory="/var/vmail/${user:?}"
	local -r log_directory="/tmp/mailbox-migration"

	if [[ ! -d "${log_directory}" ]]; then
	printf "The log directory %s does not exist!\n" "${log_directory}"
	printf "Attempting to create it...\n"
	if mkdir "${log_directory}"; then
	printf "Directory created succesfully.\n"
	else
	printf "%s\n" "${red}Could not create the directory.${nc}"
	printf "Aborting...\n"
	exit 2
	fi
	fi

	# Check if past sync failed
	# Warn the user about overwriting logs, if they wish to continue
	if [[ -f "${log_directory}/failure.txt" ]]; then
	printf "%s\n" "${red}The last download failed!${nc}"
	if prompt_confirmation "Are you sure you want to continue? This can permanently overwrite past logs!"; then
	delete_failure_txt
	else
	exit 11
	fi
	fi

	while true; do
	local password

	# Read the password without printfing it back to the shell
	read -rs -p "Password for ${user}: " password
	printf "\n" # Extra newline is required here because read relies on the newline from enter - but echo is disabled

	# Spawn the download_worker function as a background job and immediately disown it
	# This will cause the job to not be terminated in case the current shell gets closed for any reason
	download_worker &>"${log_directory}/${user}.log" &
	disown

	# Wait for the disowned process to finish
	# Play an animation of changing dots to signal the script hasn't frozen
	local -i pid=${!}
	local -i step=1
	local status_string="Copying the mailbox for ${user}"

	while kill -0 "${pid}" &>/dev/null; do
	# Move cursor at the start of the line and delete it
	tput hpa 0 && tput el

	# Print the base string
	printf "%s" "${status_string}"

	# Print the required number of dots
	# This is true, cursed bash black magic... please don't learn from this...
	printf ".%.0s" $(eval "echo {1..${step}}")

	# Increment the counter
	((step++))

	# The max number of dots can be adjusted by the upper limit of the step variable
	if ((step > 3)); then
	step=1
	fi

	# Sleep for half a second
	sleep 0.5
	done

	# Change the prompt one last time before writing the final status
	tput hpa 0 && tput el

	# Check if the process failed and attempt it again, if so
	if [[ -f "${log_directory}/failure.txt" && $(cat ${log_directory}/failure.txt) == "${user}" ]]; then
	printf "%s\n" "${red}${status_string}... failed!${nc}"
	printf "Possibly an authentication error?\n"

	if prompt_confirmation "Retry?"; then
	# All deletes are done with a plethora of checks
	# Better safe than sorry
	delete_user_mailbox
	delete_failure_txt
	else
	exit 11
	fi
	else
	printf "%s\n" "${green}${status_string}... done!${nc}"
	break
	fi
	done

	# Wrap up with a summary (and an advice how to check the copied mails
	printf "Finished copying the mailbox for %s. Total size: %s\n" "${user}" "$(du -c --si "${mail_directory}" \| tail -1 \| awk '{print $1}')"
	printf "Please verify the operation with %s.\n" "${green}'du -c --si /var/vmail/${user}'${nc}"

	exit 0
	}

	## My little programmer's defense of what you'll see ahead:
	## Most of you will see the following code is identical and say I should have extracted it into a single code somehow.
	## While there are parts that should (and will) be a function - like the wait cycle,
	## the rest of the code isn't "logically connected" - i.e. I would say the fact the following code is so similar is due to a coincidence,
	## not a logical outcome - like two procedures using the same mathematical function.
	## Also it's 10 pm, I'm tired and it's got to be working tomorrow morning.

	# Download a user's mailbox from our old IMAP server using 'doveadm sync -R1',
	# merging the remote mailbox into the local mailbox
	function sync() {
	# Print the download command's usage
	function print_usage() {
	printf "Usage:\n"
	printf './mailbox-migration-utils.sh \\\n'
	printf "\tsync -u\|--user user -r\|--remote-host host\n"
	printf "\tsync [-h\|--help]\n"
	}

	# A function to safely delete the failure message file
	function delete_failure_txt() {
	# The :? here serves as a check for undefined variable
	# If the variable is undefined, fail
	local -r l_file="${log_directory:?}/failure.txt"
	# Checks if the file actually exists before attempting to delete it
	if [[ -f "${l_file}" ]]; then
	rm -f "${l_file}"
	return
	else
	printf "Attempted delete of %s, but the file does not exist!\n" "${l_file}" >&2
	return 1
	fi
	}

	function sync_worker() {
	# Output additional info
	# Since the output of the entire sync_worker() is later being redirected into a file,
	# it'll serve as annotations for the logs
	printf "Beginning sync for user %s on " "${user}"
	date
	printf "\n"

	local -i exit_status=0

	# Measure the time it takes to sync the mailbox
	local TIMEFORMAT="Time spent processing: %R seconds."
	time {
	doveadm -c /etc/dovecot/dovecot-migration.conf \
	-o imapc_host="${remote_host:?}" \
	-o imapc_user="${user:?}" \
	-o imapc_password="${password:?}" \
	sync -R1u "${user:?}" imapc:

	# Store the 'doveadm' exit status before 'time' overwrites it
	exit_status=${?}

	# Based on 'dsync' exit statuses
	# See 'man dsync' for more info
	if ((exit_status == 0)); then
	printf "Synchronisation was done perfectly.\n"
	elif ((exit_status == 2)); then
	printf "Synchronisation was done without errors, but some changes couldn't be done, so the mailboxes aren't perfectly synchronised.\n"
	printf "This occurs if one of the mailboxes changes during the syncing.\n"
	printf "To solve this, run the 'sync' command for the same user again.\n"
	elif ((exit_status == 1 \|\| exit_status > 2)); then
	printf "Synchronisation failed.\n"
	echo -n "${user:?}" >"${log_directory:?}/failure.txt"
	fi
	}

	# The return the original exit status
	return "${exit_status}"
	}

	local user
	local remote_host
	local help_flag="false"

	# Parse the arguments using getopt (enchanced)
	# See 'man 1 getopt' and '/usr/share/doc/util-linux/getopt-example.bash'
	local temp
	temp=$(getopt -o 'u:r:h' --long 'user:,remote-host:,help' -n 'Mailbox Migration Utils' -- "$@")

	if (($? != 0)); then
	printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
	exit 1
	fi

	eval set -- "$temp"
	unset temp

	while true; do
	case "${1}" in
	"-u" \| "--user")
	user="${2}"
	shift 2
	continue
	;;
	"-r" \| "--remote-host")
	remote_host="${2}"
	shift 2
	continue
	;;
	"-h" \| "--help")
	help_flag="true"
	shift
	continue
	;;
	"--")
	shift
	break
	;;
	*)
	printf "%s\n" "${red}Internal error while parsing arguments.${nc}" >&2
	exit 1
	;;
	esac
	done

	# Declare the parsed variables read-only
	# This ensures they're never accidentally changed or unset
	declare -r user
	declare -r remote_host

	# If help was set, print usage and exit cleanly
	if [[ "${help_flag}" == "true" ]]; then
	print_usage
	exit 0
	fi

	# If either user or remote_host are unspecified, print usage and exit
	if [[ -z "${user}" \|\| -z "${remote_host}" ]]; then
	printf "%s\n" "${red}Missing arguments!${nc}"
	print_usage
	exit 1
	fi

	# Check if the migration configuration file exists
	if [[ ! -f "/etc/dovecot/dovecot-migration.conf" ]]; then
	printf "%s\n" "${red}The migration configuration file does not exist!${nc}" >&2
	printf "Aborting...\n" >&2
	exit 2
	fi

	# Check if doveadm can actually find the local user before going further
	if ! doveadm user "${user}" &>/dev/null; then
	printf "%s\n" "${red}User not found in the local Dovecot user database.${nc}" >&2
	printf "Aborting...\n" >&2
	exit 10
	fi

	# Set to the path to the mailbox and the log directory
	# Make both values read-only
	local -r mail_directory="/var/vmail/${user:?}"
	local -r log_directory="/tmp/mailbox-migration"

	if [[ ! -d "${log_directory}" ]]; then
	printf "The log directory %s does not exist!\n" "${log_directory}"
	printf "Attempting to create it...\n"
	if mkdir "${log_directory}"; then
	printf "Directory created succesfully.\n"
	else
	printf "%s\n" "${red}Could not create the directory.${nc}"
	printf "Aborting...\n"
	exit 2
	fi
	fi

	# Check if past sync failed
	# Warn the user about overwriting logs, if they wish to continue
	if [[ -f "${log_directory}/failure.txt" ]]; then
	printf "%s\n" "${red}The last synchronisation failed!${nc}"
	if prompt_confirmation "Are you sure you want to continue? This can permanently overwrite past logs!"; then
	delete_failure_txt
	else
	exit 11
	fi
	fi

	while true; do
	local password

	# Read the password without printfing it back to the shell
	read -rs -p "Password for ${user}: " password
	printf "\n" # Extra newline is required here because read relies on the newline from enter - but echo is disabled

	# Spawn the syncworker function as a background job and immediately disown it
	# This will cause the job to not be terminated in case the current shell gets closed for any reason
	sync_worker &>"${log_directory}/${user}.log" &
	disown

	# Wait for the disowned process to finish
	# Play an animation of changing dots to signal the script hasn't frozen
	local -i pid=${!}
	local -i step=1
	local status_string="Syncing mailbox for ${user}"

	while kill -0 "${pid}" &>/dev/null; do
	# Move cursor at the start of the line and delete it
	tput hpa 0 && tput el

	# Print the base string
	printf "%s" "${status_string}"

	# Print the required number of dots
	# This is true, cursed bash black magic... please don't learn from this...
	printf ".%.0s" $(eval "echo {1..${step}}")

	# Increment the counter
	((step++))

	# The max number of dots can be adjusted by the upper limit of the step variable
	if ((step > 3)); then
	step=1
	fi

	# Sleep for half a second
	sleep 0.5
	done

	# Change the prompt one last time before writing the final status
	tput hpa 0 && tput el

	# Check if the process failed and attempt it again, if so
	if [[ -f "${log_directory}/failure.txt" && $(cat ${log_directory}/failure.txt) == "${user}" ]]; then
	printf "%s\n" "${red}${status_string}... failed!${nc}"
	printf "Possibly an authentication error?\n"

	if prompt_confirmation "Retry?"; then
	# All deletes are done with a plethora of checks
	# Better safe than sorry
	delete_failure_txt
	else
	exit 11
	fi
	else
	printf "%s\n" "${green}${status_string}... done!${nc}"
	break
	fi
	done

	# Wrap up with a summary (and an advice how to check the copied mails
	printf "Finished syncing the mailbox for %s. New size: %s\n" "${user}" "$(du -c --si "${mail_directory}" \| tail -1 \| awk '{print $1}')"

	exit 0
	}

	# Backs up a local mailbox to another place
	function backup() {
	# Print the backup command's usage
	function print_usage() {
	printf "Usage:\n"
	printf './mailbox-migration-utils.sh \\\n'
	printf "\tbackup %s\n" "${red}Not yet implemented!${nc}"
	}

	exit 1
	}

	# Prints the top script usage
	function print_usage() {
	printf "Available commands:\n"
	printf "\t./mailbox-migration-utils.sh download ... [-h\|--help]\n"
	printf "\t./mailbox-migration-utils.sh sync ... [-h\|--help]\n"
	printf "\t./mailbox-migration-utils.sh backup ... [-h\|--help]\n"
	printf "\t./mailbox-migration-utils.sh --version\n"
	printf "\t./mailbox-migration-utils.sh --help\n"
	}

	# Prints the program's version
	function print_version() {
	printf "Better Mail Sync Wrapper for Dovecot, v%s\n" "${version}"
	}

	# Declare the command variable read-only
	# Populate it with the first argument... this is notoriously unsafe (should use something like getopt),
	# but until I figure out how getopt implements subparsers, this will have to do.
	declare -r command="${1}"
	# Shift away the first argument (so the rest could be passed to the local functions)
	shift
	# Call the appropriate command
	# The 'version' command is hard-coded for now as getopt is struggling with
	# subparsers for now (or maybe I'm just doing it wrong)
	case "${command}" in
	"download")
	download "${@}"
	;;
	"sync")
	sync "${@}"
	;;
	"backup")
	backup "${@}"
	;;
	"version" \| "--version")
	print_version
	exit 0
	;;
	"--help")
	print_usage
	exit 0
	;;
	*)
	printf "%s\n" "${red}Unknown command!${nc}"
	print_usage
	exit 1
	;;
	esac