When I first set up Listmonk it was to use with Amazon SES. At the time Amazon would give you free 62,000 emails/mo if you sent them from an EC2 instance. So EC2 was the best server to use. In mid 2023 Amazon ended that, so now you can use whatever server you like, which makes things much easier. It shouldn't be too hard to convert these directions to another server host of your choice.
I used Hetzner with another build, and once my free EC2 year ended the AWS t2.micro cost me $14/mo. Hetzner has better specs and costs me $5/mo, so I added an nginx vhost and moved listmonk to the same server. Here's a $20 credit for Hetzner.
There is also the possibility to use the 1-click installers for their featured hosts: https://listmonk.app/ - listed under "Hosting providers". I'm not familiar with any of them but there are lots of new guides now: https://listmonk.app/docs/installation/#tutorials
If you're using Hetzner (other hosts do this too) you have to wait a month and then request that they unblock the SMTP ports. Or use port 587.
I am a total noob and it was a massive pain in the ass to get this setup because there wasn't a detailed guide at the time. So I created this to help others. The Listmonk docs were generally pretty poor and not informative. It's an open source project so feel free to help improve it (and this guide). I moved parts of this tutorial to the official docs.
- Directions for EC2 micro install using docker
- Upgrading listmonk:
- Downgrading listmonk:
- Logs:
- Cloudflare:
- Wordpress:
- PostgreSQL:
- Installing listmonk without docker (via binary)
- Other places to find solutions:
We'll use a combination of these two links:
- https://levelup.gitconnected.com/setting-up-your-own-bulk-mail-server-using-sendy-and-aws-20d05ee01362
- https://vpsfix.com/7624/install-sendy-aws-ec2-instance-ssl/
Generally the first one, but sometimes the 2nd one will go into more detail.
Putty and FileZilla will be your main tools on Windows. Portableapps.com has both of them.
DigitalOcean has some nice guides. For example, their How To Install PostgreSQL on Ubuntu 22.04, and another for docker linked below.
- Create security group, as directed, except use port 9000 instead of 10000.
- Create EC2 instance with latest version of Ubuntu Server. t2.micro is fine.
- Set up key pair to login via SSH using Putty (don't bother with Windows 10 native option). https://vpsfix.com/3052/key-pair-login-amazon-ec2-instance/
- Set up elastic IP
- Add Elastic IP to your domain’s DNS records
- Connect to EC2 via SSH
- Update the OS. Can run
dpkg-reconfigure tzdatato change timezone. Andsudo apt autoremoveto remove unnecessary packages. - Run these commands (first, see Docker update below):
sudo apt install postgresql docker docker-compose
mkdir listmonk && cd listmonk
sudo sh -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-prod.sh)"
Note: do not use "sudo -i". It will install things into the wrong directory. If you don't want to type "sudo" all the time, "sudo bash" might work. But for these, copy-paste is fine.
EDIT: I actually removed sudo from many lines below, as it was unnecessary and possibly caused by erroneously putting sudo in front of mkdir listmonk to start with. If you have to put sudo in front of any lines below, you probably messed something up. These things are finicky, and commands have to be so precise or you can run into so many problems. Another reason a novice like me should not be having to write this guide.
Lesson: don't use sudo unnecessarily. Especially when creating directories and files. Since it sets the permissions of them to root.
If it won't load (like in my case), wipe your EC2 drive and try again:
Visit your EC2 dashboard -> Instances -> Select instance -> top right; "actions" -> monitor and troubleshoot -> replace root volume.
Still doesn't load (like in my case)? Wipe the drive and try again.
Still doesn't load (like in my case)? Get a linux expert to mess with it for a few hours. Maybe try these commands:
View running containers: sudo docker container ls
Restart containers:
sudo docker-compose restart app db
sudo docker container stop listmonk_db listmonk_app
sudo docker container rm listmonk_db listmonk_app
sudo docker-compose up -d app db
Then wipe the drive and try again. Trust me, it magically works the 3rd time.
The docker-compose command is being depreciated:
- https://www.docker.com/blog/new-docker-compose-v2-and-v1-deprecation/
- "Type
docker composeinstead ofdocker-composein your favorite terminal." https://docs.docker.com/compose/migrate/ apt install postgresql docker docker-composemay be changed toapt install postgresql docker docker-compose-pluginhttps://docs.docker.com/compose/install/linux/ orsudo apt install postgresql docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-pluginhttps://docs.docker.com/engine/install/ubuntu/#install-docker-engine.- To upgrade, you have to uninstall the old packages first https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions
- How To Install and Use Docker on Ubuntu 22.04 https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-22-04
Per the link above, and this discussion https://serverfault.com/questions/1110365/when-installing-docker-on-ubuntu-why-isnt-it-as-easy-as-apt-get-install-docker, it seems that the apt install docker docker-compose command installs an older version of docker from ubuntu. If you want to use the latest version you have to do the extra steps in the links above.
Since listmonk is updating their scripts & docs to the latest version of docker knadh/listmonk#1431 we should too.
Additionally, a link above has this tip:
Receiving errors when trying to run without root?
The docker user group exists but contains no users, which is why you’re required to use sudo to run Docker commands. Continue to Linux postinstall to allow non-privileged users to run Docker commands and for other optional configuration steps.
nano ./docker-compose.yml
nano ./config.toml
Or use FileZilla:
Host: elastic IP
Port: blank
User: ubuntu
PW: key file
Protocol: SFTP SSH file transfer
Note that if you use FileZilla with an editor like Notepad++ you will need to make sure that newly created files are using LF line breaks instead of CRLF. If you create the file with nano it does it already.
After you edit those files (IE: changing admin password) run this restart command:
sudo docker-compose restart app db
The postgres passwords in those files are read-only. If you want to change the postgres password use this command:
psql postgresql://listmonk:CURRENT_PASSWORD@127.0.0.1:9432/listmonk
or
psql -U listmonk -h localhost -p 9432 listmonk
then
ALTER USER listmonk PASSWORD 'NEW_PASSWORD';
Eg: listmonk.mysite.com
The below instructions use nginx and are a bit more complicated. You can alternatively use Caddy, which is easier.
https://yasoob.me/posts/setting-up-listmonk-opensource-newsletter-mailing/#setting-up-ssl
First copy the contents of the "nginx.conf" file and paste into text file. Replace (ctrl+h) all 4 instances of "example.com" with the subdomain hostname from the previous step -- IE: listmonk.mysite.com.
Go back to cmd:
mkdir -p data/nginx
cd data/nginx
nano ./nginx.conf
Paste in your edited nginx.conf. Save and exit.
Edit the "init-letsencrypt.sh" in a text file.
Go back to cmd:
cd
nano ./init-letsencrypt.sh
paste in the contents of the "init-letsencrypt.sh" text file. Save and exit.
Once it didn't paste in correctly and caused an error, so I had to edit the file with FileZilla instead.
If you've updated to the latest version of docker (see above) you'll need to replace all 7 instances of docker-compose with docker compose.
nano ./docker-compose.yml
paste in the NGINX and certbot lines.
Add this extra restart: unless-stopped line under certbot:
certbot:
image: certbot/certbot
restart: unless-stopped
Save and exit. Then run:
sudo bash ./init-letsencrypt.sh
sudo docker start certbot
sudo docker-compose stop ; sudo docker-compose up -d
It should be auto-renewing. If it doesn't, check to see if the certbot container is running:
sudo docker container ls.
You can check the logs with sudo docker logs certbot -t.
If you have trouble, here's a discussion about Certbot not restarting after rebooting the server: wmnnd/nginx-certbot#9 (comment)
https://listmonk.app/docs/configuration/#time-zone
Discussion: knadh/listmonk#637
SMTP credentials: The instructions in the first link for "Setting up Amazon SES" by creating a user with credentials won't work. You specifically have to create a new SMTP user knadh/listmonk#1053
STARTTLS port 587 vs TLS/SSL port 465 knadh/listmonk#1615
"HELO is SMTP specific. If your SMTP server does not ask for it, it should be left blank" knadh/listmonk#448
https://listmonk.app/docs/bounces/#external-webhooks
You can use Amazon's test emails to make sure everything's working: https://listmonk.app/docs/bounces/#verification
In the SES console, it doesn't seem like you have to verify every email you want to send from, in addition to the domain. It seems that I can send with anyemailiwant@myses-verifieddomain.com but I don't get email notifications without verifying the email and enabling "Email feedback forwarding".
It seems like you only have to enable "Feedback notifications" for the sender domain, rather than every email you want to send from.
Discussion: knadh/listmonk#166 (comment)
Exporting bounces: knadh/listmonk#863
https://listmonk.app/docs/configuration/#media-uploads
Solution: knadh/listmonk#1169 (comment)
Other discussions: knadh/listmonk#1163 (comment)
https://yasoob.me/posts/setting-up-listmonk-opensource-newsletter-mailing/#custom-static-files
You can do:
# cd to the listmonk directory with the config.toml file
cd listmonk
mkdir static && cd static
wget -O - https://github.com/knadh/listmonk/archive/master.tar.gz | tar xz --strip=2 "listmonk-master/static"
or combined in one command:
mkdir -p /home/ubuntu/listmonk/static ; wget -O - https://github.com/knadh/listmonk/archive/master.tar.gz | tar xz -C /home/ubuntu/listmonk/static --strip=2 "listmonk-master/static"
Then edit the docker-compose.yml file as shown in the yasoob link above.
You may have to pull the files from a different branch: knadh/listmonk#1708 (comment)
sudo apt update && sudo apt upgrade -y
I was using:
sudo docker-compose restart app db nginx certbot
But it seems to often not be sufficient, so I switched to:
sudo docker-compose stop ; sudo docker-compose up -d
https://listmonk.app/docs/upgrade/
Or, per https://github.com/knadh/listmonk/releases/tag/v2.5.0
# cd /directory/with/docker-compose.yml
# In our case that's already our home directory, so we can skip it.
sudo docker-compose down
sudo docker-compose pull && sudo docker-compose run --rm app ./listmonk --upgrade
sudo docker-compose up -d app db nginx certbot
https://listmonk.app/docs/upgrade/#downgrade
Discussion: knadh/listmonk#1457
Here's a FOSS application that can be used to log the email addresses of people who received your campaign:
SES Dashboard https://sesdashboard.com/ - https://github.com/Nikeev/sesdashboard
You can check certbot logs with sudo docker logs certbot -t. You can switch out certbot with the other containers.
docker logs --help for more info. https://stackoverflow.com/a/47829441/5862615
More info: knadh/listmonk#13 (comment)
knadh/listmonk#1462 -- to clarify, stdout is not saved to a file by default, so if you want to save the logs to a file you'll need to use ./listmonk > log.txt.
There are various options for the service file depending on your systemd version:
- https://stackoverflow.com/questions/37585758/how-to-redirect-output-of-systemd-service-to-a-file
- https://unix.stackexchange.com/questions/321709/redirect-systemd-service-logs-to-file
To append, use one of these:
ExecStart=/bin/sh -c 'exec /usr/bin/listmonk [arguments] >>/etc/listmonk/listmonk1.log 2>>/etc/listmonk/listmonk2.log'
or
ExecStart=/bin/sh -c '/usr/bin/listmonk --config /etc/listmonk/config.toml --static-dir /etc/listmonk/static 2>&1 > /etc/listmonk/listmonk.log'
or
ExecStart=/bin/bash -ce "exec /usr/bin/listmonk --config /etc/listmonk/config.toml --static-dir /etc/listmonk/static >>/etc/listmonk/listmonk.log 2>&1"
On CentOS 7 I'm using the 3rd one.
https://betterstack.com/community/guides/logging/how-to-start-logging-with-postgresql/
Login with the superuser account via sudo -i -u postgres. Then SHOW data_directory;.
You can use Cloudflare's free proxy to protect your server's IP address from DDoS attacks https://gist.github.com/MaximilianKohler/3bdedd0185283ac30c1f1422f9626947#ddos
Since we already have http to https redirects you'll need to set Cloudflare's "Encryption mode" to "Full or Full (strict)" to avoid the "too many redirects" error https://developers.cloudflare.com/ssl/troubleshooting/too-many-redirects/
I wondered if there would be any issues with putting a CDN in front of an email server. IE: I wondered if people might get captchas when clicking "unsubscribe" and thus be more likely to mark emails as spam. I also wonder if it may mess with statistics such as open rates. I looked into it and learned that Cloudflare is now using a non-interactive captcha method that drastically reduces the amount of puzzles that people have to solve. But see below for info on decreasing or turning off captcha.
- Best Practices For Captcha Challenges https://community.cloudflare.com/t/community-tip-best-practices-for-captcha-challenges/56301
- Ease up on challenge frequency (challenge passage setting) https://community.cloudflare.com/t/ease-up-on-challenge-frequency/60211/2
How to turn off captcha:
- Lower your security level to “Essentially off” https://community.cloudflare.com/t/removing-cloudflare-captcha/191436
- Didn't work for this guy https://community.cloudflare.com/t/how-to-turn-off-captcha/305523/13
- or create a firewall rule to ALLOW ALL from 0.0.0.0/0 https://community.cloudflare.com/t/how-to-stop-and-disable-stupid-hcaptcha-from-our-cloudflare-site/160194/6
- Check "Browser Integrity Check" too https://community.cloudflare.com/t/how-to-turn-off-cloudflare-captcha/305251/7
If you're not proxying your domain/subdomain with cloudflare, you could manually set up listmonk to use Turnstile, which can be just one click or none. Migrating to Turnstile. You'll have to edit your HTML’s <head> element. I haven't bothered trying that yet knadh/listmonk#1617.
Open source plugin for WordPress / WooCommerce users to easily send subscribers through payment or a web form to listmonk: https://github.com/post-duif/integration-listmonk-wordpress-plugin
Example of some basic psql commands: https://yasoob.me/posts/setting-up-listmonk-opensource-newsletter-mailing/#importing-old-subscribers
You can copy-paste multiple lines into the CMD prompt. Postgres will execute them once a semicolon is passed.
Here's an example of how to collect and insert data into the postgres db: knadh/listmonk#1629 (comment)
Show orphans: knadh/listmonk#1562
Advanced SQL searches from listmonk's UI: knadh/listmonk#1458 (comment)
Go to Subscribers list, click on "Advanced" button to use SQL query, and put query like this:
EXISTS(SELECT 1 FROM campaign_views WHERE campaign_views.subscriber_id=subscribers.id AND campaign_views.campaign_id=<put_id_of_campaign>)
Yet again, on the Upgrade page https://listmonk.app/docs/upgrade/ it's mentioned:
it is recommended to take a backup of the Postgres database before running the --upgrade option
but commands/method to do so isn't mentioned...
Confirmed this works:
pg_dump -U listmonk -h localhost -p 9432 listmonk > pglistmonkdb.sql - uncompressed
or:
pg_dump -U listmonk -h localhost -p 9432 listmonk -Fc > listmonk-$(date +"%Y-%m-%d---%H-%M-%S").sql - built-in compression
or:
pg_dump -U listmonk -h localhost -p 9432 listmonk | gzip > listmonk-$(date +"%Y-%m-%d--%H-%M-%S").sql.gz - compressed with gzip
Then connect to EC2 with FileZilla to download the file.
You may also want to run this command to secure the file, but I'm not sure if it's necessary: chmod 0640 /home/ubuntu/listmonk-2023*
The pro of using gzip is that you can unzip it after downloading and open in plain text with notepad++. Whereas the -Fc one is gibberish in notepad++. Though notepad++ is not the best program to use for this anyway. Let me know if there's a better program.
It sounds like there's a way to restore and backup directly from/to your PC and your server, but maybe someone else can clarify the exact commands. https://stackoverflow.com/questions/26378098/restore-dump-on-the-remote-machine
You might be able to install postgres on your PC https://www.youtube.com/watch?v=fAOBqgR9Yfo and run modified versions of the above and below commands (using your elastic IP instead of "localhost" and "127.0.0.1", and might need to change the ports too).
When I tried psql -h elasticIP -p 5432 -d listmonk -U listmonk -W -f pglistmonkdb.sql I got a timeout error.
Be careful, this will completely delete/erase your existing DB. You may want to do a web search for drop schema and/or mysql commands to familiarize yourself with the commands.
psql -h 127.0.0.1 -p 9432 -U listmonk
drop schema public cascade;
create schema public;
\q
psql -h 127.0.0.1 -p 9432 -U listmonk -W listmonk < pglistmonkdb.sql
sudo docker-compose restart app db nginx certbot
If the file is gzip'd you can run gunzip filename.sql.gz which changes the size and file type of the existing file (doesn't create a new one). You can reverse it with gzip filename.sql.
Alternatively, there are other commands pg_restore and "restore from gzip" that I haven't figured out exactly yet. But they can be avoided by using the non-gzip backup option above.
Not working:
psql listmonk < pglistmonkdb.sql --clean
psql -U listmonk listmonk < pglistmonkdb.sql
psql -U listmonk -W listmonk < pglistmonkdb.sql
I did it on a CentOS7 nginx vhost because the firewall was blocking docker from running.
I also added more links with more info here: https://listmonk.app/docs/installation/#tutorials
These commands told me I had an AMD x64 CPU:
uname -m
cat /proc/cpuinfo
So I downloaded the linux_AMD64 binary from the releases page. Uploaded (any directory, I picked "public") via FileZilla.
Grant execute permissions of the listmonk binary to "owner". 744 via FileZilla or chmod +x /path-to-listmonk/listmonk.
Timezone is set via host machine. On CentOS7 it's timedatectl to check. Then choose one: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones. Then set with timedatectl set-timezone America/Los_Angeles. System logs won't use the new time zone until you reboot the server.
You have to setup listmonk as a service to keep it running: knadh/listmonk#843
This guide mostly worked for me, but I had to make some modifications. And in addition to grant all privileges on database listmonk to listmonk; I had to run ALTER DATABASE listmonk OWNER TO listmonk;. I created a simplified version of the service file with greater compatibility for older OSs.
You can pair it with a simple nginx config. In my case, port 9000 is used by php-fpm, so I had to change it (in config.toml and proxy_pass) to something else, such as 9003.
If you leave the DB port on 5432 you'll have to manually change most of the commands above that use 9432. Or you could just change your config.toml to port = 9432 before running the install command ./listmonk --install.
If you're importing an existing database you probably have to restart the service after. Basic commands:
nano /etc/systemd/system/listmonk.service
nano /etc/listmonk/config.toml
systemctl daemon-reload && systemctl restart listmonk
systemctl enable listmonk.service
systemctl start listmonk.service
systemctl status listmonk
systemctl stop listmonk
I'm using this backup.sh file.
How to stop listmonk for upgrade knadh/listmonk#1113
If you can't find an answer in the listmonk issues you may be able to find it on the Mautic issues https://github.com/mautic/mautic/issues or their forum.
Good luck.
Thank you for this tutorial! I'm a noob at AWS and listmonk, but managed to install listmonk on an EC2 instance thanks to these instructions.
A couple of details I figured out while executing the steps, which others also might find helpful:
dockerandpostgresqlseparately. And I used your link to Digital Ocean's instructions on how to install docker.Unable to locate packageerror fordocker. I got the same error fordocker-compose-pluginwhen I tried to follow the instructions in docker docs for installing using the repository.