Instantly share code, notes, and snippets.

Embed
What would you like to do?
Create your own encrypted chat server with Riot and Matrix

Running your own encrypted chat service with Matrix and Riot

Workshop Instructor:

This workshop is distributed under a CC BY-SA 4.0 license.

What are we doing here?

The goal of this workshop is to teach you how to configure and run your own Matrix/Riot service. By the end of the workshop, you should be able to log into secure chat rooms and invite others to the same server.

Don't panic if you don't finish all of these steps today. You can take these instructions with you and continue/start again at home any time you like.

If you do finish everything within the time we have, try out some of our bonus challenges at the end of the workshop.

What are Matrix and Riot?

Disclaimer: the instructors are not getting paid by anyone or receiving any incentives to say any of this. We're telling you about it because we like it, and because we think you might find it useful.

Matrix and Riot work together to provide a chat service which behaves in a similar way to popular services like Slack. Unlike Slack, however, this tech stack is free and open source. This means you, your team, or your company can easily host your own Matrix servers so that all information that passes through there can remain within the control of your organisation instead of a third party. If you are frequently sharing sensitive things like passwords, internal URLs, or business information as part of your work - this workshop is for you!

With Matrix and Riot, you can run chatbots, integrate Giphy, create new channels for different topics, or start one-to-one conversations. You can also configure audio and video calling. The stack features Slack and IRC integration for those who don’t want to move (there’s always at least one: https://xkcd.com/1782/).

Matrix also provides native support for encrypted chat rooms. If you set it up, chat channels on your server can be end-to-end encrypted. Anyone on your server can verify their fingerprints with each other out-of-channel to make sure that the people in the room are the ones you want there.

Please remember that, as with any encrypted messaging service, the room is only as secure as the people in it. Encryption isn't magic. Don't write anything you wouldn't want to see your name against publicly.

Pre-requisites:

  • A computer
  • Internet connection (BYO or the venue wifi. If you're using the venue wifi, please be considerate.)
  • Basic knowledge of the Linux terminal, SSH, and command line text editors
  • Riot client installed on your laptop or phone (get it at https://riot.im/)

Optional pre-requisites

If you want to take your work home with you, you'll also need to supply:

  • Your own domain name
  • VM running Debian 8 on a cloud service

Installation Guide

DNS settings

Note: if you just want to learn how Matrix and Riot work today and don't want to mess around with DNS config just yet, ask one of the instructors for temporary domain details, and skip straight to "Installing the Matrix server".

Please remember that all temporary workshop domains and VMs will not live long after the workshop; to run your own service in the real world, you'll need to set up your own infrastructure.

Register a domain

If you don't already have a domain you want to use for this, you can buy one through any domain registrar. If you didn’t set one up before the workshop, it’s going to take too much setup time right now to work with it in this session, so go ahead and ask for one of our temporary hostnames and come back to this later.

(If you don’t have a registrar of choice, we recommend serversaurus.com.au - they're awesome local folk who care about privacy. Again, we are not being paid to say that, they’re just nice people.)

Add DNS records to your DNS panel

e.g.:

server01.securechatworkshop.com. | 300    IN    | A    | 1.2.3.4
_matrix._tcp.server01.securechatworkshop.com. |    300 IN | SRV | 10 0 443 server01.securechatworkshop.com.

Installing the Matrix server

The following guide will set up Synapse, which is Matrix's homeserver implementation.

Disclaimer

If you're using one of our VMs, please be considerate. We're hosting you so you can learn, and so we don't have to spend the first hour of this workshop copying a VM onto everyone's laptop with a USB stick. Please do not use our VMs for anything other than completion of the workshop.

Preparing the machine

First: launch your Debian 8 VM, and SSH in. If you didn't bring your own, you can use one of our temp VMs. Ask instructors for details.

You'll need to be a privileged user to run most of these commands. You can do this by running sudo -i.

We would also recommend typing in your commands directly instead of copying and pasting from here, to reduce the risk of encoding errors (and because pasting in commands from a website on the Internet is generally not good practice!).

The Matrix/Synapse package lives in a non-standard repository. We're going to add this repository to our machine's list of sources:

echo 'deb http://ftp.debian.org/debian jessie-backports main' >> /etc/apt/sources.list

And then we're going to make sure the machine knows that the repo is there:

apt-get update && apt-get dist-upgrade -y

Next, we need to install a few packages that will help us later. Our VMs are very barebones by default. Run the following:

apt-get install -y apt-transport-https lsof curl python python-pip
apt-get install -y certbot -t jessie-backports

At this point we need to add a configuration file to ensure that Matrix knows where to read from. The file needs to be called /etc/apt/sources.list.d/matrix.list

Open this up in your favourite text editor. We are an equal opportunity workshop and take no sides in the text editor wars. If you don't have an editor of choice and want some guidance, ask one of the instructors (or the person next to you) for a recommendation.

Inside /etc/apt/sources.list.d/matrix.list, add the following two lines:

deb https://matrix.org/packages/debian/ jessie main
deb-src https://matrix.org/packages/debian/ jessie main

Installing Matrix

With that out of the way, it's time to actually install Matrix. Run the following:

curl https://matrix.org/packages/debian/repo-key.asc | apt-key add -
apt-get update
apt-get install matrix-synapse -y

Warning:

You might get a package error at this point which will cause the matrix-synapse install to fail. If this is the case, we'll install Aptitude (an alternative Debian package manager) to help us resolve the conflict semi-painlessly.

Run apt-get install aptitude.

Install the python-cffi package with Aptitude: aptitude install python-cffi

When Aptitude offers to resolve the conflict, accept the solution which says:

Install the following packages:
1)  python-cffi-backend [1.9.1-2~bpo8+1 (jessie-backports, now) -> 1.4.2-2~bp

This will be the second or third option Aptitude gives you. Choose "no" until you see this solution offered, and continue with this installation of python-cffi.

Once that's done, let's try the Matrix install again:

apt-get install matrix-synapse

If it's still not working, we suggest a sacrifice to the apt-get gods, or asking an instructor or the person next to you for help.

Configuring the Matrix server

The installation process requires some basic config.

You will be asked to provide a hostname for your server. If you didn't configure your own hostname earlier, use the hostname from the handouts. (e.g. server01.securechatworkshop.com)

If asked for reporting anonymous stats, choose ‘no’. Nobody wants that.

Then, start your server:

systemctl start matrix-synapse

Adding encryption support

It's time to encrypt this sucker.

Use certbot to generate a Let's Encrypt certificate. Read more about it here if you're curious: https://certbot.eff.org/

Run: certbot certonly (add the --register-unsafely-without-email flag if you want. Don't use this flag in prod - but for the purposes of this workshop, this is only a temporary server, and you probably don't want the spam).

Choose the "spin up a temporary webserver" option.

Configuring nginx

To make this thing truly HTTPS-ready, we need to configure a reverse proxy. We'll use nginx for this, so install it:

apt-get install nginx -y

Then add the following configuration to /etc/nginx/conf.d/matrix.conf:

server {
    listen 443 ssl;
    server_name matrix.securechatworkshop.com;

    ssl_certificate     /etc/letsencrypt/live/matrix.securechatworkshop.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/matrix.securechatworkshop.com/privkey.pem;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;

    location /_matrix {
        proxy_pass http://localhost:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
    }
}

Make sure you replace matrix.securechatworkshop.com with the relevant server name.

Once that's saved, restart nginx by running: systemctl restart nginx

Fine-tuning Synapse

Add a shared secret to the config file at /etc/matrix-synapse/homeserver.yaml:

registration_shared_secret: <add random characters here, whatever you want your secret to be>

In that same config file, we also need to comment out the following:

#trusted_third_party_id_servers:
#    - matrix.org
#    - vector.im

Synapse caches conversation information in RAM where possible, and will use as much as you give it. For small implementations, (>50 users), you probably only need about 512MB of RAM. You can configure this by adding an environment variable to the following file: /etc/default/matrix-synapse:

SYNAPSE_CACHE_FACTOR 0.02

And to make sure it all takes, restart the service:

systemctl restart matrix-synapse

Register the first Matrix user

One of the things you probably want out of this chat server is to, y'know, chat with people. To do that, we need some user accounts, starting with your own.

Create a new user by running the following, and answering the prompts:

register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost

- New user localpart [root]: {add your name/handle here}
- Password:
- Confirm password:
- Make admin [no]: yes
- Sending registration request…
- Success.

Optional: to save having to register new users via CLI on your server every time, you can enable GUI user registration through the Riot client by editing /etc/matrix-synapse/homeserver.yaml and changing the following setting:

enable_registration: true

Otherwise, to register additional users, run register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost again to manually configure more accounts.

Don't make them all admins!

Time to Riot

Riot is the fancypants front-end client for the server we just set up.

If you don't have it already, you can download the app for your OS of choice at https://riot.im/

One you have it, run it.

Riot may try to auto-connect you to their default servers. If this happens, log out. We want the Riot login screen for the next part.

Let's connect Riot to the server we just configured. Add your hostname (either your BYO hostname, or the here's-what-we-prepared-earlier hostname on your handout):

Home server URL: (e.g. server01.securechatworkshop.com)

Identity server URL: (e.g. server01.securechatworkshop.com)

Now log in with the user you configured in your server in the previous section of this doc.

Create a new secure room

  • Click on the gear icon and turn on end-to-end encryption by ticking ‘Enable encryption’
  • Click ‘Save’

Security checkup

  • Who can access this room? -> Only people who have been invited (default)
  • Who can read history? -> Members only (since they joined)
  • URL previews -> Disable URL previews
  • To invite users in the room -> Moderator

More config

There's a bunch of stuff you can do with the Riot client. Explore the interface.

Here are some things to try:

  • Invite your friends to the room you just created
    • compare key fingerprints before chatting in encrypted rooms
  • Have someone else create a room (or prevent someone else from creating a room)
  • Integrate Giphy for maximum lulz
  • Add a GitHub bot
  • Enable voice and video calling
  • Try the mobile client

Additional things to do after the workshop

There's server config we skipped over because this is a pretty short workshop.

We'd recommend going back to your server and doing some of these things later if you actually want to use Matrix/Riot properly.

  • Deploy a firewall with iptables
  • Configure auto-update (unattended-upgrades)
  • Protect your Debian server with two-factor authentication
  • Replace sqlite database with psql (if you expect lots of users): https://github.com/matrix-org/synapse/blob/master/docs/postgres.rst
  • Configure email notifications (enable_notifs) - beware, may leak sensitive data!
  • Add room integrations
    • GitHub bot
    • RSS bot
    • Giphy
  • Add TURN support for audio/video calls (this is also not 100% secure, use with caution)
  • Set up certificate auto-renewal for Let's Encrypt by running crontab -e and inserting the following line: @daily certbot renew --quiet --post-hook "systemctl reload nginx"

You can find all of the docs you could ever need (and the Matrix community itself) right here: https://matrix.org/docs/guides/faq.html#servers

Post-event infrastructure

To set up a Matrix server on your own infrastructure, you will need to provide your own domain name and your own server. We used AWS EC2 t2.micro instances as the servers for our workshop today - you may need something bigger if you have a large organisation to host.

After you buy a domain, configure your DNS settings as outlined in the "Add DNS records" section above, and then add these into your server's config in place of our temporary ones in the guide. Don't forget to generate another SSL cert through Let's Encrypt for your new hostname.

Make sure to restart Matrix after you're done making config changes, or they won't work properly.

Happy chatting!

Credits

This workshop was originally developed for the B-Sides Canberra 2017 conference by Lilly Ryan and Gabor Szathmari.

@InsaneNaman

This comment has been minimized.

Show comment
Hide comment
@InsaneNaman

InsaneNaman Jun 15, 2017

In user registration, you haven't mentioned the port i.e. localhost:8448 as given on their documentation 😄

InsaneNaman commented Jun 15, 2017

In user registration, you haven't mentioned the port i.e. localhost:8448 as given on their documentation 😄

@rick7661

This comment has been minimized.

Show comment
Hide comment
@rick7661

rick7661 Sep 9, 2017

Great workshop. Thanks ;)

rick7661 commented Sep 9, 2017

Great workshop. Thanks ;)

@viraj2252

This comment has been minimized.

Show comment
Hide comment
@viraj2252

viraj2252 Sep 28, 2017

Very usefull

viraj2252 commented Sep 28, 2017

Very usefull

@s3459821

This comment has been minimized.

Show comment
Hide comment
@s3459821

s3459821 Sep 29, 2017

These are fantastic instructions, I was amazed at how quickly everyone got through the exercise!
I've been playing around with a server a little since AppSecDay and found the following handy for the super paranoid:
With the certbot command, I used:
certbot certonly --hsts
This allows me to force HSTS which I think is good.
The other thing I changed was in the homeserver.yaml file I changed the bind address to be '127.0.0.1' so that I didn't leave unnecessary ports exposed. This was for both the encrypted and unencrypted ports.
Not big changes at all, but rather very small changes.

s3459821 commented Sep 29, 2017

These are fantastic instructions, I was amazed at how quickly everyone got through the exercise!
I've been playing around with a server a little since AppSecDay and found the following handy for the super paranoid:
With the certbot command, I used:
certbot certonly --hsts
This allows me to force HSTS which I think is good.
The other thing I changed was in the homeserver.yaml file I changed the bind address to be '127.0.0.1' so that I didn't leave unnecessary ports exposed. This was for both the encrypted and unencrypted ports.
Not big changes at all, but rather very small changes.

@atb00ker

This comment has been minimized.

Show comment
Hide comment
@atb00ker

atb00ker Jan 14, 2018

I made an ansible script to preform these actions: https://galaxy.ansible.com/atb00ker/matrix-synapse/
(There are some changes, please read before executing)

atb00ker commented Jan 14, 2018

I made an ansible script to preform these actions: https://galaxy.ansible.com/atb00ker/matrix-synapse/
(There are some changes, please read before executing)

@ncamacho97

This comment has been minimized.

Show comment
Hide comment
@ncamacho97

ncamacho97 Feb 15, 2018

why is it so hard lol not working

ncamacho97 commented Feb 15, 2018

why is it so hard lol not working

@Len-PGH

This comment has been minimized.

Show comment
Hide comment
@Len-PGH

Len-PGH Mar 16, 2018

apt-get -t jessie-backports install matrix-synapse -y
This will get past the error and not have to use aptitude

Len-PGH commented Mar 16, 2018

apt-get -t jessie-backports install matrix-synapse -y
This will get past the error and not have to use aptitude

@Subair-tc

This comment has been minimized.

Show comment
Hide comment
@Subair-tc

Subair-tc Mar 27, 2018

Any JavaScript SDK for implementing the Riot inside my website ?

Subair-tc commented Mar 27, 2018

Any JavaScript SDK for implementing the Riot inside my website ?

@krombel

This comment has been minimized.

Show comment
Hide comment
@krombel

krombel Apr 6, 2018

Just to note something:
This configuration will not work for federation.
You mapped federation (via SRV) to port 443 and used there a letsencrypt certificate.
To let it work at least for a time synapse has to be instructed to use the same certificate for federation. If synapse points to the wrong one it tells the other servers a wrong fingerprint which lets them abort further federation.

As this "What certificate do you have?"-Check is cached every time you update the certificate federation will be blocked again until the cache times out.

Another problem: You propose server01.securechatworkshop.com as identity server here although it does not seem so that there is an identity server listening. Although this does not cause to crash it leads to the situation that identity lookups will not be able to be performed.

Just as a reference I would like to link to an issue which mentions some points for a recommended setup:
matrix-org/synapse#2438

krombel commented Apr 6, 2018

Just to note something:
This configuration will not work for federation.
You mapped federation (via SRV) to port 443 and used there a letsencrypt certificate.
To let it work at least for a time synapse has to be instructed to use the same certificate for federation. If synapse points to the wrong one it tells the other servers a wrong fingerprint which lets them abort further federation.

As this "What certificate do you have?"-Check is cached every time you update the certificate federation will be blocked again until the cache times out.

Another problem: You propose server01.securechatworkshop.com as identity server here although it does not seem so that there is an identity server listening. Although this does not cause to crash it leads to the situation that identity lookups will not be able to be performed.

Just as a reference I would like to link to an issue which mentions some points for a recommended setup:
matrix-org/synapse#2438

@RameshThota87

This comment has been minimized.

Show comment
Hide comment
@RameshThota87

RameshThota87 Jun 20, 2018

Hi,
We have Homeserver running on port number 8448(https) and 8008(http)
And we installed Matrix-Appservice-Slack with the port number 8181

We tried to execute the Slack bridge setup with the following command,
Start the actual application service. You can use forever
node app.js -c config.yaml -p $MATRIX_PORT

we did not provide any user credentials. But we are facing the following issue.

Loading config file /home/user/matrix-appservice-slack/config/config.yaml
Matrix-side listening on port 8182
Slack-side listening on port 8181 over http

Accepting pending invites
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) Body:
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) HTTP 401 Error: {"errcode":"M_UNKNOWN_TOKEN","error":"Unrecognised access token."}
Unhandled rejection M_UNKNOWN_TOKEN: Unrecognised access token.

I have few queries:

  • Is Home server(HS) and Application server (AS) are in same server or on different?
  • Are both ports required to be accessible in public network?

Please help.

Thanks in advance.

RameshThota87 commented Jun 20, 2018

Hi,
We have Homeserver running on port number 8448(https) and 8008(http)
And we installed Matrix-Appservice-Slack with the port number 8181

We tried to execute the Slack bridge setup with the following command,
Start the actual application service. You can use forever
node app.js -c config.yaml -p $MATRIX_PORT

we did not provide any user credentials. But we are facing the following issue.

Loading config file /home/user/matrix-appservice-slack/config/config.yaml
Matrix-side listening on port 8182
Slack-side listening on port 8181 over http

Accepting pending invites
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) Body:
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) HTTP 401 Error: {"errcode":"M_UNKNOWN_TOKEN","error":"Unrecognised access token."}
Unhandled rejection M_UNKNOWN_TOKEN: Unrecognised access token.

I have few queries:

  • Is Home server(HS) and Application server (AS) are in same server or on different?
  • Are both ports required to be accessible in public network?

Please help.

Thanks in advance.

@warp1337

This comment has been minimized.

Show comment
Hide comment
@warp1337

warp1337 Jul 30, 2018

Great workshop

warp1337 commented Jul 30, 2018

Great workshop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment