Related Repositories : https://github.com/qaul/qaul.net
Final Blog Post : https://blog.freifunk.net/2023/08/28/gsoc23-final-report-qaul-matrix-bridge-tutorial/
Technical Developement Book: https://harshil-jani.github.io/GSoC-Book-2.0/preface.html
Raised PRs : qaul/qaul.net#563 , qaul/qaul.net#591
Qaul.net is completely internet independent and Peer to Peer chat application where within a local network, Devices can be connected and the communication can take place. There are no chances of being wiretapped since, It does not work on the internet. Also if you get caught up in situations where the internet services are down deliberately or indeliberately then too you can become part of the network using Qaul. All you need is the working device and the application itself.
Matrix is an open protocol for decentralised, secure communications. It act as a neutral custodian for communication and to nurture it as efficiently as possible as a single unfragmented standard, for the greater benefit of the whole ecosystem, not benefiting or privileging any single player or subset of players.
There are multiple examples of places where the governments cuts down the communication links over a particular region due to riots or suspicious activities or sometimes due to political playings. With Qaul the aim is to provide local links of communication so no one feels lefted behind. For Example, In India-Pakistan border, there is always tension of riots between the nations for accquiring regions of Kashmir. Due to this, The internet services are blocked and also the communication infrastructure is damaged at times. Here, With qaul the issue can be addressed and people can still utilize communication since it falls under very basic rights of human beings.
Each device is called node and it has a cryptographic id called qaul-id. Now for discovering the peer devices, It used mDNS (Multicast Domain Name System) using which you can get the IP addresses of the peer devices without reaching or using the root DNS server (like .com or .edu or .uk etc) and no data is transfered to anyone outside the local network. The messages stays end-to-end encrypted because of the cryptographic keys used to sign them while sending and are verified on the receiving end. For routing the messages the Distance Vector Routing protocol is used which is based on the Round trip time per connection request. So under the hoods, Every 5 secsonds each device sends ping to neighbour nodes to measure RTT and every 10 seconds each node sends the routing information to all neighbouring nodes. For routing you can use any route. It could be LAN or INTERNET or even BLE (in progress). The protocol will choose the best route and send the message over the network using that route. The device id remains throughout the time even when device is not connected to the network until the application is uninstalled. So, You can easily go offline and come back and get connected over the same local network.
Initially we thought that we would run matrix bridge as a daemon process and use Go Lang to create the bridge. But, My mentor has a friend working in element, which is a matrix client, suggested us that since the world is taking on for the Rust, Matrix-SDK are actively written in Rust and there is something called as RuMa which stands for Rust Matrix. RuMa is an amazing work and thus, We have decided to do it in rust only because our Qaul has its entire backend in Rust.
I had read and researched through the RuMa and Matrix-SDK-Rust projects since this is now the part which we will be using for our project instead of GoLang. Mentor suggested that I should duplicate the qaul-cli binary and tweak it in whatever way I would need to in order to work on it. Here is the main reason why we chose the qaul-cli specifically for implementing the bridge concept.
It already has two workers set in place which will check for any activity on entire qaul network each 10ms. We have access to CLI which we use to interact with RPC protocol and protobuff messaging. I was reading through the documentation of the matrix-sdk crate and what I found was a beautifully commented codes for creating the bot inside examples/ directory. They were really helpful for me to initiate the coding part.
[On Matrix]
- Create a bot account for Qaul and specify a server to work on.
- Invite the bot to the testing matrix room.
[On Qaul]
- Create a binary copy of qaul-cli
- Code the logic to login our bot into the matrix room as soon as the qaul-cli binary is running.
- Also Code a basic testing functionality (For Eg : Call it !ping command)
[On Matrix]
- Login with our personal account [@harshil1] and send a message with !ping in the room.
- In response we should receive all the nodes connected to the network.
Instead of just an echo as response, We should pick the messages from both the ends. Send “Hi” from qaul and it should first sense in matrix without our personal human activity that there is some event triggered in qaul. Once event is detected, the message should show up into the matrix room.
Next we can reverse engineer the above feature and do the same in qaul by sending a message in matrix room.
This just follows the Version 1 functionality wise but this should be implemented for 1-on-1 direct messages. In matrix and qaul both, private DMs are nothing but a group with only two members. We need to create a use case where we can send the message in groups by inviting a bot and the bot invite the user on other application and rest remains same.
After the above completions, We can think of double puppeting the bot so now our bot is not just qaul-bridge but a real username from the qaul node.
We have built end to end Matrix to Qaul bridge working as expected for Version 1 and will be achieving the Version 2 within next week. Speaking in-depth about version 2, We already have a functionality to check if there is any new group requiring to connect with a matrix user and accordingly the Qaul-Bot opens Matrix room and invites wanted Matrix user. Then Matrix user is able to send messages into the qaul group. We are supposed to close the part where a message goes from Qaul into the Matrix.
We have completed the relay-bot implementation with private group-chats (DMs), adding and removing of users, sharing files over the network, disabling CLI and converting it into daemon process.
You should have an account on matrix that can act as a bridge bot.
For more secure communications you can opt for running your own matrix homeserver but that is not necessary since our bridge works well on the default matrix server as well.
You should have the binary to the bridge either as code or distributed package.
We are still working on packaging the binary for end users and then in place of cargo run –bin qaul-matrix-bridge you can simply run our binary.
As of now, We are supporting the bridge only as a daemon process binary without any control on it via CLI or GUI. All the logics are integrated on matrix-sdk or ruma and qaul or libqaul.
On the server where you wish to start a binary between a local qaul network, You need to have one node running the binary and rest will follow along the way.
After installing the qaul project you can
- homeserver-url : The URL for matrix homeserver. Default can be [https://matrix.org]
- bot-matrix-id : The user account of the bot on matrix. Eg : @qaul-bot:matrix.org then id is [qaul-bot]
- bot-account-password : The password for your bot account on matrix
Once the bridge is running up. You have to go to your own matrix account and create a new room. Please make sure to turn off any encryption. Now invite the bot to your matrix room and it automatically joins the room.
- Create a matrix room and disable end to end encryption
- Invite the bot account into the room
We have multiple menu options available in our Matrix Menu. You can see the list of all the possible functionalities with !help command.
I am glad to share the prototype of the bridge. It was a great experience learning from different matrix specifications and their protocol design. In future, We are looking forward to incorporate Qaul Menu, distribute the binaries over various mediums, Styling the bot responses via emotes and lot more.
A special thanks to my mentor MathJud who have helped with serious problems whenever we faced it and with him I have learned very important principles when designing a chat application. He also introduced me to Matrix Camp in Chaos Communication Camp where we presented the prototype (bit buggy at that time) to people who are working on Matrix protocol.
I would also encourage people to come and contribute to the Qaul project because this project serves an important use case to allow communication during Internet break-outs. Developing a solution to stay hidden from government will help sustain the openess of speech and freedom.