Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

A/V calls in Conversations

Table of contents

Notes for users

Where is the call button?

The call button is in the toolbar, to the left hand side of the lock icon. There are three conditions that must be met in order for the call button to show up.

  • You and your contact need Conversations 2.8.0+ or another compatible client that supports A/V calls (Siskin, Beagle, Movim, …)
  • You must have each other in your contact lists (with mutual presence subscription)
  • The contact needs to be online. This is indicated by a colorized (not gray) send button. Depending on the recipients setup you might be able to send a regular message first to wake up the recipient’s device. (In the future Conversations might gain the ability to do this automatically.)

What does ‚App failure‘ mean?

App failure can mean one of two things. Either the XMPP client on the other end isn’t fully compatible to Conversations, or Conversations was unable to setup the native library that powers A/V calls. The former is generally more likely while the latter might happen if you got the Conversations APK from dubios sources or running Conversations on a strange device or emulator. In either case there is not a lot you can do aside from reporting the issue in our community channel.

What does ‚Unable to connect call‘ mean?

A/V calls require a direct connection between the two participating devices (peer-to-peer). ‚Unable to connect call‘ means that one or both participants are in a network that prevents direct connections. Your server might be able to assist your device with establishing that connection regardless. To do that your server needs to support ‚XEP-0215: External Service Discovery‘. You can find out if your server supports that by going into your account details (either by tapping your own avatar or by going through ‚Manage accounts‘), and than selecting ‚Server Info‘ from the overflowmenu. If your server doesn’t support this you might want to contact your provider. If you run your own server see instructions below. Both your server and your contact’s server need to support this.

Notes for server admins

Most calls will require server side assistence. Depending on the network it might be enough to have a STUN server for Conversations to learn your external IP and punch a hole in the NAT. On some, more restrictive, networks however this isn’t enough and Conversations will need a TURN server. TURN servers are used to proxy the entire (encrypted) traffic through the server. (In my initial testing this was often the case when mobile networks were involved.)

To ensure best possible user experience in all situations server admins should set up both.

Conversations will use XEP-0215: External Service Discovery to learn about server-provided STUN/TURN servers, and, in case of TURN, also get short term, temporary credentials to access the TURN server.

Instructions for specific servers

ejabberd

Ejabberd 20.04 has support for XEP-0215 and has also a STUN and TURN server build in. Take a look a the new sample config (Particularly the module: ejabberd_stun section in the listen section and mod_stun_disco.)

Prosody

A combination of running coturn and mod_turncredentials should get you going. Wiktor-k has written some instructions and provided some config files.

~80% solution with public STUN server

We strongly recommend that you set up your own STUN/TURN server (see above). If you can’t do that for whatever reason (firewall, resource constraints, lack of time, …) you can use Prosody’s mod_extdisco to point to a public STUN server. That should make A/V calls work in ~80% of cases (usually WiFi to WiFi) with minimal configuration required and with the down side of leaking IP addresses to the operator of the STUN server. The configuration looks like this:

modules_enabled = {
    -- other modules ...
    "extdisco";
}

external_services = {
    ["stun.conversations.im"] = {
        port="443";
        transport="udp";
        type="stun";
    };
}
CSI

If you don’t get any notifications for incoming calls make sure that your CSI module is letting XEP-353 messages through.

OpenFire

You need an external STUN/TURN server like coturn and the External Service Discovery plugin. More information can be found here.

UDP vs TCP vs TLS

STUN and TURN can operate over three different protocols. UDP, TCP and TCP/TLS. The latter is indicated by the stuns and turns URI scheme. Using TLS does not increase security as calls are always end-to-end encrypted with DTLS-SRTP. On the down side using TCP or TLS instead of UDP might negatively impact latency and performance. The only benefit of using TURN over TLS on port 443 is that you have a higher chance of passing through restrictive firewalls. However this should only be a fallback and not the default connection mechanism.

As a general recommendation we advise you to announce the following services over XEP-0215:

  • STUN over UDP
  • TURN over UDP
  • TURNS over TLS on port 443 (requires extra IP on the server)

Ideally those three variants should exist both on IPv4 and IPv6 for a total of 6 variants.

Testing

The compliance tester will check if that discovery mechanism is working; however that doesn’t necessarily mean that TURN and STUN themselves are setup correctly. To test this you should get two Android phones and put them into different networks; connect one of them to your computer via USB and use adb logcat. Here are the lines two look out for:

Firstly if you grep for ICE with adb -d logcat -v time -s conversations | grep ICE you should see log entries like this:

04-18 12:57:59.373 D/conversations(13867): alice@example.com: discovered ICE Server: [stun:89.238.78.51:443?transport=udp] [:] [TLS_CERT_POLICY_SECURE] [] [null] [null]
04-18 12:57:59.373 D/conversations(13867): alice@example.com:: discovered ICE Server: [turn:89.238.78.51:443?transport=udp] [1587211080:a781616cb9061724:F17BHTfLXyxzOyWSEutjpmzlCrs=] [TLS_CERT_POLICY_SECURE] [] [null] [null]
04-18 12:57:59.374 D/conversations(13867): alice@example.com:: discovered ICE Server: [stun:89.238.78.51:443?transport=tcp] [:] [TLS_CERT_POLICY_SECURE] [] [null] [null]
04-18 12:57:59.374 D/conversations(13867): alice@example.com:: discovered ICE Server: [turn:89.238.78.51:443?transport=tcp] [1587211080:a781616cb9061724:F17BHTfLXyxzOyWSEutjpmzlCrs=] [TLS_CERT_POLICY_SECURE] [] [null] [null]

This means Conversations has been able to discover the servers (you should see at least one line with stun and one line with turn.

Secondly when the call connects and you grep for candidate with adb -d logcat -v time -s conversations | grep candidate you should see lines like this:

04-18 12:57:59.584 D/conversations(13867): received candidate: audio:0:candidate:2431496480 1 udp 41754623 89.238.78.51 63631 typ relay raddr 94.134.91.66 rport 22965 generation 0 ufrag JRB3::UNKNOWN
04-18 12:57:59.592 D/conversations(13867): received candidate: audio:0:candidate:3731770832 1 udp 24977151 89.238.78.51 61571 typ relay raddr 94.134.91.66 rport 22966 generation 0 ufrag JRB3::UNKNOWN
04-18 12:57:59.592 D/conversations(13867): sending candidate: audio:0:candidate:2765706476 1 udp 2122194687 10.255.12.234 42571 typ host generation 0 ufrag xJI/ network-id 3 network-cost 900::UNKNOWN
04-18 12:57:59.641 D/conversations(13867): sending candidate: audio:0:candidate:842163049 1 udp 1685987071 2.247.248.234 31910 typ srflx raddr 10.255.12.234 rport 42571 generation 0 ufrag xJI/ network-id 3 network-cost 900:stun:89.238.78.51:443:UNKNOWN
04-18 12:57:59.691 D/conversations(13867): sending candidate: audio:0:candidate:826779982 1 tcp 1518283007 2a02:303e:5014:d2d0:3188:ad11:a0db:35b6 9 typ host tcptype active generation 0 ufrag xJI/ network-id 4 network-cost 900::UNKNOWN
04-18 12:57:59.700 D/conversations(13867): sending candidate: audio:0:candidate:2431496480 1 udp 41820159 89.238.78.51 62645 typ relay raddr 2.247.248.234 rport 31910 generation 0 ufrag xJI/ network-id 3 network-cost 900:turn:89.238.78.51:443?transport=udp:UNKNOWN
04-18 12:57:59.742 D/conversations(13867): sending candidate: audio:0:candidate:3731770832 1 udp 25042687 89.238.78.51 62032 typ relay raddr 2.247.248.234 rport 6483 generation 0 ufrag xJI/ network-id 3 network-cost 900:turn:89.238.78.51:443?transport=tcp:UNKNOWN

typ srflx means STUN. typ relay means TURN. If you see entries with relay coming up that is already an OK sign. However the only true tell is if you get lines like:

04-18 12:57:59.846 D/conversations(13867): remote candidate selected: :-1:candidate:842163049 1 udp 1685921535 94.134.91.66 22965 typ srflx raddr 192.168.178.39 rport 50732 generation 0 ufrag JRB3 network-cost 10::UNKNOWN
04-18 12:57:59.846 D/conversations(13867): local candidate selected: :-1:candidate:2431496480 1 udp 41820159 89.238.78.51 62645 typ relay raddr 2.247.248.234 rport 31910 generation 0 ufrag xJI/ network-id 3 network-cost 900::CELLULAR

were at least one of them is a typ relay with your TURN server. If the call connects but you only see host or srflx it just means that you lucked out on the network and your network didn’t need turn. (And that case you should try changing networks for a better testing enviroment.

If the call doesn’t connect at all it also means that your setup might not be correct.

Testing without Conversations

If you must test without Conversations the Trickle ICE test in the WebRTC samples might give you some indication. You will probably have to manually XEP-0215 services-query your XMPP server to get temporary TURN credentials. For optimal testing (though not perfect) you should run the Trickle ICE test in Chromium since Conversations and Chromium use the same WebRTC library and there might be subtle differences between Firefox and Google’s libwebrtc.

Notes for developers

Conversations will display the call button if one of the contact’s connected clients supports all of the following namespaces:

  • urn:xmpp:jingle:1
  • urn:xmpp:jingle:transports:ice-udp:1
  • urn:xmpp:jingle:apps:rtp:1
  • urn:xmpp:jingle:apps:dtls:0
  • urn:xmpp:jingle:apps:rtp:audio and optionally also urn:xmpp:jingle:apps:rtp:video

Note: Conversations requires XEP-0115: Entity Capabilities to detect those features.

If any of the contact’s available resources annouce urn:xmpp:jingle-message:0 as a Disco feature Conversations will use XEP-0353: Jingle Message Initiation send to the bare jid of the contact to establishe a call. If not it will initialize the Jingle session directly. If you use Jingle Message Initiation make sure that you include the exact same descriptions that your following session-initiate will include as well. So for example putting only media="audio"in the propose but audio and video in the session-initiate the call will be rejected.

You will also need DTLS otherwise your call will be rejected.

Usually Conversations tries to put human readable text in the termination reasons. So if your call fails make sure to check those out. As far as the UI is concerned Conversations will display 'Unable to connect call' on network errors and 'application error' when something else goes wrong or the session got terminated. Usually this means the implementation are incompatible (again; check those <text>…</text> in the reason element. (Also the reason itself might be a hint; like getting security-error when DTLS is not sent.)

Obviously if you are the developer of another client and want to ensure inter-op feel free to contact me; You will most like already have my XMPP address. If not feel free to step by the Conversations channel.

@oksjd

This comment has been minimized.

Copy link

@oksjd oksjd commented Apr 27, 2020

Thanks for implementing this feature! But what about calling in an IPv6-only environment? Previously I've been able to use a/v calling on ATalk (https://github.com/cmeng-git/atalk-android) without any special configuration on ejabberd server. Surely with IPv6 there's no need for STUN/TURN.

@iNPUTmice

This comment has been minimized.

Copy link
Owner Author

@iNPUTmice iNPUTmice commented Apr 27, 2020

It says most calls will require STUN or TURN. Not all. Calls within the same WiFi network or any call where you have globally routed addresses (this isn’t depended on IPv6) won’t need it.

@oksjd

This comment has been minimized.

Copy link

@oksjd oksjd commented Apr 27, 2020

Thanks for the speedy reply and all the hard work! Just finished the translation of telephony related strings on Transifex. I'll be testing this when F-Droid version is out.

@wkg

This comment has been minimized.

Copy link

@wkg wkg commented May 2, 2020

Are calls encrypted?

@iNPUTmice

This comment has been minimized.

Copy link
Owner Author

@iNPUTmice iNPUTmice commented May 2, 2020

Are calls encrypted?

yes

@wkg

This comment has been minimized.

Copy link

@wkg wkg commented May 3, 2020

I have try use stunS and turnS with ejabberd, but this is do not wok, "conversations" support it?
discovered ICE Server: [turns:example.com:5349?transport=tcp] [:*****:***] [TLS_CERT_POLICY_SECURE] [] [null] [null]
discovered ICE Server: [stuns:example.com:5349?transport=tcp] [:] [TLS_CERT_POLICY_SECURE] [] [null] [null]
but if i setting only turns/stuns, without turn/stun -- calls do not work..

@iNPUTmice

This comment has been minimized.

Copy link
Owner Author

@iNPUTmice iNPUTmice commented May 3, 2020

Conversations up to and including 2.8.1 does not support turns. In general it is recommended to use turn instead of turns. TLS just adds unnecessary round trips and delay for now security gain. The media stream itself is encrypted independently of TLS.

@wkg

This comment has been minimized.

Copy link

@wkg wkg commented May 3, 2020

Conversations up to and including 2.8.1 does not support turns. In general it is recommended to use turn instead of turns. TLS just adds unnecessary round trips and delay for now security gain. The media stream itself is encrypted independently of TLS.

Thank you. In future turns planned support?

@wkg

This comment has been minimized.

Copy link

@wkg wkg commented May 3, 2020

And turn UDP do not wotk if "conversations" and my server in same network, with turn TCP it OK!

[ "conversations" ] ------- NAT ------- [ nat net ---- NAT server and ejabberd -- public ip ] -- --- iternet, other user "conersations" other server

if in this scheme ejabberd have only udp stun/turn - calls do not work, with stun/turn tcp - work

@kousu

This comment has been minimized.

Copy link

@kousu kousu commented May 5, 2020

This is amazing! Thank you for your work @iNPUTmice. I had no idea this was in the works.

@kousu

This comment has been minimized.

Copy link

@kousu kousu commented May 5, 2020

Will group calling work? MUC chats? Is Jitsi Meet compatibility in the roadmap?

@kousu

This comment has been minimized.

Copy link

@kousu kousu commented May 7, 2020

For any developers coming this way in the future I want to clarify that "supports all of the following namespaces" means XEP 0030 disco#info <feature> tags, and that Conversations also needs you to support XEP 0115 or else it won't query for your disco#info (-- this is actually specified as part of XEP 0115: "Clients should not engage in the older "disco/version flood" behavior and instead should use Entity Capabilities as specified herein"), and also there's a small typo in the list: rn:xmpp:jingle:1 should be urn:xmpp:jingle:1.

@iNPUTmice

This comment has been minimized.

Copy link
Owner Author

@iNPUTmice iNPUTmice commented May 7, 2020

Will group calling work? MUC chats? Is Jitsi Meet compatibility in the roadmap?

Group calling would require a server side component like Jitsi Meet and there is no interest on the side of the Jitsi Meet people to make this happen.

@kousu

This comment has been minimized.

Copy link

@kousu kousu commented May 7, 2020

@mase76

This comment has been minimized.

Copy link

@mase76 mase76 commented May 8, 2020

How to make sure, that CSI lets the XEP-353 messages through? I have Prosody with mod_csi and mod_csi_battery_saver. There are no config options.

@moppman

This comment has been minimized.

Copy link

@moppman moppman commented May 8, 2020

If your mod_csi_battery_saver is up to date, you're good. It has been patches recently, see https://hg.prosody.im/prosody-modules/rev/19c5bfc3a241

@mase76

This comment has been minimized.

Copy link

@mase76 mase76 commented May 8, 2020

Updated the mod. Works now.
Thx!

@zhnikita

This comment has been minimized.

Copy link

@zhnikita zhnikita commented May 11, 2020

https://gist.github.com/iNPUTmice/a28c438d9bbf3f4a3d4c663ffaa224d9#gistcomment-3281944
How audio-video calls encrypt?
Srtp, zrtp, etc?

iNPUTmice - thanks for your work! It's cool!!!

@wkg

This comment has been minimized.

Copy link

@wkg wkg commented May 11, 2020

https://gist.github.com/iNPUTmice/a28c438d9bbf3f4a3d4c663ffaa224d9#gistcomment-3281944
How audio-video calls encrypt?
Srtp, zrtp, etc?

iNPUTmice - thanks for your work! It's cool!!!

"as calls are always end-to-end encrypted with DTLS-SRTP"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.