This version of the document is no longer canonical. You can find the canonical version hosted at Gitlab and Github.
PART 2 IS OUT, INCLUDING THE DISCLOSURE OF A GLOBAL FEDERATION DATA LEAK, AND THE ANATOMY OF A GDPR DATA REQUEST HANDLED BY MATRIX.ORG. SEE THE REPOS ABOVE.
You have been well-informed about mxisd capabilities which you even mentioned in FOSDEM presentations in 2017 (or maybe 2018, possibly both). Dave is in the room and his role as a core board member is to be aware of what is going on. You have been purposefully lying about this for years now in an attempt to hide what mxisd could accomplish while remaining spec compliant. Why?
You are accusing me of "purposefully lying for years" in paragraphs of shouty bold text, while simultaneously acknowledging that I've been promoting mxisd and its capabilities in my talks. (And for that matter we've also been promoting it on matrix.org/blog). This is an almost farcical paradox. It's also starting to sound as if the root of all your unhappiness here may be due to feeling upset that mxisd doesn't get more recognition...
I maintain that the point of an IS is to discover people to talk to on Matrix based on 3PID, and the only way today to usably publish yourself as discoverable is by using the logically centralised sydent servers at vector.im/matrix.org. You can of course run a local IS like mxisd, but it will only render you discoverable to users on the same identity server (unless it delegates to the central servers), which is of very limited usage. The mxisd could try querying other ISes based on the domain of the 3pid (if it has a domain), but this relies on the 3pid having the same domain as the mxid, which is pretty unusual in the real world.
I would much rather (as I said to you years ago when trying to provide input into your IS work) that we spent the time instead providing a reliable and privacy-preserving way of letting users globally publish their 3PID->mxid mappings (if they want to), which doesn't depend on centralised servers at all - which is why we haven't prioritised the federated bodge.
An alternative approach would be to use DNS itself or a DNS-equivalent architecture, similar to ENUM, to let servers recurse identity lookups to a root server and then back down to a given deployment-specific server. But as far as I know, mxisd doesn't support that, nor has anyone proposed it as a design. Plus it would still depend on root servers being a SPOF/SPOC, and would suffer all the same attacks as DNS itself - and be tantamount to reinventing the DNS.
This is why we'd rather pursue an approach which is fully decentralised - i.e. a signed shared datastructure of hashed-3pid -> mxids, scoped to whatever visibility the publisher desires, which anyone can participate in hosting, rather than depending on centralised ISes for global lookups to function.
You also never fail to slander us for something which is your own fault in the official Matrix rooms.
I'm sorry you consider this slander(!)
This sort of hyperbolic rhetoric is why we gave up trying to work with you months ago.
I believe this changes some of your comments where you claim ignorance or oversight.
No, the word oversight means "an unintentional failure to [...] do something." - i.e. the issue in question is an unintentional failure, and one which we haven't solved yet.
We also mentionned having Google TURN servers hardcoded, and it seems like this issue is still happening on mobile clients. Could you confirm or deny?
Yup, the bug is still open, and Google STUN is still hooked up as a last resort to help people who have failed to configure their own VoIP servers:
https://github.com/matrix-org/matrix-ios-sdk/blob/develop/MatrixSDK/VoIP/MXCallManager.m#L34
https://github.com/matrix-org/matrix-android-sdk/blob/master/matrix-sdk/src/main/java/org/matrix/androidsdk/call/MXWebRtcCall.java#L602
This is a good example of where a relatively low priority (for us) issue has not been higher prioritised, as it only affects people who have failed to configure TURN servers on their homeservers.
You are accusing me of "purposefully lying for years" in paragraphs of shouty bold text, while simultaneously acknowledging that I've been promoting mxisd and its capabilities in my talks. (And for that matter we've also been promoting it on matrix.org/blog).
Please don't take my sentence out of context and only reply to it like it was a standalone sentence. I give you the courtesy to not do so, please do the same. My sentence is a reply to your comment, which claims mxisd cannot do those things [1] and shifts the blame on us for not moving Matrix forward.
I have shown proof that:
[1] "those things" refer to the two restricted actions of your comment #8, quoting:
Even if you refute we told you, you had every chance to check your facts before making such a statement. So yes, I have no problem claiming you are purposefully lying. The question is to what goal. Please do not comment further on this point unless you can prove you did in fact do your background check of mxisd features/source code and that mxisd cannot technically perform lookup on the central IS due to the lack of code for it.
Yup, the bug is still open, and Google STUN is still hooked up as a last resort to help people who have failed to configure their own VoIP servers:
Thanks, we'll incorporate this important info and create a new section about VoIP altogether. Thank you for this useful feedback.
Edit: It's here
I will not reply to the rest of your comments given their personal nature, and their lack of relevance for the document.
[Edit: removed section that was replying to an irrelevant section by mistake, clarify what those things are]
So after double-checking again, it seems like Comment 38 is not factually correct and that Cloudflare DOES TLS termination, directly having access to all the data in clear.
Here is a Client request done now:
$ curl -sv https://matrix.org/_matrix/client/r0/login
* Trying 2606:4700:10::6814:15ec...
* TCP_NODELAY set
* Connected to matrix.org (2606:4700:10::6814:15ec) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS Unknown, Certificate Status (22):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS change cipher, Client hello (1):
* (304) (OUT), TLS Unknown, Certificate Status (22):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using unknown / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=www.matrix.org
* start date: Jun 11 11:32:44 2019 GMT
* expire date: Sep 9 11:32:44 2019 GMT
* subjectAltName: host "matrix.org" matched cert's "matrix.org"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55974db54580)
* (304) (OUT), TLS Unknown, Unknown (23):
> GET /_matrix/client/r0/login HTTP/2
> Host: matrix.org
> User-Agent: curl/7.58.0
> Accept: */*
>
* (304) (IN), TLS Unknown, Certificate Status (22):
* (304) (IN), TLS handshake, Newsession Ticket (4):
* (304) (IN), TLS handshake, Newsession Ticket (4):
* (304) (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (IN), TLS Unknown, Unknown (23):
* (304) (IN), TLS Unknown, Unknown (23):
< HTTP/2 200
< date: Sun, 07 Jul 2019 09:39:09 GMT
< content-type: application/json
< set-cookie: __cfduid=dc79ff628c73af629e2cb1ccbe2c117be1562492349; expires=Mon, 06-Jul-20 09:39:09 GMT; path=/; domain=.matrix.org; HttpOnly
< cache-control: no-cache, no-store, must-revalidate
< access-control-allow-origin: *
< access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
< access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 4f28d9820c6bd4b4-BRU
<
{
"flows": [
{
"type": "m.login.password"
}
]
}
* (304) (IN), TLS Unknown, Unknown (23):
* Connection #0 to host matrix.org left intact
Here is a Federation request done now:
$ curl -sv https://matrix.org:8443/_matrix/federation/v1/version
* Trying 2606:4700:10::6814:14ec...
* TCP_NODELAY set
* Connected to matrix.org (2606:4700:10::6814:14ec) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS Unknown, Certificate Status (22):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS change cipher, Client hello (1):
* (304) (OUT), TLS Unknown, Certificate Status (22):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using unknown / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=www.matrix.org
* start date: Jun 11 11:32:44 2019 GMT
* expire date: Sep 9 11:32:44 2019 GMT
* subjectAltName: host "matrix.org" matched cert's "matrix.org"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x56503e398580)
* (304) (OUT), TLS Unknown, Unknown (23):
> GET /_matrix/federation/v1/version HTTP/2
> Host: matrix.org:8443
> User-Agent: curl/7.58.0
> Accept: */*
>
* (304) (IN), TLS Unknown, Certificate Status (22):
* (304) (IN), TLS handshake, Newsession Ticket (4):
* (304) (IN), TLS handshake, Newsession Ticket (4):
* (304) (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* (304) (OUT), TLS Unknown, Unknown (23):
* (304) (IN), TLS Unknown, Unknown (23):
* (304) (IN), TLS Unknown, Unknown (23):
< HTTP/2 200
< date: Sun, 07 Jul 2019 09:36:05 GMT
< content-type: application/json
< set-cookie: __cfduid=db97ca0304b4159c2ac95eb7e37e734851562492163; expires=Mon, 06-Jul-20 09:36:03 GMT; path=/; domain=.matrix.org; HttpOnly
< cache-control: no-cache, no-store, must-revalidate
< access-control-allow-origin: *
< access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
< access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 4f28d4f6fcf5442b-BRU
<
{
"server": {
"name": "Synapse",
"version": "1.1.0rc1 (b=matrix-org-hotfixes,43e01be15)"
}
}
* (304) (IN), TLS Unknown, Unknown (23):
* Connection #0 to host matrix.org left intact
Edit: vector.im as an identity server:
$ curl -sv https://vector.im/_matrix/identity/api/v1 | jq
* Trying 2606:4700:30::681f:475f...
* TCP_NODELAY set
* Connected to vector.im (2606:4700:30::681f:475f) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
} [5 bytes data]
* (304) (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS Unknown, Certificate Status (22):
{ [1 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [15 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [2170 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* (304) (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* (304) (OUT), TLS Unknown, Certificate Status (22):
} [1 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using unknown / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=CA; L=San Francisco; O=CloudFlare, Inc.; CN=vector.im
* start date: Feb 14 00:00:00 2019 GMT
* expire date: Feb 14 12:00:00 2020 GMT
* subjectAltName: host "vector.im" matched cert's "vector.im"
* issuer: C=US; ST=CA; L=San Francisco; O=CloudFlare, Inc.; CN=CloudFlare Inc ECC CA-2
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* (304) (OUT), TLS Unknown, Unknown (23):
} [1 bytes data]
* (304) (OUT), TLS Unknown, Unknown (23):
} [1 bytes data]
* (304) (OUT), TLS Unknown, Unknown (23):
} [1 bytes data]
* Using Stream ID: 1 (easy handle 0x560bab614580)
} [5 bytes data]
* (304) (OUT), TLS Unknown, Unknown (23):
} [1 bytes data]
> GET /_matrix/identity/api/v1 HTTP/2
> Host: vector.im
> User-Agent: curl/7.58.0
> Accept: */*
>
{ [5 bytes data]
* (304) (IN), TLS Unknown, Certificate Status (22):
{ [1 bytes data]
* (304) (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* (304) (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* (304) (IN), TLS Unknown, Unknown (23):
{ [1 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
} [5 bytes data]
* (304) (OUT), TLS Unknown, Unknown (23):
} [1 bytes data]
* (304) (IN), TLS Unknown, Unknown (23):
{ [1 bytes data]
* (304) (IN), TLS Unknown, Unknown (23):
{ [1 bytes data]
< HTTP/2 200
< date: Sun, 07 Jul 2019 22:57:56 GMT
< content-type: application/json
< set-cookie: __cfduid=d14ef1f6254ebb5c211fdb3493dfabfbf1562540276; expires=Mon, 06-Jul-20 22:57:56 GMT; path=/; domain=.vector.im; HttpOnly
< access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
< access-control-allow-origin: *
< access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 4f2d6b993a53d4b4-BRU
<
{ [2 bytes data]
* (304) (IN), TLS Unknown, Unknown (23):
{ [1 bytes data]
* Connection #0 to host vector.im left intact
{}
In all cases, we can see the headers set-cookie, server, cf-ray and expect-ct with values set by Cloudflare, which would not be possible if TLS termination was done directly on matrix.org/vector.im servers.
@ara4n Following some links in my own links, I came accross this old issue: prism-break#1936 where I list the main concerns that I've re-documented in details in this research paper.
To which you reply. So you've been aware of the major points of this document for more than a year now. I believe this changes some of your comments where you claim ignorance or oversight.
We also mentionned having Google TURN servers hardcoded, and it seems like this issue is still happening on mobile clients. Could you confirm or deny?