Skip to content

Instantly share code, notes, and snippets.

View erikjohnston's full-sized avatar
🦊

Erik Johnston erikjohnston

🦊
  • Element / Matrix.org
  • United Kingdom
  • 00:26 (UTC +01:00)
View GitHub Profile
@erikjohnston
erikjohnston / s3_storage_provider.py
Created January 18, 2018 21:03
A storage provider that fetches media from an S3 bucket
from twisted.internet import defer, threads, reactor
from twisted.python.failure import Failure
from synapse.rest.media.v1.storage_provider import StorageProvider
from synapse.rest.media.v1._base import Responder
import boto3
import botocore
import logging
import threading

Fundamental Requirements

Requirements for room event storage:

  1. Lossless storage of the event JSON as the signatures need to remain valid (this is most easily done by simply storing the serialized JSON in a binary column)
  2. Able to work out the state of the room at any given event.
  3. Able to traverse the rooms event in topological order (for federation).
  4. Able to fetch new events for users (including events in rooms that the user is in and membership changes in any room, e.g. invites)
  5. Able to fetch the most recent N events in each room that the user is in (for initial syncs). In the future initial syncs will most likely be paginated and so the server will need to be able to fetch the most recent recent N events in the most recent M rooms.
  6. Able to fetch events older than a given point in a room (for back pagination).
@erikjohnston
erikjohnston / synapse2neo4j.py
Created February 10, 2015 22:32
synapse->neo4j
from py2neo import Graph, Node, Relationship
from synapse.events import FrozenEvent
import json
import sqlite3
import sys
def main(db_name, room_id):
@erikjohnston
erikjohnston / log_config.yaml
Created February 10, 2015 15:48
Example synapse log config
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
filters:
context:
(): synapse.util.logcontext.LoggingContextFilter
request: ""
@erikjohnston
erikjohnston / email2matrix.py
Last active August 29, 2015 14:13
Takes an email from stdin and uploads to matrix.
import email
import json
import sys
import random
from matrix_client.api import MatrixHttpApi
html_template = (
"<b>From</b>: %(from)s\n"
"<b>To</b>: %(to)s\n"

Keybase proof

I hereby claim:

  • I am erikjohnston on github.
  • I am erikjohnston (https://keybase.io/erikjohnston) on keybase.
  • I have a public key whose fingerprint is BA6B B0C8 FB58 2CBD 8E32 161D 3ADA 06ED C753 D11E

To claim this, I am signing this object: