Skip to content

Instantly share code, notes, and snippets.

@turt2live
Last active August 3, 2018 22:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save turt2live/ef4ec647605b666cfae1f82f96aa5de6 to your computer and use it in GitHub Desktop.
Save turt2live/ef4ec647605b666cfae1f82f96aa5de6 to your computer and use it in GitHub Desktop.
Automatically redacts spam in matrix, usually sent by freenode spam bots.
config.json
node_modules
config.json
node_modules

matrix-spam-redact

Automatically redacts messages that aren't wanted. This works by using two accounts: one for monitoring rooms and finding banned messages and another for actually redacting them. This is so that moderators with large accounts can redact spam messages without having to /sync their potentially very large account. Moderators which do not have a second account can just use the same account as both the monitor and redactor.

The requirements for the monitor are:

  • Must be in the rooms being monitored
  • Must be a valid account
  • Must be on the same homeserver as the account issuing the redactions

The requirements for the redactor are:

  • Must be in the rooms being monitored
  • Must be a valid account
  • Must be on the same homeserver as the account listening for redactions
  • Must have permission to redact messages (usually "moderator" or higher)

This script works with NodeJS 8.x or higher.

Running (Manual)

git clone https://gist.github.com/ef4ec647605b666cfae1f82f96aa5de6.git matrix-spam-redact
cd matrix-spam-redact
npm install
cp config.example.json config.json
vi config.json   # Use your favourte editor to change the config
node index.js

Running (Docker)

# Copy the config.example.json to somewhere like /etc/matrix-spam-redact/config.json

vi /etc/matrix-spam-redact/config.json   # Use your favourte editor to change the config
docker run -v /etc/matrix-spam-redact/config.json:/data/config.json turt2live/matrix-spam-redact
{
"hs_url": "https://t2l.io",
"monitor_access_token": "your_token_here",
"redactor_access_token": "your_token_here",
"user_prefix": "@freenode",
"user_suffix": ":matrix.org",
"room_ids": [
"!QtykxKocfZaZOUrTwp:matrix.org"
],
"banned_messages": [
"Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/",
"<script type=\"text/javascript\" src=\"http://web.nba1001.net:8888/tj/tongji.js\"></script>",
"Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate",
"This message was brought to you by Private Internet Access",
"or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/",
"This message was brought to you by Private Internet Access. Voice your opinions at https://webchat.freenode.net/?channels=%23freenode"
],
"github": {
"enable_redactions": true,
"user_id": "@_neb_github:matrix.org",
"numbers": [1]
}
}
FROM node:8.11-alpine
COPY . /
RUN cd / && npm install
ENV SPAM_CONFIG_PATH=/data/config.json
VOLUME /data
ENTRYPOINT [ "node", "index.js" ]
const MatrixClient = require("matrix-bot-sdk").MatrixClient;
const config = require(process.env["SPAM_CONFIG_PATH"] || "./config.json");
const monitor = new MatrixClient(config['hs_url'], config['monitor_access_token']);
const redactor = new MatrixClient(config['hs_url'], config['redactor_access_token']);
const MSG_TYPE = "m.text";
const EVENT_TYPE = "m.room.message"; // technically can't be anything else
monitor.on("room.message", (roomId, event) => {
if (config['room_ids'].indexOf(roomId) === -1) return;
if (event['type'] !== EVENT_TYPE || !event['content']) return;
if (!event['sender'].startsWith(config['user_prefix']) || !event['sender'].endsWith(config['user_suffix'])) return;
if (event['content']['msgtype'] !== MSG_TYPE) return;
if (config['banned_messages'].indexOf(event['content']['body']) === -1) return;
console.log("Banned message encountered: " + event['event_id']);
redactor.doRequest("PUT", "/_matrix/client/r0/rooms/" + roomId + "/redact/" + event['event_id'] + "/" + (new Date().getDate() + event['event_id']), null, {})
.then(() => console.log("redacted"))
.catch(err => console.error(err));
});
if (config["github"] && config["github"]["enable_redactions"]) {
monitor.on("room.message", (roomId, event) => {
if (config['room_ids'].indexOf(roomId) === -1) return;
if (event['type'] !== EVENT_TYPE || !event['content']) return;
if (event['sender'] !== config["github"]["user_id"]) return;
if (event["content"]["msgtype"] !== "m.notice") return; // How would github even create a non-notice?
for (let number of config["github"]["numbers"]) {
const regex = new RegExp("^https:\/\/github\.com\/.*\/(issues|pull)\/" + number + " :");
if (regex.test(event["content"]["body"])) {
console.log("Github has helped in response to a message - redacting " + event['event_id']);
redactor.doRequest("PUT", "/_matrix/client/r0/rooms/" + roomId + "/redact/" + event['event_id'] + "/" + (new Date().getDate() + event['event_id']), null, {})
.then(() => console.log("redacted github help"))
.catch(err => console.error(err));
}
}
})
}
monitor.start().then(() => console.log("Monitor account started"));
{
"name": "matrix-spam-redact",
"version": "1.0.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"ajv": {
"version": "5.5.2",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-5.5.2.tgz",
"integrity": "sha1-c7Xuyj+rZT49P5Qis0GtQiBdyWU=",
"requires": {
"co": "^4.6.0",
"fast-deep-equal": "^1.0.0",
"fast-json-stable-stringify": "^2.0.0",
"json-schema-traverse": "^0.3.0"
}
},
"asn1": {
"version": "0.2.3",
"resolved": "https://registry.npmjs.org/asn1/-/asn1-0.2.3.tgz",
"integrity": "sha1-2sh4dxPJlmhJ/IGAd36+nB3fO4Y="
},
"assert-plus": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz",
"integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU="
},
"asynckit": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k="
},
"aws-sign2": {
"version": "0.7.0",
"resolved": "https://registry.npmjs.org/aws-sign2/-/aws-sign2-0.7.0.tgz",
"integrity": "sha1-tG6JCTSpWR8tL2+G1+ap8bP+dqg="
},
"aws4": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/aws4/-/aws4-1.7.0.tgz",
"integrity": "sha512-32NDda82rhwD9/JBCCkB+MRYDp0oSvlo2IL6rQWA10PQi7tDUM3eqMSltXmY+Oyl/7N3P3qNtAlv7X0d9bI28w=="
},
"bcrypt-pbkdf": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz",
"integrity": "sha1-pDAdOJtqQ/m2f/PKEaP2Y342Dp4=",
"optional": true,
"requires": {
"tweetnacl": "^0.14.3"
}
},
"bluebird": {
"version": "3.5.1",
"resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.5.1.tgz",
"integrity": "sha512-MKiLiV+I1AA596t9w1sQJ8jkiSr5+ZKi0WKrYGUn6d1Fx+Ij4tIj+m2WMQSGczs5jZVxV339chE8iwk6F64wjA=="
},
"caseless": {
"version": "0.12.0",
"resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz",
"integrity": "sha1-G2gcIf+EAzyCZUMJBolCDRhxUdw="
},
"co": {
"version": "4.6.0",
"resolved": "https://registry.npmjs.org/co/-/co-4.6.0.tgz",
"integrity": "sha1-bqa989hTrlTMuOR7+gvz+QMfsYQ="
},
"combined-stream": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.6.tgz",
"integrity": "sha1-cj599ugBrFYTETp+RFqbactjKBg=",
"requires": {
"delayed-stream": "~1.0.0"
}
},
"core-util-is": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz",
"integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac="
},
"dashdash": {
"version": "1.14.1",
"resolved": "https://registry.npmjs.org/dashdash/-/dashdash-1.14.1.tgz",
"integrity": "sha1-hTz6D3y+L+1d4gMmuN1YEDX24vA=",
"requires": {
"assert-plus": "^1.0.0"
}
},
"delayed-stream": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk="
},
"ecc-jsbn": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/ecc-jsbn/-/ecc-jsbn-0.1.1.tgz",
"integrity": "sha1-D8c6ntXw1Tw4GTOYUj735UN3dQU=",
"optional": true,
"requires": {
"jsbn": "~0.1.0"
}
},
"extend": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
"integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g=="
},
"extsprintf": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/extsprintf/-/extsprintf-1.3.0.tgz",
"integrity": "sha1-lpGEQOMEGnpBT4xS48V06zw+HgU="
},
"fast-deep-equal": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-1.1.0.tgz",
"integrity": "sha1-wFNHeBfIa1HaqFPIHgWbcz0CNhQ="
},
"fast-json-stable-stringify": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.0.0.tgz",
"integrity": "sha1-1RQsDK7msRifh9OnYREGT4bIu/I="
},
"forever-agent": {
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/forever-agent/-/forever-agent-0.6.1.tgz",
"integrity": "sha1-+8cfDEGt6zf5bFd60e1C2P2sypE="
},
"form-data": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-2.3.2.tgz",
"integrity": "sha1-SXBJi+YEwgwAXU9cI67NIda0kJk=",
"requires": {
"asynckit": "^0.4.0",
"combined-stream": "1.0.6",
"mime-types": "^2.1.12"
}
},
"getpass": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/getpass/-/getpass-0.1.7.tgz",
"integrity": "sha1-Xv+OPmhNVprkyysSgmBOi6YhSfo=",
"requires": {
"assert-plus": "^1.0.0"
}
},
"har-schema": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/har-schema/-/har-schema-2.0.0.tgz",
"integrity": "sha1-qUwiJOvKwEeCoNkDVSHyRzW37JI="
},
"har-validator": {
"version": "5.0.3",
"resolved": "https://registry.npmjs.org/har-validator/-/har-validator-5.0.3.tgz",
"integrity": "sha1-ukAsJmGU8VlW7xXg/PJCmT9qff0=",
"requires": {
"ajv": "^5.1.0",
"har-schema": "^2.0.0"
}
},
"http-signature": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/http-signature/-/http-signature-1.2.0.tgz",
"integrity": "sha1-muzZJRFHcvPZW2WmCruPfBj7rOE=",
"requires": {
"assert-plus": "^1.0.0",
"jsprim": "^1.2.2",
"sshpk": "^1.7.0"
}
},
"is-typedarray": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/is-typedarray/-/is-typedarray-1.0.0.tgz",
"integrity": "sha1-5HnICFjfDBsR3dppQPlgEfzaSpo="
},
"isstream": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/isstream/-/isstream-0.1.2.tgz",
"integrity": "sha1-R+Y/evVa+m+S4VAOaQ64uFKcCZo="
},
"jsbn": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/jsbn/-/jsbn-0.1.1.tgz",
"integrity": "sha1-peZUwuWi3rXyAdls77yoDA7y9RM=",
"optional": true
},
"json-schema": {
"version": "0.2.3",
"resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.2.3.tgz",
"integrity": "sha1-tIDIkuWaLwWVTOcnvT8qTogvnhM="
},
"json-schema-traverse": {
"version": "0.3.1",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-0.3.1.tgz",
"integrity": "sha1-NJptRMU6Ud6JtAgFxdXlm0F9M0A="
},
"json-stringify-safe": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz",
"integrity": "sha1-Epai1Y/UXxmg9s4B1lcB4sc1tus="
},
"jsprim": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/jsprim/-/jsprim-1.4.1.tgz",
"integrity": "sha1-MT5mvB5cwG5Di8G3SZwuXFastqI=",
"requires": {
"assert-plus": "1.0.0",
"extsprintf": "1.3.0",
"json-schema": "0.2.3",
"verror": "1.10.0"
}
},
"matrix-bot-sdk": {
"version": "0.1.10",
"resolved": "https://registry.npmjs.org/matrix-bot-sdk/-/matrix-bot-sdk-0.1.10.tgz",
"integrity": "sha512-edD7wERbEhcl6/MKy2H7h+ArCmbBBUtCRoEQvr3YdtoTSL6OxJtwEvl4FYqXqgR/juIg1m53OgZRHHSKedXByQ==",
"requires": {
"bluebird": "^3.5.1",
"request": "^2.83.0"
}
},
"mime-db": {
"version": "1.35.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.35.0.tgz",
"integrity": "sha512-JWT/IcCTsB0Io3AhWUMjRqucrHSPsSf2xKLaRldJVULioggvkJvggZ3VXNNSRkCddE6D+BUI4HEIZIA2OjwIvg=="
},
"mime-types": {
"version": "2.1.19",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.19.tgz",
"integrity": "sha512-P1tKYHVSZ6uFo26mtnve4HQFE3koh1UWVkp8YUC+ESBHe945xWSoXuHHiGarDqcEZ+whpCDnlNw5LON0kLo+sw==",
"requires": {
"mime-db": "~1.35.0"
}
},
"oauth-sign": {
"version": "0.8.2",
"resolved": "https://registry.npmjs.org/oauth-sign/-/oauth-sign-0.8.2.tgz",
"integrity": "sha1-Rqarfwrq2N6unsBWV4C31O/rnUM="
},
"performance-now": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/performance-now/-/performance-now-2.1.0.tgz",
"integrity": "sha1-Ywn04OX6kT7BxpMHrjZLSzd8nns="
},
"punycode": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/punycode/-/punycode-1.4.1.tgz",
"integrity": "sha1-wNWmOycYgArY4esPpSachN1BhF4="
},
"qs": {
"version": "6.5.2",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.5.2.tgz",
"integrity": "sha512-N5ZAX4/LxJmF+7wN74pUD6qAh9/wnvdQcjq9TZjevvXzSUo7bfmw91saqMjzGS2xq91/odN2dW/WOl7qQHNDGA=="
},
"request": {
"version": "2.87.0",
"resolved": "https://registry.npmjs.org/request/-/request-2.87.0.tgz",
"integrity": "sha512-fcogkm7Az5bsS6Sl0sibkbhcKsnyon/jV1kF3ajGmF0c8HrttdKTPRT9hieOaQHA5HEq6r8OyWOo/o781C1tNw==",
"requires": {
"aws-sign2": "~0.7.0",
"aws4": "^1.6.0",
"caseless": "~0.12.0",
"combined-stream": "~1.0.5",
"extend": "~3.0.1",
"forever-agent": "~0.6.1",
"form-data": "~2.3.1",
"har-validator": "~5.0.3",
"http-signature": "~1.2.0",
"is-typedarray": "~1.0.0",
"isstream": "~0.1.2",
"json-stringify-safe": "~5.0.1",
"mime-types": "~2.1.17",
"oauth-sign": "~0.8.2",
"performance-now": "^2.1.0",
"qs": "~6.5.1",
"safe-buffer": "^5.1.1",
"tough-cookie": "~2.3.3",
"tunnel-agent": "^0.6.0",
"uuid": "^3.1.0"
}
},
"safe-buffer": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.2.tgz",
"integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="
},
"safer-buffer": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="
},
"sshpk": {
"version": "1.14.2",
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.14.2.tgz",
"integrity": "sha1-xvxhZIo9nE52T9P8306hBeSSupg=",
"requires": {
"asn1": "~0.2.3",
"assert-plus": "^1.0.0",
"bcrypt-pbkdf": "^1.0.0",
"dashdash": "^1.12.0",
"ecc-jsbn": "~0.1.1",
"getpass": "^0.1.1",
"jsbn": "~0.1.0",
"safer-buffer": "^2.0.2",
"tweetnacl": "~0.14.0"
}
},
"tough-cookie": {
"version": "2.3.4",
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-2.3.4.tgz",
"integrity": "sha512-TZ6TTfI5NtZnuyy/Kecv+CnoROnyXn2DN97LontgQpCwsX2XyLYCC0ENhYkehSOwAp8rTQKc/NUIF7BkQ5rKLA==",
"requires": {
"punycode": "^1.4.1"
}
},
"tunnel-agent": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
"integrity": "sha1-J6XeoGs2sEoKmWZ3SykIaPD8QP0=",
"requires": {
"safe-buffer": "^5.0.1"
}
},
"tweetnacl": {
"version": "0.14.5",
"resolved": "https://registry.npmjs.org/tweetnacl/-/tweetnacl-0.14.5.tgz",
"integrity": "sha1-WuaBd/GS1EViadEIr6k/+HQ/T2Q=",
"optional": true
},
"uuid": {
"version": "3.3.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-3.3.2.tgz",
"integrity": "sha512-yXJmeNaw3DnnKAOKJE51sL/ZaYfWJRl1pK9dr19YFCu0ObS231AB1/LbqTKRAQ5kw8A90rA6fr4riOUpTZvQZA=="
},
"verror": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/verror/-/verror-1.10.0.tgz",
"integrity": "sha1-OhBcoXBTr1XW4nDB+CiGguGNpAA=",
"requires": {
"assert-plus": "^1.0.0",
"core-util-is": "1.0.2",
"extsprintf": "^1.2.0"
}
}
}
}
{
"name": "matrix-spam-redact",
"version": "1.0.0",
"description": "Redacts spam in matrix",
"main": "index.js",
"author": "Travis Ralston",
"license": "MIT",
"dependencies": {
"matrix-bot-sdk": "^0.1.10"
}
}
@krombel
Copy link

krombel commented Aug 2, 2018

@turt2live: Thanks for providing this.
I wanted to use this with my relatively big account so introduced a filter. It might be a bigger than needed but as the sdk as a first step sends it to the server and continues using a filter id it should not be a problem). Hopefully that is of help for sb.

I changed

monitor.start().then(() => console.log("Monitor account started"));

to

monitor.start({
        "account_data": {"not_types": ["*"]},
        "presence": {"not_types": ["*"]},
        "room": {
                "account_data": {"not_types": ["*"]},
                "ephemeral": {"not_types": ["*"]},
                "timeline": {
                        "rooms": config['room_ids'],
                        "types": ["m.room.message"]
                },
                "state": {"not_types": ["*"]}
        }
}).then(() => console.log("Monitor account started"));

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment