Skip to content

Instantly share code, notes, and snippets.

View mcgill-a's full-sized avatar

Alex McGill mcgill-a

5 followers · 3 following
  • Scott Logic
  • Edinburgh
View GitHub Profile
@mcgill-a
mcgill-a / process_csv.py
Created May 11, 2020 14:25
Process data from a CSV file to merge comments that span over multiple pages
import sys
import csv
def load_data(filename):
with open(filename, 'r', encoding='UTF-8') as file:
reader = csv.reader(file)
for row in reader:
a.append(row[0])
b.append(row[1])
@mcgill-a
mcgill-a / get_pdf.py
Last active May 20, 2020 11:43
Download and extract ZIP files from URLs in a CSV file
import sys
import csv
from tqdm import tqdm
import requests, zipfile, io
def load_data(filename):
arr = []
with open(filename, 'r', encoding='UTF-8') as file:
reader = csv.reader(file)
@mcgill-a
mcgill-a / separate_companies.py
Last active June 29, 2020 11:04
Separate fields with multiple companies in the global coal plant tracker data exported from https://alexmcgill.net/research/coal_tracker
import sys
import csv
import argparse
import re
from pathlib import Path
def load_data(filename):
data = []
with open(filename, 'r', encoding='latin-1') as file:
@mcgill-a
mcgill-a / get_files.py
Created June 15, 2020 16:10
Download, unzip, and categorise the files scraped from the Environmental Impact Statement (EIS) Database exported to a CSV file
import sys, csv, requests, zipfile, io, re
from tqdm import tqdm
from pathlib import Path
def load_data(filename):
arr = []
with open(filename, 'r', encoding='UTF-8') as file:
reader = csv.reader(file)
for row in reader:
@mcgill-a
mcgill-a / get_laws.py
Created June 26, 2020 12:48
Extract laws from any textual PDF document within a specific directory + sub directories
import sys
import csv
import argparse
import re
import io
import os
import os.path
from tqdm import tqdm
from pathlib import Path
from pdfminer.converter import TextConverter
@mcgill-a
mcgill-a / link_investors.py
Last active July 4, 2020 17:22
Generate a CSV that links investors with companies they invest in from a list of target companies
@mcgill-a
mcgill-a / index.ts
Created September 25, 2023 09:31
A minimal backend controller
userService.change.subscribe((change) =>
io.sockets.emit(SocketChannel.user, JSON.stringify(change))
);
app.post("/users/join", (req, res) => {
try {
res.json(userService.join(<User>req.body));
} catch (error) {
res.status(400).send(error);
}
@mcgill-a
mcgill-a / user.service.ts
Last active September 25, 2023 14:57
A minimal backend user service
export class UserService {
private _change = new Subject<UserChange>();
private _users = new Map<string, User>();
public readonly change = this._change.asObservable();
public get users(): User[] {
return Array.from(this._users.values());
}
@mcgill-a
mcgill-a / socket.service.ts
Last active September 25, 2023 12:36
A minimal frontend service for listening to web socket changes in Angular
@Injectable({ providedIn: 'root' })
export class SocketService {
constructor(private socket: Socket) {}
public readonly userChange = this.socket
.fromEvent<string>(SocketChannel.user)
.pipe(map((x) => JSON.parse(x) as UserChange));
}
@mcgill-a
mcgill-a / user.service.ts
Created September 25, 2023 12:51
A minimal frontend service for updating user data in Angular
@Injectable({ providedIn: 'root' })
constructor(private socketService: SocketService) {
this.subs.add(
this.socketService.userChange.subscribe((change: UserChange) => {
const {user, action} = change;
switch (change.action) {
case UserAction.join:
case TeamAction.position:
this._users.set(user.email, user);