Skip to content

Instantly share code, notes, and snippets.

Malcolm Greaves malcolmgreaves

Block or report user

Report or block malcolmgreaves

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
malcolmgreaves /
Created Jun 16, 2019
General bash functions
# Exectuable scripts
export PATH="${PATH}:${PROGRAMS}"
# Programs
port-pid() {
malcolmgreaves /
Last active Jun 14, 2019
Snippets for using custom metrics -- f1, precision, and recall -- in a Keras model.
import numpy as np
import tensorflow as tf
from keras import Input, Model
from keras import backend as K
from keras.engine import InputLayer
from keras.layers import Dense, Dropout, Embedding, LSTM, Bidirectional, Lambda
from keras.models import Sequential
from keras.preprocessing import sequence
malcolmgreaves /
Created Jun 14, 2019 — forked from cbaziotis/
SelfAttention implementation in PyTorch
class SelfAttention(nn.Module):
def __init__(self, attention_size, batch_first=False, non_linearity="tanh"):
super(SelfAttention, self).__init__()
self.batch_first = batch_first
self.attention_weights = Parameter(torch.FloatTensor(attention_size))
self.softmax = nn.Softmax(dim=-1)
if non_linearity == "relu":
self.non_linearity = nn.ReLU()
malcolmgreaves /
Last active May 22, 2019
A value-or-error disjunction. Inspired by \/ in Scala.
from typing import TypeVar, Generic, Optional, Callable, cast
V = TypeVar("V")
E = TypeVar("E")
T = TypeVar("T")
class ValueOrError(Generic[V, E]):
"""A disjunction for holding either a value of type V or an error of type E.
malcolmgreaves /
Created Apr 30, 2019
Key transfer learning insight: when adapating a pre-trained DL model, one must freeze the inner network and change the input & output layers, then train on new data. The fine-tuning is all w/in the _new_ (unfrozen!) I/O layers.
from import *
from zoo.pipeline.api.keras.layers import *
from zoo.pipeline.api.keras.models import *
def transfer_learn_model(def_path:str, model_path:str) -> Model:
# load the pre-trained model
full_model = Net.load_caffe(def_path, model_path)
malcolmgreaves /
Created Apr 27, 2019
Playground for Python reflection & runtime object inspection.
import inspect
from typing import Any, Sequence
DEFAULT_TERMINAL_TYPES: Sequence[type] = (str, int, float, list, dict)
"""These are types that should be considered terminal cases in value inspection.
I.e. once reached in the descent of an object's attributes, inspection
should terminate the current inspection path. These are common, built-ins that
are considered default terminal types.
malcolmgreaves /
Created Apr 5, 2019
Example showing the a multi-input node Keras model that's trained on toy data and saved as a TensorFlow Serving model artifact.
import os
import sys
import numpy as np
import tensorflow as tf
from tensorflow.python.keras import Input, Model
from tensorflow.python.keras.layers import Dense, concatenate
View PomToSbt.scala
object PomToSbt extends App {
import scala.xml._
val lines = (XML.load("pom.xml") \\ "dependencies") \ "dependency" map { dependency =>
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
malcolmgreaves /
Created Dec 19, 2018
List all files in directory.
import os
def list_files(directory: str) -> Iterable[str]:
for dirpath, dirnames, filenames in os.walk(directory):
for f in filenames:
fpath = os.path.join(dirpath, f)
if os.path.isfile(fpath):
yield fpath
malcolmgreaves /
Last active Dec 14, 2018
Data type class for Python's Iterable & pySpark's RDD. Inspired by the Scala Data typeclass:
from collections import defaultdict
from typing import TypeVar, Generic, Iterable, Union, Callable, Sequence, Optional, Iterator, Tuple
from pyspark.rdd import RDD
A = TypeVar('A')
B = TypeVar('B')
You can’t perform that action at this time.