Skip to content

Instantly share code, notes, and snippets.

Malcolm Greaves malcolmgreaves

Block or report user

Report or block malcolmgreaves

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@malcolmgreaves
malcolmgreaves / programs.sh
Created Jun 16, 2019
General bash functions
View programs.sh
# Exectuable scripts
PROGRAMS="$HOME/programs/py:$HOME/programs/sh"
export PATH="${PATH}:${PROGRAMS}"
# Programs
port-pid() {
PORT="$1"
@malcolmgreaves
malcolmgreaves / keras_custom_metrics_f1_precision_recall.py
Last active Jun 14, 2019
Snippets for using custom metrics -- f1, precision, and recall -- in a Keras model.
View keras_custom_metrics_f1_precision_recall.py
import numpy as np
import tensorflow as tf
from keras import Input, Model
from keras import backend as K
from keras.engine import InputLayer
from keras.layers import Dense, Dropout, Embedding, LSTM, Bidirectional, Lambda
from keras.models import Sequential
from keras.preprocessing import sequence
@malcolmgreaves
malcolmgreaves / SelfAttention.py
Created Jun 14, 2019 — forked from cbaziotis/SelfAttention.py
SelfAttention implementation in PyTorch
View SelfAttention.py
class SelfAttention(nn.Module):
def __init__(self, attention_size, batch_first=False, non_linearity="tanh"):
super(SelfAttention, self).__init__()
self.batch_first = batch_first
self.attention_weights = Parameter(torch.FloatTensor(attention_size))
self.softmax = nn.Softmax(dim=-1)
if non_linearity == "relu":
self.non_linearity = nn.ReLU()
@malcolmgreaves
malcolmgreaves / either.py
Last active May 22, 2019
A value-or-error disjunction. Inspired by \/ in Scala.
View either.py
from typing import TypeVar, Generic, Optional, Callable, cast
V = TypeVar("V")
E = TypeVar("E")
T = TypeVar("T")
class ValueOrError(Generic[V, E]):
"""A disjunction for holding either a value of type V or an error of type E.
"""
@malcolmgreaves
malcolmgreaves / key_transfer_learning_insight.py
Created Apr 30, 2019
Key transfer learning insight: when adapating a pre-trained DL model, one must freeze the inner network and change the input & output layers, then train on new data. The fine-tuning is all w/in the _new_ (unfrozen!) I/O layers.
View key_transfer_learning_insight.py
# https://github.com/intel-analytics/analytics-zoo#transfer-learning
from zoo.pipeline.api.net import *
from zoo.pipeline.api.keras.layers import *
from zoo.pipeline.api.keras.models import *
def transfer_learn_model(def_path:str, model_path:str) -> Model:
# load the pre-trained model
full_model = Net.load_caffe(def_path, model_path)
@malcolmgreaves
malcolmgreaves / reflect.py
Created Apr 27, 2019
Playground for Python reflection & runtime object inspection.
View reflect.py
import inspect
from typing import Any, Sequence
DEFAULT_TERMINAL_TYPES: Sequence[type] = (str, int, float, list, dict)
"""These are types that should be considered terminal cases in value inspection.
I.e. once reached in the descent of an object's attributes, inspection
should terminate the current inspection path. These are common, built-ins that
are considered default terminal types.
"""
@malcolmgreaves
malcolmgreaves / multi_input_keras_model_tf_serving_save.py
Created Apr 5, 2019
Example showing the a multi-input node Keras model that's trained on toy data and saved as a TensorFlow Serving model artifact.
View multi_input_keras_model_tf_serving_save.py
import os
import sys
import numpy as np
import tensorflow as tf
from tensorflow.python.keras import Input, Model
from tensorflow.python.keras.layers import Dense, concatenate
View PomToSbt.scala
object PomToSbt extends App {
import scala.xml._
val lines = (XML.load("pom.xml") \\ "dependencies") \ "dependency" map { dependency =>
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
@malcolmgreaves
malcolmgreaves / file_list.py
Created Dec 19, 2018
List all files in directory.
View file_list.py
import os
def list_files(directory: str) -> Iterable[str]:
for dirpath, dirnames, filenames in os.walk(directory):
for f in filenames:
fpath = os.path.join(dirpath, f)
if os.path.isfile(fpath):
yield fpath
@malcolmgreaves
malcolmgreaves / data_typeclass.py
Last active Dec 14, 2018
Data type class for Python's Iterable & pySpark's RDD. Inspired by the Scala Data typeclass: https://github.com/malcolmgreaves/data-tc
View data_typeclass.py
from collections import defaultdict
from typing import TypeVar, Generic, Iterable, Union, Callable, Sequence, Optional, Iterator, Tuple
from pyspark.rdd import RDD
A = TypeVar('A')
B = TypeVar('B')
You can’t perform that action at this time.