Skip to content

Instantly share code, notes, and snippets.

View farizrahman4u's full-sized avatar
🏠
Working from home

Fariz Rahman farizrahman4u

🏠
Working from home
View GitHub Profile
import numpy as np
__author__ = 'Fariz Rahman'
def eq(x, y):
return x.lower().replace(" ", "") == y.lower().replace(" ", "")
def get_words(x):
x = x.replace(" ", " ")
from keras.layers import Recurrent
from keras.models import Sequential
from keras import backend as K
def _isRNN(layer):
return issubclass(layer.__class__, Recurrent)
def _zeros(shape):
import numpy as np
def vectorize(x):
# vectorize a string
if len(x) > 1:
return np.sum([vectorize(c) for c in x], axis=0)
if x == '.':
i = 27
elif x == ' ':
i = 26
import numpy as np
from simple_classifier import SimpleClassifier
def vectorize(x):
# vectorize a string
if len(x) > 1:
return np.sum([vectorize(c) for c in x], axis=0)
if x == '.':
i = 27
elif x == ' ':
@farizrahman4u
farizrahman4u / pom.xml
Last active February 19, 2018 11:28
dl4j-maven
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.deeplearning4j</groupId>
<artifactId>dl4j</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>dl4j</name>
<url>http://maven.apache.org</url>
@farizrahman4u
farizrahman4u / jumpy_bug.py
Created February 19, 2018 21:50
Jumpy bug
import jumpy
import numpy as np
jumpy.init()
def numpy(jp_array):
# convert back to numpy array
array = jp_array.array # INDArray instance
get = array.getDouble
shape = array.shape()
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] nd4j
[INFO] nd4j-shade
[INFO] jackson
[INFO] nd4j-common
[INFO] nd4j-context
[INFO] nd4j-buffer
  • On the Java end, exposing an op from libnd4j involves adding classes and methods to 3 different places. We should consider having a universal function/op factory.

  • Blur the lines between Nd4j/samediff. All the ops from the universal function factory mentioned above should be available under Nd4j namespace. In other words, Nd4j.some_op should work on both INDArray as well as SDVariable inputs.

  • More concrete shape inference. Shape inference in Samediff seems to be flaky, and dependent on op execution. Shape inference should be greedy with provision for unknown dimensions (this would be more involved and require changes in libnd4j). Should also consider symbolic shapes, i.e, SDVariable.shape() would return a 1-d SDVariable.

[100%] Linking CXX shared library libnd4jcpu.dll
cd /C/libnd4j/blasbuild/cpu/blas && /C/msys64/mingw64/bin/cmake.exe -E remove -f CMakeFiles/nd4jcpu.dir/objects.a
cd /C/libnd4j/blasbuild/cpu/blas && /C/msys64/mingw64/bin/ar.exe cr CMakeFiles/nd4jcpu.dir/objects.a "CMakeFiles/nd4jcpu.dir/cpu/NativeOps.cpp.obj" "CMakeFiles/nd4jcpu.dir/cpu/GraphExecutioner.cpp.obj" "CMak eFiles/nd4jcpu.dir/cpu/NativeOpExcutioner.cpp.obj" "CMakeFiles/nd4jcpu.dir/cpu/NDArray.cpp.obj" "CMakeFiles/nd4jcpu.dir/cpu/NDArrayFactory.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include/cnpy/cnpy.cpp.obj" "CMak eFiles/nd4jcpu.dir/Environment.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include/loops/cpu/broadcasting.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include/loops/cpu/indexreduce.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include /loops/cpu/random.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include/loops/cpu/reduce.cpp.obj" "CMakeFiles/nd4jcpu.dir/__/include/loops/cpu/scalar.cpp.obj" "
import json
with open('ner_dataset.csv', 'r') as f:
lines = f.readlines()
lines.pop(0)
lines.pop(0)
sentences = []