Skip to content

Instantly share code, notes, and snippets.

View sujitpal's full-sized avatar

Sujit Pal sujitpal

505 followers · 0 following
View GitHub Profile
@sujitpal
sujitpal / 01-understanding_ppi.ipynb
Last active March 22, 2024 19:17
Understanding Prediction Powered Inference (PPI) used in ARES
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / sparknlp-stack-trace.txt
Created February 18, 2022 20:12
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 12.0 failed 4 times, most recent failure: Lost task 0.3 in stage 12.0 (TID 31) (10.0.230.73 executor 0): com.databricks.sql.io.FileReadException: Error while reading file dbfs:/mnt/els/labs/projects/sparknlp-cm3-test/inputs/sparknlp-input.tsv.
at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1$$anon$2.logFileNameAndThrow(FileScanRDD.scala:396)
at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1$$anon$2.getNext(FileScanRDD.scala:369)
at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.$anonfun$hasNext$1(FileScanRDD.scala:301)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at com.databricks.spark.util.ExecutorFrameProfiler$.record(ExecutorFrameProfiler.scala:110)
at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:296)
at scala.collection.Iterato
@sujitpal
sujitpal / 13-bayesian_nn.ipynb
Created July 31, 2021 16:12
Bayesian Network example (unrolled) from NYU Pytorch DL course -- adapted from atcold/pytorch-Deep-Learning
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / 12-regularization.ipynb
Last active July 31, 2021 16:10
Regularization Notebook from DL-Pytorch NYU course (unrolled) - adapted from atcold/Pytorch-Deep-Learning
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / 10a-autoencoder.ipynb
Created April 3, 2021 19:34
Unrolled 10-autoencoder.ipynb + Contractive AE adapted from atcold/pytorch-Deep-Learning
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / 07b-viterbi-gist.py
Created August 6, 2020 20:25
Entity Disambiguation for entities identified for a sentence by SciSpacy + UMLS integration using Viterbi's algorithm
import argparse
import itertools
import numpy as np
import operator
import os
import pickle
import spacy
import scispacy
import time
@sujitpal
sujitpal / inexact-search-aho-corasick.ipynb
Created April 20, 2020 01:46
Code demonstrating building and querying an Aho-Corasick FSM for inexact search
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / 02-symptom-checker-experiments.ipynb
Last active April 6, 2020 01:42
Symptom Checker and Other Experiments -- notebook using dataset from Nigam Shah's lab
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / bow-vs-sif-embedding-for-text-classification.ipynb
Created May 20, 2017 21:49
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sujitpal
sujitpal / babi_memnn.py
Last active October 3, 2016 01:21
My modified version of keras/examples/babi_memnn.py using the new functional API instead of Sequential API (changes between comment lines "construct network" and "compile network"). Originally driven by keras issue 3927.
'''Trains a memory network on the bAbI dataset.
References:
- Jason Weston, Antoine Bordes, Sumit Chopra, Tomas Mikolov, Alexander M. Rush,
"Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks",
http://arxiv.org/abs/1502.05698
- Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus,
"End-To-End Memory Networks",
http://arxiv.org/abs/1503.08895