Skip to content

Instantly share code, notes, and snippets.

View EntilZha's full-sized avatar
👨‍🔬

Pedro Rodriguez EntilZha

👨‍🔬
View GitHub Profile
@EntilZha
EntilZha / export_acl_visited_papers.js
Last active July 7, 2020 14:31
Export a list of visited/favorited papers from virtual.acl2020.org
// allpapers is stored in a cookie
for (p of allPapers) {
if (p.content.read) {
// Chop off the .pdf
console.log(p.content.pdf_url.slice(0, -4))
}
}
// Improved version that doesn't have line numbers
console.log(allPapers.filter(p => p.content.read).map(p => p.content.pdf_url.slice(0, -4)).join('\n'))
from typing import Text
import socket
import os
import comet_ml
import toml
from allennlp.training.callbacks.callback import Callback, handle_event
from allennlp.training.callbacks.events import Events
@Callback.register("log_to_comet")
name: qb
dependencies:
- python=3.7
- pytorch=1.4.*
- cudatoolkit=10.1
- numpy
- scipy
- pandas=1.0.*
- requests
- flask
@EntilZha
EntilZha / model.log
Created December 16, 2019 23:13
Example of AllenNLP Logging
$ allennlp make-vocab --include-package zero -s /tmp/stuff config/manual/qanta/ai2.jsonnet (zero)
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
2019-12-16 18:11:49,080 - INFO - allennlp.common.params - random_seed = 13370
2019-12-16 18:11:49,081 - INFO - allennlp.common.params - numpy_seed = 1337
2019-12-16 18:11:49,081 - INFO - allennlp.common.params - pytorch_seed = 133
2019-12-16 18:11:49,182 - INFO - allennlp.common.checks - Pytorch version: 1.0.1
2019-12-16 18:11:49,185 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.data.dataset_readers.dataset_reader.DatasetReader'> from params {'evidence': True, 'lazy': False, 'type': 'qanta_rc'} and extras {}
2019-12-16 18:11:49,185 - INFO - allennlp.common.params - dataset_reader.type = qanta_rc
2019-12-16 18:11:49,186 - INFO - allennlp.common.from_params - instantiating class <class 'zero.datasets.qanta.
@EntilZha
EntilZha / qanta.py
Created November 14, 2019 00:02
AllenNLP Reader for Qanta Dataset
from typing import Dict, List, Union
import json
from overrides import overrides
from allennlp.data import DatasetReader, TokenIndexer, Instance
from allennlp.data.fields import TextField, LabelField, Field, MetadataField, ArrayField, ListField
from allennlp.data.token_indexers import SingleIdTokenIndexer, TokenCharactersIndexer
from allennlp.data.tokenizers import Tokenizer, WordTokenizer, Token
from allennlp.data.tokenizers.word_splitter import SpacyWordSplitter, WordSplitter
(gdb) thread apply all backtrace
Thread 17 (Thread 0x7fc90f3ff700 (LWP 3228)):
#0 0x00007fc90fe56945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00005563e30ed814 in rayon_core::sleep::Sleep::sleep::h403e051017a83b73 ()
#2 0x00005563e30ebd0e in rayon_core::registry::WorkerThread::wait_until_cold::h4c91d94806702f48 ()
#3 0x00005563e30ec54b in rayon_core::registry::main_loop::hbbba263316bb2911 ()
#4 0x00005563e30ed17c in std::panicking::try::do_call::h8a19372e663d596b ()
#5 0x00005563e310c48f in __rust_maybe_catch_panic () at /checkout/src/libpanic_unwind/lib.rs:101
#6 0x00005563e30e983e in _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h24e52bc8c236002e ()
@EntilZha
EntilZha / guess.py
Created November 9, 2017 17:41
Running Qanta Models
from qanta.guesser.tfidf import TfidfGuesser
guesser = TfidfGuesser.load('output/guesser/qanta.guesser.tfidf.TfidfGuesser')
questions = [
"Name this first president of the united states",
"This man invented the theory of general relativity"
]
n_guesses = 1
In [9]: model.predict(np.random.random((10, 3, 2)))
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
/home/pedro/anaconda3/lib/python3.6/site-pac
[package]
name = "preferential_attachment"
version = "0.1.0"
[dependencies]
rand = "0.3"
dummy_element_consumer
data_size: 100, passes: 100
grouper_it_0 3 0.003933991072699428
grouper_it_1 3 0.0036308339331299067
grouper_it_2 3 0.0039052229840308428
grouper_impl 3 0.0020833380986005068
grouper_it_0 10 0.0014350449200719595
grouper_it_1 10 0.0015790120232850313
grouper_it_2 10 0.0019499310292303562
grouper_impl 10 0.001163481967523694