Skip to content

Instantly share code, notes, and snippets.

👨‍🔬

Pedro Rodriguez EntilZha

👨‍🔬
View GitHub Profile
@EntilZha
EntilZha / export_acl_visited_papers.js
Last active Jul 7, 2020
Export a list of visited/favorited papers from virtual.acl2020.org
View export_acl_visited_papers.js
// allpapers is stored in a cookie
for (p of allPapers) {
if (p.content.read) {
// Chop off the .pdf
console.log(p.content.pdf_url.slice(0, -4))
}
}
// Improved version that doesn't have line numbers
console.log(allPapers.filter(p => p.content.read).map(p => p.content.pdf_url.slice(0, -4)).join('\n'))
View comet_ml_callback.py
from typing import Text
import socket
import os
import comet_ml
import toml
from allennlp.training.callbacks.callback import Callback, handle_event
from allennlp.training.callbacks.events import Events
@Callback.register("log_to_comet")
View environment.yaml
name: qb
dependencies:
- python=3.7
- pytorch=1.4.*
- cudatoolkit=10.1
- numpy
- scipy
- pandas=1.0.*
- requests
- flask
@EntilZha
EntilZha / model.log
Created Dec 16, 2019
Example of AllenNLP Logging
View model.log
$ allennlp make-vocab --include-package zero -s /tmp/stuff config/manual/qanta/ai2.jsonnet (zero)
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
2019-12-16 18:11:49,080 - INFO - allennlp.common.params - random_seed = 13370
2019-12-16 18:11:49,081 - INFO - allennlp.common.params - numpy_seed = 1337
2019-12-16 18:11:49,081 - INFO - allennlp.common.params - pytorch_seed = 133
2019-12-16 18:11:49,182 - INFO - allennlp.common.checks - Pytorch version: 1.0.1
2019-12-16 18:11:49,185 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.data.dataset_readers.dataset_reader.DatasetReader'> from params {'evidence': True, 'lazy': False, 'type': 'qanta_rc'} and extras {}
2019-12-16 18:11:49,185 - INFO - allennlp.common.params - dataset_reader.type = qanta_rc
2019-12-16 18:11:49,186 - INFO - allennlp.common.from_params - instantiating class <class 'zero.datasets.qanta.
@EntilZha
EntilZha / qanta.py
Created Nov 14, 2019
AllenNLP Reader for Qanta Dataset
View qanta.py
from typing import Dict, List, Union
import json
from overrides import overrides
from allennlp.data import DatasetReader, TokenIndexer, Instance
from allennlp.data.fields import TextField, LabelField, Field, MetadataField, ArrayField, ListField
from allennlp.data.token_indexers import SingleIdTokenIndexer, TokenCharactersIndexer
from allennlp.data.tokenizers import Tokenizer, WordTokenizer, Token
from allennlp.data.tokenizers.word_splitter import SpacyWordSplitter, WordSplitter
View backtrace
(gdb) thread apply all backtrace
Thread 17 (Thread 0x7fc90f3ff700 (LWP 3228)):
#0 0x00007fc90fe56945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00005563e30ed814 in rayon_core::sleep::Sleep::sleep::h403e051017a83b73 ()
#2 0x00005563e30ebd0e in rayon_core::registry::WorkerThread::wait_until_cold::h4c91d94806702f48 ()
#3 0x00005563e30ec54b in rayon_core::registry::main_loop::hbbba263316bb2911 ()
#4 0x00005563e30ed17c in std::panicking::try::do_call::h8a19372e663d596b ()
#5 0x00005563e310c48f in __rust_maybe_catch_panic () at /checkout/src/libpanic_unwind/lib.rs:101
#6 0x00005563e30e983e in _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h24e52bc8c236002e ()
@EntilZha
EntilZha / guess.py
Created Nov 9, 2017
Running Qanta Models
View guess.py
from qanta.guesser.tfidf import TfidfGuesser
guesser = TfidfGuesser.load('output/guesser/qanta.guesser.tfidf.TfidfGuesser')
questions = [
"Name this first president of the united states",
"This man invented the theory of general relativity"
]
n_guesses = 1
View gist:06a8bbd2279d01a61a1c7a5e61d882b4
In [9]: model.predict(np.random.random((10, 3, 2)))
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
/home/pedro/anaconda3/lib/python3.6/site-pac
View Cargo.toml
[package]
name = "preferential_attachment"
version = "0.1.0"
[dependencies]
rand = "0.3"
View Results.txt
dummy_element_consumer
data_size: 100, passes: 100
grouper_it_0 3 0.003933991072699428
grouper_it_1 3 0.0036308339331299067
grouper_it_2 3 0.0039052229840308428
grouper_impl 3 0.0020833380986005068
grouper_it_0 10 0.0014350449200719595
grouper_it_1 10 0.0015790120232850313
grouper_it_2 10 0.0019499310292303562
grouper_impl 10 0.001163481967523694
You can’t perform that action at this time.