Skip to content

Instantly share code, notes, and snippets.

View Eshan-Agarwal's full-sized avatar
🏠
Work From Home

Eshan Agarwal Eshan-Agarwal

🏠
Work From Home
View GitHub Profile
@Eshan-Agarwal
Eshan-Agarwal / qmnist_dataset_info.json
Created February 28, 2020 07:41
This is qmnist dataset information stored in json file
{
"citation": "\n @article{DBLP:journals/corr/abs-1905-10498,\n author = {Chhavi Yadav and\n L{'{e}}on Bottou},\n title = {Cold Case: The Lost {MNIST} Digits},\n journal = {CoRR},\n volume = {abs/1905.10498},\n year = {2019},\n url = {http://arxiv.org/abs/1905.10498},\n archivePrefix = {arXiv},\n eprint = {1905.10498},\n timestamp = {Mon, 03 Jun 2019 13:42:33 +0200},\n biburl = {https://dblp.org/rec/journals/corr/abs-1905-10498.bib},\n bibsource = {dblp computer science bibliography, https://dblp.org}\n}\n",
"description": "\nThe QMNIST dataset was generated from the original data found in the NIST Special Database 19\nwith the goal to match the MNIST preprocessing as closely as possible.\nThe exact preprocessing steps used to construct the MNIST dataset have long been lost. \nThis leaves us with no reliable way to associate its characters with the ID of the writer \nand little hope to recover the full MNIST testing set that had 60K images but was nev
@Eshan-Agarwal
Eshan-Agarwal / dataset_info.json
Created March 23, 2020 21:39
dataset_info.json for scientific_papers datasets
{
"citation": "\n@article{Cohan_2018,\n title={A Discourse-Aware Attention Model for Abstractive Summarization of\n Long Documents},\n url={http://dx.doi.org/10.18653/v1/n18-2097},\n DOI={10.18653/v1/n18-2097},\n journal={Proceedings of the 2018 Conference of the North American Chapter of\n the Association for Computational Linguistics: Human Language\n Technologies, Volume 2 (Short Papers)},\n publisher={Association for Computational Linguistics},\n author={Cohan, Arman and Dernoncourt, Franck and Kim, Doo Soon and Bui, Trung and Kim, Seokhwan and Chang, Walter and Goharian, Nazli},\n year={2018}\n}\n",
"description": "\nScientific papers datasets contains two sets of long and structured documents.\nThe datasets are obtained from ArXiv and PubMed OpenAccess repositories.\n\nBoth \"arxiv\" and \"pubmed\" have two features:\n - article: the body of the document, pagragraphs seperated by \"/n\".\n - abstract: the abstract of the document, pagragraphs seperated by
@Eshan-Agarwal
Eshan-Agarwal / dataset_info.json
Created April 2, 2020 06:53
dataset_info.json for Stanford_dogs.py
{
"citation": "@inproceedings{KhoslaYaoJayadevaprakashFeiFei_FGVC2011,\nauthor = \"Aditya Khosla and Nityananda Jayadevaprakash and Bangpeng Yao and\n Li Fei-Fei\",\ntitle = \"Novel Dataset for Fine-Grained Image Categorization\",\nbooktitle = \"First Workshop on Fine-Grained Visual Categorization,\n IEEE Conference on Computer Vision and Pattern Recognition\",\nyear = \"2011\",\nmonth = \"June\",\naddress = \"Colorado Springs, CO\",\n}\n@inproceedings{imagenet_cvpr09,\n AUTHOR = {Deng, J. and Dong, W. and Socher, R. and Li, L.-J. and\n Li, K. and Fei-Fei, L.},\n TITLE = {{ImageNet: A Large-Scale Hierarchical Image Database}},\n BOOKTITLE = {CVPR09},\n YEAR = {2009},\n BIBSOURCE = \"http://www.image-net.org/papers/imagenet_cvpr09.bib\"}\n",
"description": "The Stanford Dogs dataset contains images of 120 breeds of dogs from around\nthe world. This dataset has been built using images and annotation from\nImageNet for the task of fine-
@Eshan-Agarwal
Eshan-Agarwal / registered_test.txt
Created April 4, 2020 09:11
Test output of registered_test.py after adding unitests
2020-04-04 09:05:14.877488: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
Running tests under Python 3.6.9: /usr/bin/python3
[ RUN ] RegisteredTest.test_abstract
[ OK ] RegisteredTest.test_abstract
[ RUN ] RegisteredTest.test_builder_fullname
[ OK ] RegisteredTest.test_builder_fullname
[ RUN ] RegisteredTest.test_builder_with_kwargs
[ OK ] RegisteredTest.test_builder_with_kwargs
[ RUN ] RegisteredTest.test_duplicate_dataset
[ OK ] RegisteredTest.test_duplicate_dataset
@Eshan-Agarwal
Eshan-Agarwal / Logs.txt
Created April 6, 2020 17:24
Traceback for subsplit
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-6-ba8aad3a0131> in <module>()
3 s1, s2, s3 = tfds.Split.TRAIN.subsplit(weighted = [2, 1, 1])
4
----> 5 dataset,info= tfds.load('mnist',with_info=True, split=s1)
13 frames
/content/datasets/tensorflow_datasets/core/tfrecords_reader.py in _str_to_relative_instruction(spec)
355 res = _SUB_SPEC_RE.match(spec)
@Eshan-Agarwal
Eshan-Agarwal / stacktrace.txt
Created April 17, 2020 19:41
Stacktrace for caltech_birds.py
AssertionError Traceback (most recent call last)
<ipython-input-1-0cac24b5abed> in <module>
1 import tensorflow_datasets as tfds
2
----> 3 ds, ds_info = tfds.load("caltech_birds2011", with_info=True)
4 print(ds_info)
~\Desktop\TF_TFDS\datasets\tensorflow_datasets\core\api_utils.py in disallow_positional_args_dec(fn, instance, args, kwargs)
51 _check_no_positional(fn, args, ismethod, allowed=allowed)
52 _check_required(fn, kwargs)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/IPython/core/history.py", line 786, in writeout_cache
self._writeout_input_cache(conn)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/history.py", line 770, in _writeout_input_cache
(self.session_number,)+line)
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140422523496192 and this is thread id 140424189716352.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/IPython/core/history.py", line 786, in writeout_cache
[Errno 2] No such file or directory: 'datasets'
/content/datasets
Downloading and preparing dataset nsynth/gansynth_subset/2.3.2 (download: 73.08 GiB, generated: 20.73 GiB, total: 93.80 GiB) to gs://beamm/nsynth/gansynth_subset/2.3.2...
WARNING:absl:Dataset nsynth is hosted on GCS. It will automatically be downloaded to your
local data directory. If you'd instead prefer to read directly from our public
GCS bucket (recommended if you're running on GCP), you can instead pass
`try_gcs=True` to `tfds.load` or set `data_dir=gs://tfds-data/datasets`.
********************************************************************************
prefix : datasets/nsynth/gansynth_subset/2.3.2
@Eshan-Agarwal
Eshan-Agarwal / dataset_info.json
Created April 29, 2020 19:33
dataset_info for horses_or_humans
{
"citation": "@ONLINE {horses_or_humans,\nauthor = \"Laurence Moroney\",\ntitle = \"Horses or Humans Dataset\",\nmonth = \"feb\",\nyear = \"2019\",\nurl = \"http://laurencemoroney.com/horses-or-humans-dataset\"\n}",
"description": "A large set of images of horses and humans.",
"downloadSize": "161055054",
"location": {
"urls": [
"http://laurencemoroney.com/horses-or-humans-dataset"
]
},
"name": "horses_or_humans",
@Eshan-Agarwal
Eshan-Agarwal / dataset_info.json
Created April 29, 2020 19:34
dataset_info for rock_paper_scissors
{
"citation": "@ONLINE {rps,\nauthor = \"Laurence Moroney\",\ntitle = \"Rock, Paper, Scissors Dataset\",\nmonth = \"feb\",\nyear = \"2019\",\nurl = \"http://laurencemoroney.com/rock-paper-scissors-dataset\"\n}",
"description": "Images of hands playing rock, paper, scissor game.",
"downloadSize": "230198979",
"location": {
"urls": [
"http://laurencemoroney.com/rock-paper-scissors-dataset"
]
},
"name": "rock_paper_scissors",