Skip to content

Instantly share code, notes, and snippets.

View AdityaSoni19031997's full-sized avatar
🎯
Focusing

Aditya Soni AdityaSoni19031997

🎯
Focusing
117 followers · 122 following
View GitHub Profile
@AdityaSoni19031997
AdityaSoni19031997 / redis.py
Created April 18, 2021 09:39 — forked from hanneshapke/redis.py
Load word vectors into a redis db
import bz2
import pickle
from django.conf import settings
from djang_redis import get_redis_connection
from tqdm import tqdm
from .constants import GOOGLE_WORD2VEC_MODEL_NAME
@AdityaSoni19031997
AdityaSoni19031997 / file_streaming_pytorch.py
Created April 21, 2020 02:19
In this gist i have tried to explain a very smart way of loading datasets by streaming them from bytes into PyTorch; It can be achieved in multiple ways, but here my focus was confined to David's idea of streaming records from a bytes file;
import torch
import io
import pandas as pd
import gc
import numpy as np
import transformers
'''
Original Code Author [@dlibenzi](https://github.com/dlibenzi)
@AdityaSoni19031997
AdityaSoni19031997 / itera_dataloader_example.py
Created April 14, 2020 18:47 — forked from ccj5351/itera_dataloader_example.py
Modified code with the article "How to Build a Streaming DataLoader with PyTorch" at https://medium.com/speechmatics/how-to-build-a-streaming-dataloader-with-pytorch-a66dd891d9dd.
import random
from itertools import chain, cycle, islice
import torch.utils.data as data
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import time
import torch
import numpy as np
@AdityaSoni19031997
AdityaSoni19031997 / colab_setup
Created February 19, 2020 00:49
# Installs PyTorch, PyTorch/XLA, and Torchvision
# Copy this cell into your own notebooks to use PyTorch on Cloud TPUs
# Warning: this may take a couple minutes to run
import collections
from datetime import datetime, timedelta
import os
import requests
import threading
@AdityaSoni19031997
AdityaSoni19031997 / av_sig.md
Created January 9, 2019 04:07

1.251.423.0 1.279.441.0 1.257.387.0 1.275.1078.0 1.249.88.0 1.275.1080.0 1.257.401.0 1.275.1744.0 1.275.1346.0 1.249.715.0

@AdityaSoni19031997
AdityaSoni19031997 / bounding_box.md
Created December 22, 2018 18:03
bbox Generation
@AdityaSoni19031997
AdityaSoni19031997 / current_stats_1-10.md
Last active December 15, 2018 07:43
@AdityaSoni19031997
AdityaSoni19031997 / analyze_stack_python.py
Created October 27, 2018 08:03
#!/usr/bin/env python
# encoding: utf-8
import sys
def trace_calls_and_returns(frame, event, arg):
co = frame.f_code
func_name = co.co_name
if func_name == 'write':
# Ignore write() calls from print statements
@AdityaSoni19031997
AdityaSoni19031997 / kaggle_av_nlp_preprocess.py
Created October 23, 2018 15:41
#from my repo
#https://github.com/AdityaSoni19031997/Machine-Learning/blob/master/AV/AV_Enigma_NLP_functional_api.ipynb
def preprocess_word(word):
# Remove punctuation
word = word.strip('\'"?!,.():;')
# Convert more than 2 letter repetitions to 2 letter
# funnnnny --> funny
word = re.sub(r'(.)\1+', r'\1\1', word)
# Remove - & '
@AdityaSoni19031997
AdityaSoni19031997 / VisualRandomForests.ipynb
Created October 21, 2018 10:50 — forked from tylerwx51/VisualRandomForests.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.