Skip to content

Instantly share code, notes, and snippets.

Avatar
🎯
Focusing

Aditya Soni AdityaSoni19031997

🎯
Focusing
100 followers · 102 following · 640
View GitHub Profile
@AdityaSoni19031997
AdityaSoni19031997 / redis.py
Created Apr 18, 2021 — forked from hanneshapke/redis.py
Load word vectors into a redis db
View redis.py
import bz2
import pickle
from django.conf import settings
from djang_redis import get_redis_connection
from tqdm import tqdm
from .constants import GOOGLE_WORD2VEC_MODEL_NAME
@AdityaSoni19031997
AdityaSoni19031997 / file_streaming_pytorch.py
Created Apr 21, 2020
In this gist i have tried to explain a very smart way of loading datasets by streaming them from bytes into PyTorch; It can be achieved in multiple ways, but here my focus was confined to David's idea of streaming records from a bytes file;
View file_streaming_pytorch.py
import torch
import io
import pandas as pd
import gc
import numpy as np
import transformers
'''
Original Code Author [@dlibenzi](https://github.com/dlibenzi)
@AdityaSoni19031997
AdityaSoni19031997 / itera_dataloader_example.py
Created Apr 14, 2020 — forked from ccj5351/itera_dataloader_example.py
Modified code with the article "How to Build a Streaming DataLoader with PyTorch" at https://medium.com/speechmatics/how-to-build-a-streaming-dataloader-with-pytorch-a66dd891d9dd.
View itera_dataloader_example.py
import random
from itertools import chain, cycle, islice
import torch.utils.data as data
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import time
import torch
import numpy as np
@AdityaSoni19031997
AdityaSoni19031997 / colab_setup
Created Feb 19, 2020
View colab_setup
# Installs PyTorch, PyTorch/XLA, and Torchvision
# Copy this cell into your own notebooks to use PyTorch on Cloud TPUs
# Warning: this may take a couple minutes to run
import collections
from datetime import datetime, timedelta
import os
import requests
import threading
@AdityaSoni19031997
AdityaSoni19031997 / av_sig.md
Created Jan 9, 2019
View av_sig.md

1.251.423.0 1.279.441.0 1.257.387.0 1.275.1078.0 1.249.88.0 1.275.1080.0 1.257.401.0 1.275.1744.0 1.275.1346.0 1.249.715.0

@AdityaSoni19031997
AdityaSoni19031997 / bounding_box.md
Created Dec 22, 2018
bbox Generation
View bounding_box.md
@AdityaSoni19031997
AdityaSoni19031997 / current_stats_1-10.md
Last active Dec 15, 2018
View current_stats_1-10.md
@AdityaSoni19031997
AdityaSoni19031997 / analyze_stack_python.py
Created Oct 27, 2018
View analyze_stack_python.py
#!/usr/bin/env python
# encoding: utf-8
import sys
def trace_calls_and_returns(frame, event, arg):
co = frame.f_code
func_name = co.co_name
if func_name == 'write':
# Ignore write() calls from print statements
@AdityaSoni19031997
AdityaSoni19031997 / kaggle_av_nlp_preprocess.py
Created Oct 23, 2018
View kaggle_av_nlp_preprocess.py
#from my repo
#https://github.com/AdityaSoni19031997/Machine-Learning/blob/master/AV/AV_Enigma_NLP_functional_api.ipynb
def preprocess_word(word):
# Remove punctuation
word = word.strip('\'"?!,.():;')
# Convert more than 2 letter repetitions to 2 letter
# funnnnny --> funny
word = re.sub(r'(.)\1+', r'\1\1', word)
# Remove - & '
@AdityaSoni19031997
AdityaSoni19031997 / VisualRandomForests.ipynb
Created Oct 21, 2018 — forked from tylerwx51/VisualRandomForests.ipynb
View VisualRandomForests.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.