Skip to content

Instantly share code, notes, and snippets.

Avatar
🎯
Focusing

Aditya Soni AdityaSoni19031997

🎯
Focusing
View GitHub Profile
@AdityaSoni19031997
AdityaSoni19031997 / redis.py
Created Apr 18, 2021 — forked from hanneshapke/redis.py
Load word vectors into a redis db
View redis.py
import bz2
import pickle
from django.conf import settings
from djang_redis import get_redis_connection
from tqdm import tqdm
from .constants import GOOGLE_WORD2VEC_MODEL_NAME
@AdityaSoni19031997
AdityaSoni19031997 / file_streaming_pytorch.py
Created Apr 21, 2020
In this gist i have tried to explain a very smart way of loading datasets by streaming them from bytes into PyTorch; It can be achieved in multiple ways, but here my focus was confined to David's idea of streaming records from a bytes file;
View file_streaming_pytorch.py
import torch
import io
import pandas as pd
import gc
import numpy as np
import transformers
'''
Original Code Author [@dlibenzi](https://github.com/dlibenzi)
View itera_dataloader_example.py
import random
from itertools import chain, cycle, islice
import torch.utils.data as data
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import time
import torch
import numpy as np
View colab_setup
# Installs PyTorch, PyTorch/XLA, and Torchvision
# Copy this cell into your own notebooks to use PyTorch on Cloud TPUs
# Warning: this may take a couple minutes to run
import collections
from datetime import datetime, timedelta
import os
import requests
import threading
View av_sig.md

1.251.423.0 1.279.441.0 1.257.387.0 1.275.1078.0 1.249.88.0 1.275.1080.0 1.257.401.0 1.275.1744.0 1.275.1346.0 1.249.715.0

View analyze_stack_python.py
#!/usr/bin/env python
# encoding: utf-8
import sys
def trace_calls_and_returns(frame, event, arg):
co = frame.f_code
func_name = co.co_name
if func_name == 'write':
# Ignore write() calls from print statements
View kaggle_av_nlp_preprocess.py
#from my repo
#https://github.com/AdityaSoni19031997/Machine-Learning/blob/master/AV/AV_Enigma_NLP_functional_api.ipynb
def preprocess_word(word):
# Remove punctuation
word = word.strip('\'"?!,.():;')
# Convert more than 2 letter repetitions to 2 letter
# funnnnny --> funny
word = re.sub(r'(.)\1+', r'\1\1', word)
# Remove - & '
View VisualRandomForests.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.