Skip to content

Instantly share code, notes, and snippets.


Aditya Soni AdityaSoni19031997

View GitHub Profile
AdityaSoni19031997 /
Created Apr 18, 2021 — forked from hanneshapke/
Load word vectors into a redis db
import bz2
import pickle
from django.conf import settings
from djang_redis import get_redis_connection
from tqdm import tqdm
from .constants import GOOGLE_WORD2VEC_MODEL_NAME
AdityaSoni19031997 /
Created Apr 21, 2020
In this gist i have tried to explain a very smart way of loading datasets by streaming them from bytes into PyTorch; It can be achieved in multiple ways, but here my focus was confined to David's idea of streaming records from a bytes file;
import torch
import io
import pandas as pd
import gc
import numpy as np
import transformers
Original Code Author [@dlibenzi](
import random
from itertools import chain, cycle, islice
import as data
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import time
import torch
import numpy as np
View colab_setup
# Installs PyTorch, PyTorch/XLA, and Torchvision
# Copy this cell into your own notebooks to use PyTorch on Cloud TPUs
# Warning: this may take a couple minutes to run
import collections
from datetime import datetime, timedelta
import os
import requests
import threading

1.251.423.0 1.279.441.0 1.257.387.0 1.275.1078.0 1.275.1080.0 1.257.401.0 1.275.1744.0 1.275.1346.0 1.249.715.0

#!/usr/bin/env python
# encoding: utf-8
import sys
def trace_calls_and_returns(frame, event, arg):
co = frame.f_code
func_name = co.co_name
if func_name == 'write':
# Ignore write() calls from print statements
#from my repo
def preprocess_word(word):
# Remove punctuation
word = word.strip('\'"?!,.():;')
# Convert more than 2 letter repetitions to 2 letter
# funnnnny --> funny
word = re.sub(r'(.)\1+', r'\1\1', word)
# Remove - & '
View VisualRandomForests.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.