Skip to content

Instantly share code, notes, and snippets.

View lopuhin's full-sized avatar

Konstantin Lopuhin lopuhin

454 followers · 0 following
View GitHub Profile
@lopuhin
lopuhin / pytorch_high_mem.py
Created October 15, 2019 10:06
pytorch high memory usage on CPU inference with variable input shapes
import os
os.environ['OMP_NUM_THREADS'] = '1'
import argparse
import resource
import sys
import time
import numpy as np
import torch
from torchvision.models import resnet34
@lopuhin
lopuhin / mkldnn_489.py
Created June 12, 2019 18:35
Repro script for https://github.com/intel/mkl-dnn/issues/489
import os
os.environ['OMP_NUM_THREADS'] = '1'
import argparse
import resource
import sys
import time
import numpy as np
import torch
from torch import nn
@lopuhin
lopuhin / resnet_mkldnn.py
Created June 6, 2019 18:27
Faster ResNet CPU inference with MKLDNN + PyTorch 1.1
$ pip install torch==1.1.0 torchvision==0.3.0
$ OMP_NUM_THREADS=1 ipython
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
In [1]: import torch
...: from torchvision.models import resnet50
In [2]: def forward(m, x):
...: """ resnet without average pooling """
...: x = m.conv1(x)
@lopuhin
lopuhin / RPython traceback and lldb backtrace.txt
Last active March 3, 2019 15:04
cpyext_tp_new_tuple error: pytorch + pypy3.6
$ python
Python 3.6.1 (dab365a46514, Feb 05 2019, 16:36:40)
[PyPy 7.0.0-alpha0 with GCC 6.2.0 20160901] on linux
$ lldb python -- -c 'import torch'
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) settings set -- target.run-args "-c" "import torch"
(lldb) run
Process 9293 launched: '/home/kostia/ml/pytorch/pypy3.6-v7.0.0/bin/python' (x86_64)
@lopuhin
lopuhin / kaggle-trackml-2018-viz3d.py
Last active March 3, 2020 09:55
Simple OpenGL visualization starter for Kaggle TrackML 2018
#!/usr/bin/env python3
"""
Install glumpy dependencies::
pip install Cython pyopengl
brew install glfw freetype # on OS X
Install glumpy::
pip install glumpy
@lopuhin
lopuhin / groups.csv
Created September 12, 2017 10:43
folds for drivendata fish identification
video_id group
00WK7DR6FyPZ5u3A 0
01wO3HNwawJYADQw 1
02p3Yn87z0b5grhL 0
05RagNzyjVdXeUFR 0
06aFbpjpYHVBLJZa 2
09GA0YdSz0NvJUJx 3
0D5zA3WRD5pXjr8j 4
0EmM5wsVVNqaKNaM 5
0L5wBDT4CEazgAw6 6
@lopuhin
lopuhin / pympler_heap.py
Created January 31, 2017 13:10
Show python heap with pympler
from pympler.summary import summarize, print_
from pympler.muppy import get_object
all_objects = get_objects()
summary = summarize(all_objects)
print_(summary)
@lopuhin
lopuhin / sklearn_model_serialization.py
Last active November 25, 2016 13:24
We want to serialize a model that might have complex preprocessing, lambdas, etc., so it is not directly pickleable. Here we add custom pickle support and serialize only model parameters.
from typing import Dict, Any
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegressionCV
from sklearn.pipeline import make_pipeline, FeatureUnion
class BaseModel:
def __init__(self, **kwargs):
self._kwargs = kwargs
@lopuhin
lopuhin / django downloads 2 vs 3.sql
Created May 23, 2016 21:21
SELECT
REGEXP_EXTRACT(details.python, r"^([^\.]+)") as python_version,
COUNT(*) as total_downloads,
FROM
TABLE_DATE_RANGE(
[the-psf:pypi.downloads],
TIMESTAMP("20160521"),
TIMESTAMP("20160524")
)
WHERE
@lopuhin
lopuhin / vmprof data
Created October 23, 2015 14:14
!! ['Process:', 'pypy', '[30403]']
!! ['Path:', '/Users/kostia/apps/netdb_demo_2/pypy/bin/pypy']
!! ['Load', 'Address:', '0x100000000']
!! ['Identifier:', 'pypy']
!! ['Version:', '???']
!! ['Code', 'Type:', 'X86-64']
!! ['Parent', 'Process:', 'bash', '[26151]']
!! ['']
!! ['Date/Time:', '2015-10-23', '17:05:13.105', '+0300']
!! ['Launch', 'Time:', '2015-10-23', '17:05:12.611', '+0300']