Skip to content

Instantly share code, notes, and snippets.

View snakers4's full-sized avatar
🚀
It is by will alone I set my mind in motion.

Alexander Veysov snakers4

🚀
It is by will alone I set my mind in motion.
View GitHub Profile
<p></p><p><img src="https://pics.spark-in.me/upload/7b09411ab7b91bc8da7e79c763311795.jpg" style="width: 700px;"></p><p><span style="font-style: italic;">Как всегда надо начать с заманчивой картинки, но не пояснять ее смысл!</span></p><hr><p></p><p style="text-align: justify; ">Пару дней назад ко мне обратился далекий знакомый, мол у нас политесы в компании и мы не можем никак выбрать направление развития. Ситуация типичная - <span style="font-weight: bold;">лебедь, рак и щука</span> и надо бы очень быстро <span style="font-weight: bold;">проанализировать сайты его конкурентов</span>, вот держи картинку со списком самых крупных компаний в этой сфере. Оказалось, что конкуренты из сферы форекса. Это меня смутило, но не остановило. Я посмотрел на их сайты (а, забегая вперед, некоторые из них просто огромны - миллионы страниц) и тут у меня родилась гениальная идея.</p><p style="text-align: justify; ">Я занимался ей пару суток почти без остановки и накопал относительно интересные результаты, которыми и хочу поделит
from keras.models import Model
from keras.layers import Input, concatenate, Conv2D, MaxPooling2D, Activation, UpSampling2D, BatchNormalization,multiply
from keras.optimizers import RMSprop
from model.losses import bce_dice_loss, dice_loss, weighted_bce_dice_loss, weighted_dice_loss, dice_coeff
import params
orig_width = 1918
orig_height = 1280
FROM nvidia/cuda:8.0-cudnn6-devel
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:YOUR_PASSWORD' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# SSH login fix. Otherwise user is kicked off after login
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
@snakers4
snakers4 / DS NN guidelines
Created October 19, 2017 07:39
Complete system installation for Data Science / Neural Networks from scratch / bare metal (assuming you have assembled the PC)
# BASIC SYSTEM SETUP
# First donwload Ubuntu iso file from https://www.ubuntu.com/download/desktop
# Use 16.04 LTS (17 is also ok, but it's better to use LTS versions, also 18 will be very mature in terms of systemd)
# Dowload Linux live USB creator and install the iso to your USB stick https://www.linuxliveusb.com
# Boot your system, go to BIOS on boot (usually Del) or boot menu (usually F12) and choose your USB stick as boot medium
# Install Linux (these steps can be omitted if clean Ubuntu installation is provided as service by admins / cloud provider / etc)
# Minor trick unplug ALL of your hard disks (unless you are an avanced user) except for the disk for your system
import torchvision.models as models
import torch
import torch.nn as nn
class FineTuneModel(nn.Module):
def __init__(self,
original_model,
arch,
num_classes,
freeze
@snakers4
snakers4 / Dockerfile
Last active February 22, 2018 09:30
Dockerfile update
# add 7z tar and zip archivers
FROM nvidia/cuda:9.0-cudnn7-devel
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:Ubuntu@41' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# SSH login fix. Otherwise user is kicked off after login
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
@snakers4
snakers4 / abstractions.py
Last active May 27, 2022 07:12
My XGB boilerplate
# https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/
#Import libraries:
import pandas as pd
import numpy as np
import xgboost as xgb
from xgboost.sklearn import XGBClassifier
from sklearn import cross_validation, metrics #Additional scklearn functions
from sklearn.grid_search import GridSearchCV #Perforing grid search
import matplotlib.pylab as plt
@snakers4
snakers4 / multiprocessing_tqdm.py
Last active May 25, 2018 10:13
Using tqdm with multiprocessing
import tqdm
import pandas as pd
import numpy as np
from multiprocessing import Pool
import os
# drop all the unknown points and all closed points
# for each SK_ID_CURR calculate the counts of time in each status
# normalize by the max len (we know of) in any of the meaningful statuses
@snakers4
snakers4 / PyTorchMlP.py
Created May 28, 2018 07:48
Playing with MLP + embeddings in PyTorch
import torch
import torch.nn as nn
from torch.autograd import Variable
class NaiveClassifier(nn.Module):
def __init__(self,
cat_sizes=None,
numerical_features=117,
mlp_sizes=[1024,2048,1024,512,256,128,2],
embedding_factor=3,
@snakers4
snakers4 / Dockerfile
Created June 4, 2018 10:25
Atmyra Dockerfile
# add 7z tar and zip archivers
FROM nvidia/cuda:9.0-cudnn7-devel
# https://docs.docker.com/engine/examples/running_ssh_service/
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:Ubuntu@41' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
RUN mkdir ~/.ssh/