Skip to content

Instantly share code, notes, and snippets.

View mGalarnyk's full-sized avatar

Michael Galarnyk mGalarnyk

View GitHub Profile
@stephanie-wang
stephanie-wang / sort.py
Created March 18, 2021 18:19
Distributed sort on Ray
import ray
import numpy as np
@ray.remote
def map(data, npartitions):
outputs = [list() for _ in range(npartitions)]
for row in data:
outputs[int(row * npartitions)].append(row)
return tuple(sorted(output) for output in outputs)
@stephanie-wang
stephanie-wang / dask_on_ray.py
Created February 16, 2021 19:34
Data processing support in Ray
import ray
from ray.util.dask import ray_dask_get
import dask
import dask.dataframe as dd
import pandas as pd
import numpy as np
dask.config.set(scheduler=ray_dask_get) # Sets Ray as the default backend.
import torch
from torch.utils.data import DataLoader
from torchvision.datasets import CIFAR10
import torchvision.transforms as transforms
import ray
from ray.util.sgd.torch import TorchTrainer
from ray.util.sgd.torch import TrainingOperator
# https://github.com/kuangliu/pytorch-cifar/blob/master/models/resnet.py
from ray.util.sgd.torch.resnet import ResNet18
import numpy as np
import torch
import torch.optim as optim
from ray import tune
from ray.tune.examples.mnist_pytorch import get_data_loaders, train, test
import ray
import sys
if len(sys.argv) > 1:
@mGalarnyk
mGalarnyk / generate_loan_table.py
Created December 7, 2018 19:03
Calculate Car loan table. How much of a monthly payment (EMI) go to interest and how much go to principal
P = 34689.96
term = 60
def generate_loan_table(P, term, interest_rate=0.0702):
def calc_emi(P, n, interest_rate):
r = interest_rate / 12
numerator = (r *((1 + r)**(n)) )
denominator = ((1 + r)**(n)) - 1
emi = P * (numerator / denominator)
@mGalarnyk
mGalarnyk / project1.md
Last active April 12, 2023 11:18
The Data Scientist's Toolbox Project John Hopkins Data Science Specialization Coursera for the github repo https://github.com/mGalarnyk/datasciencecoursera

The Data Scientist's Toolbox Project (JHU) Coursera

A image video link walkthrough of the project + answers is available at the end of this quiz.

github repo for rest of specialization: Data Science Coursera

Part 1

The Course Project for this course is pretty straightforward. Now that you’ve learned about the basic tools found in the data scientist’s toolbox, we want to make sure that you’ve assembled those tools and are ready to begin using them. This easy project is your chance to demonstrate that you’ve done the basic software setup (R, RStudio, and Github) that will get you through the rest of the Data Science Specialization.

  1. Install R
@sachaarbonel
sachaarbonel / gist:c0a29118dcaaa7f0665803a72ac74742
Last active February 23, 2017 08:21
Spark Install on Ubuntu
http://www.xavierdupre.fr/app/ensae_teaching_cs/helpsphinx/td_3a_spark.html#installation-de-spark-sous-linux
https://github.com/mGalarnyk/Installations_Mac_Ubuntu_Windows.git