Skip to content

Instantly share code, notes, and snippets.

View leriomaggio's full-sized avatar
🧙

Valerio Maggio leriomaggio

🧙
View GitHub Profile
@leriomaggio
leriomaggio / solved-flower-hands-on-tutorial-pytorch.ipynb
Created June 19, 2023 04:08
SOLVED:Flower Hands-on Tutorial-PyTorch.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@leriomaggio
leriomaggio / lazy_filter.md
Last active December 12, 2022 11:06
Functional Python Lazy Filtering pipeline

Case Study: Textual Data

Now it is time to crack on with our first case study 🙌.

Assume we have some data in textual format that we want to use as input to some ML model. To make it even more fun, instead of working with standard textual data, let's imagine we have some source code listings we want to process.

Our examplar data will be two functions, written in our most favourite programming language$^5$: init_random_weights_tensor, and init_random_weights_array:

[5]: Examples adapted from Learning PyTorch with Examples tutorial.

@leriomaggio
leriomaggio / multiprocessing_seed_numpy.py
Last active April 11, 2021 11:34
Python and Random State Sharing with NumPy and PyTorch, depending on different multiprocessing start method (i.e. fork vs spawn)
import numpy as np
import multiprocessing as mp
from argparse import ArgumentParser
def get_current_seed(wid):
s = np.random.get_state()[1][0]
print(f"Pool Worker {wid}: Random State {s}")
@leriomaggio
leriomaggio / pytrousse_protocol_data_pipeline_with_validation.py
Last active October 23, 2020 10:47
PyTrousse Protocol-based Data Pipeline with Simple Validation Algorithm
from typing import Tuple, List, Optional, Union, NewType
from typing import Protocol, runtime_checkable
from abc import ABC, abstractmethod
import pandas as pd
try:
import torch as t
from torch import Tensor as Tensor
except ImportError: # fallback to Numpy, if torch not available - this is just as a POP!
import numpy as t
from numpy import ndarray as Tensor
@leriomaggio
leriomaggio / borda.py
Last active December 16, 2022 16:42
Numpy based implementation of the Borda Selection Algorithm for Rankings
"""Python/Numpy implementation of the Borda Count"""
import numpy as np
from nptyping import NDArray
from typing import Tuple, Union, List, Any
RankingsList = Union[NDArray[(Any, Any), np.int], List[List[int]]]
Ranking = NDArray[(Any,), np.int]
Counts = NDArray[(Any,), np.int]
AveragePositions = NDArray[(Any,), np.float]
import umap
import numpy as np
import scipy as sp
print('umap: ', umap.__version__)
print('Numpy: ', np.__version__)
print('Scipy: ', sp.__version__)
# umap: 0.4.0
# Numpy: 1.17.5
@leriomaggio
leriomaggio / kale-euroscipy.md
Last active September 4, 2019 13:53
Kubeflow Kale: from Jupyter Notebook to Complex Pipelines

Kubeflow Kale: from Jupyter Notebook to Complex Pipelines

Abstract

In this talk I will present a new solution to automatically scale Jupyter notebooks to complex and reproducibility pipelines based on Kubernetes and KubeFlow.

Description

Nowadays, most of the High Performance Computing (HPC) tasks are carried out in the Cloud, and this is as much

@leriomaggio
leriomaggio / pyconx_conference_talks_ranking.md
Last active February 10, 2019 05:37
PyConX Conference Talks Ranking
import pandas as pd
import numpy as np
from IPython.display import HTML 
@leriomaggio
leriomaggio / MSCX-Tutorial.md
Last active August 31, 2018 22:39
NetworkX Tutorial @ MSCX - Setup Instructions

Network Analysis Tutorial using Python & networkx

MSCX Ph.D. Summer School, Salina (ME) - Italy

Author: Valerio Maggio

PostDoc Data Scientist @ FBK/MPBA

import pandas as pd
import numpy as np