leriomaggio

leriomaggio

"""Python/Numpy implementation of the Borda Count"""

import numpy as np
from nptyping import NDArray
from typing import Tuple, Union, List, Any

RankingsList = Union[NDArray[(Any, Any), np.int], List[List[int]]]
Ranking = NDArray[(Any,), np.int]
Counts = NDArray[(Any,), np.int]
AveragePositions = NDArray[(Any,), np.float]

leriomaggio

Case Study: Textual Data

Now it is time to crack on with our first case study 🙌.

Assume we have some data in textual format that we want to use as input to some ML model. To make it even more fun, instead of working with standard textual data, let's imagine we have some source code listings we want to process.

Our examplar data will be two functions, written in our most favourite programming language$^5$: init_random_weights_tensor, and init_random_weights_array:

[5]: Examples adapted from Learning PyTorch with Examples tutorial.

leriomaggio

import numpy as np
import multiprocessing as mp
from argparse import ArgumentParser


def get_current_seed(wid):
    s = np.random.get_state()[1][0]
    print(f"Pool Worker {wid}: Random State {s}")

leriomaggio

from typing import Tuple, List, Optional, Union, NewType
from typing import Protocol, runtime_checkable
from abc import ABC, abstractmethod
import pandas as pd
try:
    import torch as t
    from torch import Tensor as Tensor
except ImportError: # fallback to Numpy, if torch not available - this is just as a POP!
    import numpy as t
    from numpy import ndarray as Tensor

leriomaggio

#----------------------------------------------------------------------------------------
# model.py
#----------------------------------------------------------------------------------------
from django.db import models
from numpy.random import random_sample

class CustomModel(models.Model):
    # Put your fields here

    def get_data(self):

leriomaggio

import umap
import numpy as np
import scipy as sp

print('umap: ', umap.__version__)
print('Numpy: ', np.__version__)
print('Scipy: ', sp.__version__)

# umap:  0.4.0
# Numpy:  1.17.5

leriomaggio

import pandas as pd
import numpy as np

leriomaggio

Kubeflow Kale: from Jupyter Notebook to Complex Pipelines

Abstract

In this talk I will present a new solution to automatically scale Jupyter notebooks to complex and reproducibility pipelines based on Kubernetes and KubeFlow.

Description

Nowadays, most of the High Performance Computing (HPC) tasks are carried out in the Cloud, and this is as much

leriomaggio

import pandas as pd
import numpy as np
from IPython.display import HTML