rmporsch

def batching(iterable: Sized, n: int = 1) -> Iterator:
    """
    Batch a list or iterable
    :param iterable:
    :param n: items per batch
    :return: iterator
    """
    length = len(iterable)
    for ndx in range(0, length, n):
        yield iterable[ndx:min(ndx + n, length)]

rmporsch

# sets to old installer; usefull when encountering issues in which poetry cannot find the relevant wheel (numpy)
poetry config experimental.new-installer false

rmporsch

import timeit
from functools import partial

def some_func(x):
	pass

data = ["some_data"]

REPEATS = 3
NUMBER = 1000

rmporsch

import asyncio


async def queue(task_name):
    i = 0
    while True:
        if i >= 5:
            return True
        print(f"checking queue for task {task_name}")
        i += 1

rmporsch

# raw python
def batch(iterable, n=1):
    l = len(iterable)
    for ndx in range(0, l, n):
        yield iterable[ndx:min(ndx + n, l)]

        
def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in xrange(0, len(lst), n):

rmporsch

# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install numpy

rmporsch

Model explorations and hyperparameter search with W&B and Kubernetes

In every machine learning project we have to continuously tweek and experiment with our models. This is necessary, not only to further improve performance, but also to explore underlying model characteristics. These constant experiments require rigorous logging and performance tracking. Hence, various different provider have come up with solutions to facilitate this tracking such as Tensorboard, Comet, W&B, as well as others. Here at Apoidea we make use of W&B.

Within this blog post we would like to give a more practical overview of how we run machine learning experiments and track their performance. Specifically, how we quickly set up clusters in the cloud and train our models. We hope this might help others, as well as improve our current practices by enganging in a discussion with the wider machine learning community.

Within this post we will out

rmporsch

make.lists <- function (mat) {
  traits <- mat[,1]
  prs <- mat[,2]
  d <- t(combn(c(traits, prs), 2))
  d <- rbind(d, d[, c(2,1)])

  out <- vector()
  i <- 1
  for (i in 1:nrow(d)) {
    temp <- d[i,] 

rmporsch

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import re
newEntry = False
words = []
defs = []
definition = []
with open("dict.email", 'r') as f:
    for line in f:

rmporsch

" Ignore some folders and files for CtrlP indexing
let g:ctrlp_custom_ignore = {
  \ 'dir':  '\.git$\|\.yardoc\|public$|log\|tmp$',
  \ 'file': '\.so$\|\.dat$|\.DS_Store$'
  \ }
  
  " Use The Silver Searcher https://github.com/ggreer/the_silver_searcher
if executable('ag')
  " Use Ag over Grep
  set grepprg=ag\ --nogroup\ --nocolor