DmitryBe

import scala.concurrent.Await
import scala.concurrent.ExecutionContext
import scala.concurrent.Future
import scala.concurrent.blocking
import scala.concurrent.duration.Deadline
import scala.concurrent.duration.Duration
import scala.concurrent.duration.DurationInt
import scala.concurrent.duration.DurationLong
import scala.concurrent.future
import scala.concurrent.promise

DmitryBe

This will only work with ranges with a step (by) of 1 so far, but is a simple solution to splitting a range into a given number of sub-ranges:

  def splitRange(r: Range, chunks: Int): Seq[Range] = {
    if (r.step != 1) 
      throw new IllegalArgumentException("Range must have step size equal to 1")
      
    val nchunks = scala.math.max(chunks, 1)
    val chunkSize = scala.math.max(r.length / nchunks, 1)
    val starts = r.by(chunkSize).take(nchunks)
    val ends = starts.map(_ - 1).drop(1) :+ r.end

DmitryBe

import scala.collection.AbstractIterator

class CustomIterator[T,U](it: Iterator[T])(f: (T => U)) extends AbstractIterator[U] {
  override def hasNext: Boolean = it.hasNext
  override def next(): U = f(it.next)
}

private def resourceAsStream(resource: String): Iterator[(String, String)] = {
  val stream : java.io.InputStream = getClass.getResourceAsStream(resource)
  val it = scala.io.Source.fromInputStream( stream ).getLines

DmitryBe

"""
A bare bones examples of optimizing a black-box function (f) using
Natural Evolution Strategies (NES), where the parameter distribution is a 
gaussian of fixed standard deviation.
"""

import numpy as np
np.random.seed(0)

# the function we want to optimize

DmitryBe

"\e[A": history-search-backward
"\e[B": history-search-forward
set show-all-if-ambiguous on
set completion-ignore-case on
set bell-style none

DmitryBe

Setting up a GPU Enabled Kubernetes for Deep Learning

Kubernetes spread like a wildfire in 2017, No kidding! Here are some numbers from Scott's post:

“For companies with more than 5000 employees, Kubernetes is used by 48% and the primary orchestration tool for 33%.”

“79% of the sample chose Docker as their primary container technology.”

Riding the wave of Kubernetes, 2017 was a particular fun year for Infrastructure/DevOps folks. Finally we have some cool tools to play with after years of darkness. We started thinking what we could do with such paradigm shift. We tried to optimize the developer velocity with Jenkins and Helm Chart and many other more to come :D

One thing I hold dear in my heart is democratizing Kubernetes for Data team. It's a well known fact that today's Data teams have to muster an array of bleeding edge technologies in or

DmitryBe

---
- name: Create Instance in AWS
  hosts: localhost
  connection: local
  gather_facts: false

  vars:
    aws_access_key: "xxxxxx"
    aws_secret_key: "xxxxxx"
    security_token: "xxxxxx"