Skip to content

Instantly share code, notes, and snippets.

Library

You where asked to help with the library system. It's simple SQL db with three tables User, Book and Borrow. Borrow table is journal. For each change in library - borrowing - you will have new row. E.g when you borrow a book, you will have new row in Borrow table with current timestamp. When you will return the book, you will have new row in Borrow table with current timestamp.

Below you can find schema for them:

User

id name
@explicite
explicite / reduceByKey.scala
Created March 22, 2019 12:29
reduce by key on DS in Spark
def reduceBy[A, B](merge: (A, A) => A)(
by: A => B
)(ds: Dataset[A])(implicit session: SparkSession, encoderA: Encoder[A], encoderB: Encoder[B]): Dataset[A] = {
def reducePartition(iter: Iterator[A]): Iterator[A] = {
iter.toList
.groupBy(by)
.mapValues(values => values.reduce(merge))
.values
.toIterator

Keybase proof

I hereby claim:

  • I am explicite on github.
  • I am jpaw (https://keybase.io/jpaw) on keybase.
  • I have a public key whose fingerprint is 2A3B D40D 54BF 22DE BCAC 3CB5 9637 9FB8 B4A8 62D1

To claim this, I am signing this object:

@explicite
explicite / trained_elephant.md
Last active November 6, 2017 14:14
Trained Elephant

Trained elephant

Why?

Because of problems defined in (Hadoop is like elephant crushing) :

I want to define requirements for data lake from perspective of:

  • developer
    • collaboration (scm, review, system modularity)
  • testing
@explicite
explicite / keybase.md
Last active August 9, 2017 08:20
keybase.md

Keybase proof

I hereby claim:

  • I am explicite on github.
  • I am jpaw (https://keybase.io/jpaw) on keybase.
  • I have a public key whose fingerprint is 4E0A 840C 16BB B667 D271 4FD4 5F7C 14C6 89D5 8545

To claim this, I am signing this object:

@explicite
explicite / rpicluster.md
Last active February 28, 2018 11:07
RPI Cluster

Why?

Skupienie sie na pachyderm i pokazanie czy rzeczywiscie może być przyszłością Jakiś przykład w pachyderm ML na klastrze raspberry pi - tutaj musiał bym usiąść po pracy

  • cluster z raspberry pi
  • kubernetis - in progress
  • pachyderm
  • przyklad w pachyderm
  • wnioski

RPI Cluster

@explicite
explicite / datalake.md
Last active September 19, 2017 10:14
Data Lake

Data Lake Trylogy

[1] Diving in the data lake

  • przedstawię czym jest DL
  • pisze czemu wogóle to jest i jakie są korzysci
  • komponenty?
  • data swamp - czyli czego nie powinno sie robić
  • jakieś małe podsumowanie
/*
OOP modeling
*/
//class
function Man(firstName, secondName, age, sex) {
//public
this.firstName = firstName;
this.secondName = secondName;
this.age = age || 0;
GET "LIBHDR"
LET START() BE
$(
WRITES("Hello, world!*N")
$)
@explicite
explicite / ds12b20.go
Last active August 29, 2015 14:27
ds12b20
package main
import (
"flag"
"fmt"
"io/ioutil"
"log"
"strconv"
"time"
)