Skip to content

Instantly share code, notes, and snippets.

@zezutom
zezutom / Main.scala
Created May 6, 2017 10:32
Coursera, progfun1: Recursion
package recfun
object Main {
def main(args: Array[String]) {
println("Pascal's Triangle")
for (row <- 0 to 10) {
for (col <- 0 to row)
print(pascal(col, row) + " ")
println()
}
@zezutom
zezutom / Lists.scala
Last active September 17, 2020 21:32
Coursera, progfun1: Solution to the Example assignment
package example
import java.util.NoSuchElementException
object Lists {
/**
* This method computes the sum of all elements in the list xs. There are
* multiple techniques that can be used for implementing this method, and
@zezutom
zezutom / TextAnalyser.scala
Last active December 20, 2015 20:57
A list of commonly used words as a broadcast variable
class TextAnalyser(val sc: SparkContext, ...) {
...
// Instance variable
val _commonWords = sc.broadcast(TextAnalyser.loadCommonWords())
...
// In a worker thread
def analyse(rdd: RDD[String]): TextStats = {
// To prevent the whole class to be 'sucked' into serialization
val commonWords = _commonWords
...
@zezutom
zezutom / TextAnalyser.scala
Created December 20, 2015 20:39
Accumulators as counters
class TextAnalyser(val sc: SparkContext, ...) {
...
// Instance variables
val _totalChars = sc.accumulator(0, "Total Characters")
val _totalWords = sc.accumulator(0, "Total Words")
...
// In a worker thread
def analyse(rdd: RDD[String]): TextStats = {
...
// This limits serialization to the reference variables only
@zezutom
zezutom / WordCount.scala
Created December 13, 2015 21:52
Spark's RDD transformations to arrive at a word count solution
rdd
.flatMap(_.split("\\s")) // Split on any white character
.map(_.replaceAll(
"[,.!?:;]", "") // Remove punctuation and transfer to lowercase
.trim
.toLowerCase)
.filter(!_.isEmpty) // Filter out any non-words
.map(word => (word, 1)) // Finally, count words
.reduceByKey(_ + _)
.sortByKey() // and sort the word counts in a lexical order
@zezutom
zezutom / sbt-eclipse.sh
Created November 3, 2015 22:46
Bash script generating an SBT project ready to be imported into Eclipse
#!/bin/bash
cwd="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
project_name=${1:-sbt_eclipse_project}
project_version=${2:-1.0}
scala_version=${3:-2.11.7}
@zezutom
zezutom / build.gradle
Created October 10, 2015 19:14
An example of how to use the Cobertura plugin with Gradle
plugins {
id 'net.saliman.cobertura' version '2.2.7'
}
configurations.all {
resolutionStrategy {
// Cobertura includes an ASM version that can't handle Java 8, ASM 5.0.3 handles Java8
// Please note that doesn't resolve all of the issues, such as the lack of support for Lambda expressions
force 'org.ow2.asm:asm:5.0.3'
forcedModules = [ 'org.ow2.asm:asm:5.0.3' ]
@zezutom
zezutom / build.gradle
Created October 10, 2015 18:52
An example of how to use the JaCoCo plugin
apply plugin: 'java'
apply plugin: 'jacoco'
repositories {
jcenter()
}
dependencies {
compile 'org.slf4j:slf4j-api:1.7.12'
testCompile 'junit:junit:4.12'