Skip to content

Instantly share code, notes, and snippets.

View andrewpalumbo's full-sized avatar

Andrew Palumbo andrewpalumbo

View GitHub Profile
@andrewpalumbo
andrewpalumbo / Output of run_conda.sh
Last active February 14, 2019 18:37
conda build of py-arrow with CUDA support
./run_conda.sh: 1: ./run_conda.sh: conda: not found
./run_conda.sh: 2: ./run_conda.sh: conda: not found
./run_conda.sh: 3: ./run_conda.sh: conda: not found
fatal: destination path 'arrow' already exists and is not an empty directory.
./run_conda.sh: 12: ./run_conda.sh: conda: not found
-- Building using CMake version: 3.5.1
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
@andrewpalumbo
andrewpalumbo / Matrices
Last active April 25, 2016 17:57
Problem when empty partitions are encountered.
Matrix A:{
0 => {0:1.0,1:2.0}
1 => {0:3.0,1:4.0}
}
Matrix B:{
0 => {0:3.0,1:4.0}
1 => {0:5.0,1:6.0}
}
@andrewpalumbo
andrewpalumbo / ALS.scala
Last active April 21, 2016 01:02
dals failure in FlinkDistributedTestSuite
def dals[K](
drmA: DrmLike[K],
k: Int = 50,
lambda: Double = 0.0,
maxIterations: Int = 10,
convergenceThreshold: Double = 0.10
): Result[K] = {
assert(convergenceThreshold < 1.0, "convergenceThreshold")
assert(maxIterations >= 1, "maxIterations")
@andrewpalumbo
andrewpalumbo / FlinkOpABt.scala
Created April 10, 2016 16:13
Flink Kryo StackOverflowError
/** Contains DataSet plans for ABt operator */
object FlinkOpABt {
private final implicit val log = getLog(FlinkOpABt.getClass)
/**
* General entry point for AB' operator.
*
* @param operator the AB' operator
@andrewpalumbo
andrewpalumbo / Compiler error
Last active April 8, 2016 19:39
"Object creation impossible" compiler error when trying to Extend an Abstract java class as an anonymous class in scala.
[ERROR] /home/andy/sandbox/mahout/flink/src/main/scala/org/apache/mahout/flinkbindings/blas/FlinkOpABt.scala:137:
error: object creation impossible, since method combine in class RichGroupCombineFunction of type
(x$1: Iterable[(Int, (Array[K], Array[Int], org.apache.mahout.math.Matrix))], x$2:
org.apache.flink.util.Collector[(Array[K], org.apache.mahout.math.Matrix)])Unit is not defined