Skip to content

Instantly share code, notes, and snippets.

@echeran
echeran / pom.xml
Last active December 27, 2015 07:42
mvn-lein-test - pom.xml
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.app</groupId>
<artifactId>my-app</artifactId>
<version>1</version>
<repositories>
<repository>
<id>clojars</id>
<name>Clojars</name>
@echeran
echeran / project.clj
Last active December 27, 2015 07:42
mvn-lein-test - project.clj
(defproject mvn-lein-test "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.7.0"]
[medley "0.7.0"]
[leiningen "2.5.3"]]
:main mvnleintest.core
:aot :all
@echeran
echeran / stacktrace.txt
Created December 27, 2015 08:25
stacktrace from running java -jar on Maven-built uberjar
2015-12-27 00:22:56,112 DEBUG com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory - Provided Arguments: {}
Exception in thread "main" java.lang.IllegalArgumentException: interface TestWordCountOptions is not visible from class loader
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:581)
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557)
at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230)
at java.lang.reflect.WeakCache.get(WeakCache.java:127)
at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
at java.lang.reflect.Proxy.getProxyClass(Proxy.java:371)
at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.validateWellFormed(PipelineOptionsFactory.java:587)
at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1242)
/*
Copyright 2018 Google LLC.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
/**
Copyright 2018 Google LLC.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
@echeran
echeran / transpiler-veterans-quesitons.md
Last active April 11, 2020 00:51
Questions for Transpiler Veterans
  1. For your transpiler, how did you decide which source language and target language(s) to focus on?
  2. Were there any existing transpiler solutions that solved part of all of the problem surface you were addressing? If so, which factors led to you choosing to create your own transpiler instead of reusing existing solutions?
  3. Who were your stakeholders when starting the project?
  4. What were obstacles that you faced when starting the project?
  5. What were obstacles that you faced during the implementation of the project?
  6. What factors led to the success of the project?
  7. How much time has been spent on the project since the beginning?
  8. Are there any remaining features that you would still like to fix in the transpiling project?
  9. What are the technical considerations / challenges for anyone considering a transpiling approach? Are there unwritten pearls of wisdom about transpilers / compilers?
  10. What is your opinion on why there are few (no?) “universal” general purpose one-to-all transpilers? Ex:
@echeran
echeran / clojure-multithreaded-spark.clj
Created March 13, 2021 22:12
Mulit-threaded Spark jobs via Clojure with Claypoole (custom pmap) + mapPartitions glue code
(require '[claypoole.core :as tpool])
;;
;; helper fns
;;
(defn your-multi-threaded-spark-fn
"Create a parallelized version of your-fn using a fixed-size threadpool.
This fn is created to fit the signature of .mapPartitions in Spark."
[extra-args-to-set-up-your-fn]
(fn
@echeran
echeran / clojure-serialized-fns-flambo-spark.clj
Last active March 13, 2021 22:17
Reusing your fns as Clojure serialized fns to use in Flambo (Spark wrapper for Clojure)
(require '[clj-time.core :as t])
(require '[flambo.api :as f])
(defn date-filter-fn
"Given a LocalDate, returns a serializable fn that takes a DateTime
object and returns true if the DateTime fields match those of the
provided LocalDate, else returns false."
[local-date]
;; returning a serializable fn to be used with Flambo/Spark's filter operation
(f/fn
@echeran
echeran / async_util.clj
Last active March 1, 2024 23:46
Utility functions for core.async 0.1.346.0-17112a-alpha to create a linear pipeline of go-routines in a Storm-esque way.
(ns your.namespace.async-util
(:require
[clojure.core.async :as a :refer [>! <! >!! <!! go chan buffer
close! thread alts! alts!! timeout]]
[clojure.core.match :as match :refer [match]]))
;;
;; core.async util fns
;;