Skip to content

Instantly share code, notes, and snippets.

View bmc's full-sized avatar

Brian Clapper bmc

View GitHub Profile
@bmc
bmc / printSchemaAsCode.scala
Last active October 2, 2018 12:18
Patch a printSchemaAsCode() method into Spark DataFrame, in Python and Scala
import org.apache.spark.sql.types._
import org.apache.spark.sql._
object Implicits {
def schemaAsCode(schema: StructType, indentation: Int = 2): String = {
def prettyStruct(st: StructType, indentationLevel: Int): String = {
val indentSpaces = " " * (indentationLevel * indentation)
val prefix = s"${indentSpaces}StructType(List(\n"
val fieldIndentSpaces = " " * ((indentationLevel + 1) * indentation)
val fieldStrings: Seq[String] = for (field <- st.fields) yield {
@bmc
bmc / ContextualThing.scala
Last active March 27, 2017 00:40
Attempt to embed any type in a Contextual interpolator
package org.clapper
import scala.language.implicitConversions
import contextual._
object ThingTest1 {
case class Thing(s: String)
@bmc
bmc / FS2ReadInputStream.scala
Last active July 28, 2019 11:04
Read a CSV-like, pipe-delimited (jar) resource into Person records, via FS2
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import java.util.Date
import java.text.SimpleDateFormat
import fs2._
// id and gender should really be more strongly-typed, but
// this is just a demo...
case class Person(id: Int,
firstName: String,
@bmc
bmc / StaticFile.scala
Last active September 3, 2021 20:08
I needed code to serve static files from an Akka HTTP server. I wanted to use fs2 to read the static file. Michael Pilquist recommended using streamz to convert from an fs2 Task to an Akka Source. This is what I came up with. (It does actually work.)
object StaticFile {
// Various necessary imports. Notes:
//
// 1. fs2 is necessary. See https://github.com/functional-streams-for-scala/fs2
// 2. streamz is necessary. See https://github.com/krasserm/streamz
// 3. Apache Tika is used to infer MIME types from file names, because it's more reliable and
// fully-featured than using java.nio.file.Files.probeContentType().
//
// If using SBT, you'll want these library dependencies and resolvers:
@bmc
bmc / BrainDeadHTTP.scala
Last active April 13, 2016 21:21
Brain Dead HTTP server in Scala
package grizzled.testutil
import java.io.{PrintWriter, OutputStreamWriter}
import java.net.InetAddress
import java.text.SimpleDateFormat
import java.util.Date
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.io.Source
@bmc
bmc / promo.md
Created January 13, 2016 21:21
Draft promotional language for Typelevel page

Plan to hang around after the Typelevel Summit, because this year, Northeast Scala Symposium 2016 will be held at the same location on March 4th and 5th. RSVPs open January 16th. Join in the fun!

Keybase proof

I hereby claim:

  • I am bmc on github.
  • I am bmc (https://keybase.io/bmc) on keybase.
  • I have a public key whose fingerprint is B57B 6B4B 451A F540 4D21 BEA4 1588 40B4 A8A7 FD98

To claim this, I am signing this object:

@bmc
bmc / build-extract.sbt
Created April 28, 2014 21:58
Getting 2.10-style reflection to work in 2.11
libraryDependencies <<= (scalaVersion, libraryDependencies) { (sv, deps) =>
// If we're compiling on 2.11, we need to haul the reflection library
// in separately, since it's no longer bundled in 2.11.
if (sv.startsWith("2.11.")) {
deps :+ "org.scala-lang" % "scala-reflect" % "2.11.0"
}
else {
deps
}
@bmc
bmc / LocalHelpers.scala
Created May 3, 2013 15:30
Version of Play's @select helper that handles 'data args. Assumes the existence of a LocalHelpers module that contains a deCamelCase function (shown below)
package views
object LocalHelpers {
/** Default target for offsite anchors.
*/
val OffSiteAnchorTarget = "link"
/** Convert a symbol to a string, for use in HTML.
*/
@bmc
bmc / async.scala
Created April 6, 2013 16:09
Example comparison of Future functions and async library
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent._
import scalaz._
import Scalaz._
import scala.async.Async.{async, await}
import lib.ScalazUtil
// Stubs, to get this to compile. Fill in with real stuff later.
case class User(username: String, password: String)
case class GitHubData(user: User)