Skip to content

Instantly share code, notes, and snippets.

@rberenguel
Last active March 8, 2018 14:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rberenguel/6741f49d4f26e9281bce2d150ba5dfb6 to your computer and use it in GitHub Desktop.
Save rberenguel/6741f49d4f26e9281bce2d150ba5dfb6 to your computer and use it in GitHub Desktop.
Converts a DataFrame schema output (StructType) into something you can paste as a schema, as valid Scala code. Useful when working with Apache Zeppelin or any other REPL.
import org.apache.spark.sql.types._ // You'll need this to evaluate its output
object StructFmt {
def asScala(field: StructField): String = field.dataType match {
case struct: StructType => s"""StructField("${field.name}",""" + asScala(struct) + s", ${field.nullable})"
case _ => s"""StructField("${field.name}", ${field.dataType}, ${field.nullable})"""
}
def asScala(struct: StructType): String = "StructType(Seq(" + (for(field <- struct) yield asScala(field)).mkString(",") + "))"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment