Skip to content

Instantly share code, notes, and snippets.

@olegchir
Created September 22, 2020 06:53
Show Gist options
  • Save olegchir/37c1d4ab1434ddd1d5542cd96a7de936 to your computer and use it in GitHub Desktop.
Save olegchir/37c1d4ab1434ddd1d5542cd96a7de936 to your computer and use it in GitHub Desktop.
{
"op": "PARAGRAPH_APPEND_OUTPUT",
"data": {
"data": "{\"bank\":{\"type\":\"org.apache.spark.sql.DataFrame\",\"value\":{\"schema()\":{\"type\":\"org.apache.spark.sql.types.StructType\",\"value\":[{\"jvm-type\":\"org.apache.spark.sql.types.StructField\",\"value\":{\"metadata\":{\"jvm-type\":\"org.apache.spark.sql.types.Metadata\",\"type\":\"org.apache.spark.sql.types.Metadata\",\"value\":\"{}\"},\"nullable\":{\"type\":\"Boolean\",\"value\":true},\"dataType\":{\"jvm-type\":\"org.apache.spark.sql.types.IntegerType$\",\"type\":\"org.apache.spark.sql.types.DataType\",\"value\":\"IntegerType\"},\"name\":{\"type\":\"String\",\"value\":\"age\"}}},{\"jvm-type\":\"org.apache.spark.sql.types.StructField\",\"value\":{\"metadata\":{\"ref\":\"bank.schema()[0].metadata\"},\"nullable\":{\"type\":\"Boolean\",\"value\":true},\"dataType\":{\"jvm-type\":\"org.apache.spark.sql.types.StringType$\",\"type\":\"org.apache.spark.sql.types.DataType\",\"value\":\"StringType\"},\"name\":{\"type\":\"String\",\"value\":\"job\"}}},{\"jvm-type\":\"org.apache.spark.sql.types.StructField\",\"value\":{\"metadata\":{\"ref\":\"bank.schema()[0].metadata\"},\"nullable\":{\"type\":\"Boolean\",\"value\":true},\"dataType\":{\"ref\":\"bank.schema()[1].dataType\"},\"name\":{\"type\":\"String\",\"value\":\"marital\"}}},{\"jvm-type\":\"org.apache.spark.sql.types.StructField\",\"value\":{\"metadata\":{\"ref\":\"bank.schema()[0].metadata\"},\"nullable\":{\"type\":\"Boolean\",\"value\":true},\"dataType\":{\"ref\":\"bank.schema()[1].dataType\"},\"name\":{\"type\":\"String\",\"value\":\"education\"}}},{\"jvm-type\":\"org.apache.spark.sql.types.StructField\",\"value\":{\"metadata\":{\"ref\":\"bank.schema()[0].metadata\"},\"nullable\":{\"type\":\"Boolean\",\"value\":true},\"dataType\":{\"ref\":\"bank.schema()[0].dataType\"},\"name\":{\"type\":\"String\",\"value\":\"balance\"}}}]},\"getStorageLevel()\":{\"type\":\"org.apache.spark.storage.StorageLevel\",\"value\":\"StorageLevel(1 replicas)\"}}},\"bankText\":{\"type\":\"org.apache.spark.rdd.RDD[String]\",\"value\":{\"getNumPartitions()\":{\"type\":\"Int\",\"value\":2},\"partitioner\":{\"type\":\"Option[org.apache.spark.Partitioner]\",\"value\":\"None\"},\"name\":{\"type\":\"String\"},\"id\":{\"type\":\"Int\",\"value\":29},\"getStorageLevel()\":{\"type\":\"org.apache.spark.storage.StorageLevel\",\"value\":\"StorageLevel(1 replicas)\"}}}}\n",
"index": 0,
"noteId": "2A94M5J1Z",
"paragraphId": "paragraph_1600755864554_1214181900"
},
"ticket": "anonymous",
"principal": "anonymous",
"roles": ""
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment