Skip to content

Instantly share code, notes, and snippets.

@harschware
Created May 11, 2018 19:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save harschware/a29dc3224d3dea6bb749790244d39d72 to your computer and use it in GitHub Desktop.
Save harschware/a29dc3224d3dea6bb749790244d39d72 to your computer and use it in GitHub Desktop.
Livy DataTypes are Missing from Schema
{
"code": "import org.apache.spark.sql._\nvar d = sqlContext.sql(\"SELECT tbl10.`venueid`, tbl10.`venuename`, tbl10.`venuecity`, tbl10.`venuestate`, tbl10.`venueseats`, tbl10.`processing_dttm` AS `venues_processing_dttm` FROM `concerts`.`venues` tbl10\")\nval e = d.take(2)\n%json e\n",
"id": 13,
"output": {
"data": {
"application/json": [
{
"schema": [
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venueid",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuename",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuecity",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuestate",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venueseats",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venues_processing_dttm",
"nullable": true
}
],
"values": [
1,
"Toyota Park",
"Bridgeview",
"IL",
0,
"1520786524754"
]
},
{
"schema": [
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venueid",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuename",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuecity",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venuestate",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venueseats",
"nullable": true
},
{
"dataType": {},
"metadata": {
"map": {}
},
"name": "venues_processing_dttm",
"nullable": true
}
],
"values": [
2,
"Columbus Crew Stadium",
"Columbus",
"OH",
0,
"1520786524754"
]
}
]
},
"execution_count": 13,
"status": "ok"
},
"progress": 1.0,
"state": "available"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment