sparksql-nested-json.md

Test data:

{"maker":"Apple", "products":[{"name": "iPhobe", "price": 100000}, {"name": "iPad", "price": 120000}]}
{"maker":"ASUS", "products":[{"name": "Zenfone", "price": 20000}]}

DataFrame:

scala> val df = ds.select(ds("maker"),explode(ds("products")).as("p"))
scala> df.select("maker", "p.name", "p.price").show
+-----+-------+------+
|maker|   name| price|
+-----+-------+------+
|Apple| iPhobe|100000|
|Apple|   iPad|120000|
| ASUS|Zenfone| 20000|
+-----+-------+------+

SQL:

scala> val ds = spark.read.json("/Users/takezoe/test.json")
scala> ds.createOrReplaceTempView("test")
scala> val df = spark.sql("SELECT maker, p.name, p.price FROM (SELECT maker, explode(products) as p FROM test)")
scala> df.show
+-----+-------+------+
|maker|   name| price|
+-----+-------+------+
|Apple| iPhobe|100000|
|Apple|   iPad|120000|
| ASUS|Zenfone| 20000|
+-----+-------+------+