$ sbt clean compile test

gistfile1.txt

[info] welcome to sbt 1.3.13 (AdoptOpenJDK Java 11.0.7)
[info] loading global plugins from C:\Users\paul\.sbt\1.0\plugins
[info] loading settings for project sparksql-scalapb-build from plugins.sbt ...
[info] loading project definition from C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\project
[info] loading settings for project root from build.sbt,sonatype.sbt,version.sbt ...
[info] set current project to root (in build file:/C:/Users/paul/Documents/GitHub/forks/sparksql-scalapb/)
[info] Executing in batch mode. For better performance use sbt's shell
[success] Total time: 0 s, completed Sep 29, 2020, 4:12:35 PM
[info] Compiling 9 Scala sources to C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\scala-2.12\classes ...
[info] Done compiling.
[success] Total time: 8 s, completed Sep 29, 2020, 4:12:43 PM
[info] Compiling 8 protobuf files to C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\scala-2.12\src_managed\test
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\all_types2.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\base.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\defaultsv3.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\wrappers.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\defaults.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\maps.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\demo.proto
[info] Compiling schema C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\all_types3.proto
protoc-jar: protoc version: 3.11.4, detected platform: windows-x86_64 (windows 10/amd64)
protoc-jar: embedded: bin/3.11.4/protoc-3.11.4-windows-x86_64.exe
protoc-jar: executing: [C:\Users\paul\AppData\Local\Temp\protocjar893352014095741815\bin\protoc.exe, --plugin=protoc-gen-jvm_0=C:\Users\paul\AppData\Local\Temp\protocbridge14870202225594409922.bat, --jvm_0_out=C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\scala-2.12\src_managed\test, -IC:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\main\protobuf, -IC:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\protobuf_external_src, -IC:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\protobuf_external, -IC:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\base.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\wrappers.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\demo.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\defaultsv3.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\all_types2.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\maps.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\defaults.proto, C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\src\test\protobuf\all_types3.proto]
[info] Compiling 60 Scala sources to C:\Users\paul\Documents\GitHub\forks\sparksql-scalapb\sparksql-scalapb\target\scala-2.12\test-classes ...
[info] Done compiling.
[info] DefaultsSpec:
[info] Proto2 RDD[DefaultsRequired]
[info] - should have non-null default values after converting to Dataframe
[info] Proto2 RDD[DefaultsOptional]
[info] - should have null values after converting to Dataframe
[info] Proto3 RDD[DefaultsV3]
[info] - should have non-null default values after converting to Dataframe
[info] SchemaOptionsSpec:
[info] converting df with primitive wrappers
[info] - should unpack primitive wrappers by default
[info] converting df with primitive wrappers
[info] - should retain value field when option is set
[info] schema
[info] - should use scalaNames when option is set
+------------+
|  attributes|
+------------+
|[foo -> bar]|
+------------+

[info] MapsSpec:
[info] converting maps to df
[info] - should work
root
 |-- data: binary (nullable = true)

root
 |-- sha1(data): string (nullable = true)

+------+---+------+----+-------------------+----+-----+----------+
|  name|age|gender|tags|          addresses|base|inner|      data|
+------+---+------+----+-------------------+----+-----+----------+
|Owen M| 35|  MALE|  []|[[, San Francisco]]|null| null|[01 02 03]|
+------+---+------+----+-------------------+----+-----+----------+

root
 |-- name: string (nullable = true)
 |-- age: integer (nullable = true)
 |-- gender: string (nullable = true)
 |-- tags: array (nullable = true)
 |    |-- element: string (containsNull = true)
 |-- addresses: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- street: string (nullable = true)
 |    |    |-- city: string (nullable = true)
 |-- base: struct (nullable = true)
 |-- inner: struct (nullable = true)
 |    |-- inner_value: string (nullable = true)
 |-- data: binary (nullable = true)

+------+---+--------------------+------+----+----+-----+----------+-------+----+
|  name|age|           addresses|gender|tags|base|inner|      data|address|nums|
+------+---+--------------------+------+----+----+-----+----------+-------+----+
|Owen M| 35|[[foo, bar], [baz...|  MALE|  []|null| [V1]|[01 02 03]|   null|  []|
+------+---+--------------------+------+----+----+-----+----------+-------+----+

+-------+--------------------+----------+
|eventId|              action|       foo|
+-------+--------------------+----------+
|    xyz|[type.googleapis....|[pK, foo]|
+-------+--------------------+----------+

[info] PersonSpec:
[info] mapping datasets
[info] - should work
[info] Creating person dataset
[info] - should work
[info] Creating enum dataset
[info] - should work
[info] Creating bytestring dataset
[info] - should work
[info] Dataset[Person]
[info] - should work
[info] as[SimplePerson]
[info] - should work for manual building *** FAILED ***
[info]   org.apache.spark.sql.AnalysisException: Try to map struct<name:string,age:int,addresses:array<struct<street:string,city:string>>,gender:string,tags:array<string>,base:struct<>,inner:struct<inner_value:string>,data:binary,address:struct<street:string,city:string>,nums:array<int>> to Tuple5, but failed as the number of fields does not line up.;
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$.fail(Analyzer.scala:3098)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveDeserializer$$validateTopLevelTupleFields(Analyzer.scala:3115)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$$anonfun$apply$31$$anonfun$applyOrElse$170.applyOrElse(Analyzer.scala:3067)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$$anonfun$apply$31$$anonfun$applyOrElse$170.applyOrElse(Analyzer.scala:3059)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:309)
[info]   at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:309)
[info]   at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$transformExpressionsDown$1(QueryPlan.scala:96)
[info]   at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$mapExpressions$1(QueryPlan.scala:118)
[info]   at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
[info]   ...
[info] as[Person]
[info] - should work for manual building *** FAILED ***
[info]   org.apache.spark.sql.AnalysisException: Try to map struct<name:string,age:int,addresses:array<struct<street:string,city:string>>,gender:string,tags:array<string>,base:struct<>,inner:struct<inner_value:string>,data:binary,address:struct<street:string,city:string>,nums:array<int>> to Tuple8, but failed as the number of fields does not line up.;
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$.fail(Analyzer.scala:3098)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveDeserializer$$validateTopLevelTupleFields(Analyzer.scala:3115)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$$anonfun$apply$31$$anonfun$applyOrElse$170.applyOrElse(Analyzer.scala:3067)
[info]   at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer$$anonfun$apply$31$$anonfun$applyOrElse$170.applyOrElse(Analyzer.scala:3059)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:309)
[info]   at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:309)
[info]   at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$transformExpressionsDown$1(QueryPlan.scala:96)
[info]   at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$mapExpressions$1(QueryPlan.scala:118)
[info]   at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
[info]   ...
[info] converting from rdd to dataframe
[info] - should work
[info] selecting message fields into dataset should work
[info] - should work
[info] serialize and deserialize
[info] - should work on dataset of bytes
[info] UDFs that involve protos
[info] - should work when using ProtoSQL.udfs
[info] UDFs that returns protos
[info] - should work when using ProtoSQL.createDataFrame
[info] UDFs that returns protos
[info] - should work when reading local files
[info] OuterCaseClass
[info] - should use our type encoders
[info] AllTypesSpec:
[info] AllTypes
[info] - should work for int32 *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(Int32Test(Some(1),Some(1229914175),Some(0),Some(0),Some(-2147483648),-1,1,2147483647,-2147483648,1666010783,Vector(),Vector(),Vector(),Vector(),Vector())) // 3 shrinks
[info]     )
[info] - should work for int64 *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(Int64Test(Some(1),Some(-9223372036854775808),None,Some(-9223372036854775808),Some(0),-3885044029316536083,6832096517673684120,8348352819039621292,1,6809668556522415544,Vector(-8854076712796512039),Vector(),Vector(),Vector(0),Vector())) // 3 shrinks
[info]     )
[info] - should work for bools *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(BoolTest(Some(true),true,Vector(true))) // 3 shrinks
[info]     )
[info] - should work for strings *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: Array(null) did not contain the same elements as Vector(StringTest(None,욂Ⱪ炠,Vector()))
[info]     Location: (AllTypesSpec.scala:43)
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(StringTest(None,욂Ⱪ炠,Vector())) // 2 shrinks
[info]     )
[info] - should work for floats *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(FloatTest(Some(1.950832E-16),-1.8113771E-22,Vector())) // 2 shrinks
[info]     )
[info] - should work for doubles *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(DoubleTest(Some(1.0324295084573865E243),1.2392118057976329E-49,Vector(-4.2813577237598723E-72)))
[info]     )
[info] - should work for bytes *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: Array(null) did not contain the same elements as Vector(BytesTest(None,<ByteString@3f9cd00d size=3 contents="\2153\244">,Vector(<ByteString@75e79093 size=2 contents="\2425">, <ByteString@6966f434 size=2 contents="\000\005">)))
[info]     Location: (AllTypesSpec.scala:43)
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(BytesTest(None,<ByteString@3f9cd00d size=3 contents="\2153\244">,Vector(<ByteString@75e79093 size=2 contents="\2425">, <ByteString@6966f434 size=2 contents="\000\005">))) // 3 shrinks
[info]     )
[info] - should work for enums *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(EnumTest(Some(BAZ),BAZ,Vector(BAZ),Some(FOREIGN_BAZ),FOREIGN_BAR,Vector())) // 2 shrinks
[info]     )
[info] - should work for messages *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(MessageTest(Some(NestedMessage(Some(-1575184072))),NestedMessage(Some(1)),Vector(NestedMessage(Some(-1560820724))),Some(TopLevelMessage(Some(1))),TopLevelMessage(Some(1242203210)),Vector(TopLevelMessage(Some(1))))) // 3 shrinks
[info]     )
[info] - should work for oneofs *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: Array(null) did not contain the same elements as Vector(OneofTest(OneofBytes(<ByteString@2acddafe size=8 contents="\177A\365\177#\001oc">)))
[info]     Location: (AllTypesSpec.scala:43)
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(OneofTest(OneofBytes(<ByteString@2acddafe size=8 contents="\177A\365\177#\001oc">))) // 2 shrinks
[info]     )
[info] - should work for levels *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: Array(null) did not contain the same elements as Vector(Level1(None,Some(频獰⮟)))
[info]     Location: (AllTypesSpec.scala:43)
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(Level1(None,Some(频獰⮟))) // 4 shrinks
[info]     )
[info] - should work for any *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: Array(null) did not contain the same elements as Vector(AnyTest(None))
[info]     Location: (AllTypesSpec.scala:43)
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(AnyTest(None)) // 3 shrinks
[info]     )
[info] - should work for time types *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(WellKnownTypes(Some(Timestamp(1589022557055990817,1,UnknownFieldSet(Map()))),Some(Duration(1,-2147483648,UnknownFieldSet(Map()))),Vector(Timestamp(7578350744175665721,-2147483648,UnknownFieldSet(Map()))),Vector(Duration(5320457540890993270,1564860612,UnknownFieldSet(Map()))))) // 1 shrink
[info]     )
[info] - should work for wrapper types *** FAILED ***
[info]   IllegalArgumentException was thrown during property evaluation.
[info]     Message: requirement failed: Join keys from two sides should have same types
[info]     Occurred when passed generated values (
[info]       arg0 = Vector(WrappersTest(Some(false),Some(<ByteString@3ad274c2 size=0 contents="">),None,Some(1.2095752E-18),Some(14745403),Some(0),Some(),Some(1),Some(1),Vector(),Vector(<ByteString@15b8507f size=0 contents="">),Vector(),Vector(),Vector(),Vector(9223372036854775807),Vector(),Vector(),Vector())) // 1 shrink
[info]     )
[info] - should work for maps
[info] ScalaTest
[info] Run completed in 17 seconds, 294 milliseconds.
[info] Total number of tests run: 36
[info] Suites: completed 5, aborted 0
[info] Tests: succeeded 20, failed 16, canceled 0, ignored 0, pending 0
[info] *** 16 TESTS FAILED ***
[error] Failed: Total 36, Failed 16, Errors 0, Passed 20
[error] Failed tests:
[error] 	scalapb.spark.PersonSpec
[error] 	scalapb.spark.AllTypesSpec
[error] (sparkSqlScalaPB / Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 55 s, completed Sep 29, 2020, 4:13:38 PM