Skip to content

Instantly share code, notes, and snippets.

@ericbartell
Created May 4, 2021 20:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ericbartell/34e38a8076b5f48e30fcaa3f55157572 to your computer and use it in GitHub Desktop.
Save ericbartell/34e38a8076b5f48e30fcaa3f55157572 to your computer and use it in GitHub Desktop.
hail error: py4j.protocol.Py4JNetworkError v2
2021-05-04 19:17:20 Hail: INFO: Running Hail version 0.2.66-50e9d4de51e1
2021-05-04 19:17:20 SparkContext: WARN: Using an existing SparkContext; some configuration may not take effect.
2021-05-04 19:17:21 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:21 root: INFO: globbing path /broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen returned 1 files: ukb_imp_chr22_v3.bgen
2021-05-04 19:17:22 MemoryStore: INFO: Block broadcast_0 stored as values in memory (estimated size 34.7 KiB, free 4.1 GiB)
2021-05-04 19:17:22 MemoryStore: INFO: Block broadcast_0_piece0 stored as bytes in memory (estimated size 3.2 KiB, free 4.1 GiB)
2021-05-04 19:17:22 BlockManagerInfo: INFO: Added broadcast_0_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 3.2 KiB, free: 4.1 GiB)
2021-05-04 19:17:22 SparkContext: INFO: Created broadcast 0 from broadcast at SparkBackend.scala:275
2021-05-04 19:17:22 root: INFO: decoder cache miss (0 hits, 1 misses, 0.000
2021-05-04 19:17:22 root: INFO: instruction count: 3: __C1FSContainer.<init>
2021-05-04 19:17:22 root: INFO: instruction count: 3: __C1FSContainer.<clinit>
2021-05-04 19:17:23 root: INFO: instruction count: 3: __C2etypeDecode.<init>
2021-05-04 19:17:23 root: INFO: instruction count: 5: __C2etypeDecode.apply
2021-05-04 19:17:23 root: INFO: instruction count: 30: __C2etypeDecode.__m3DECODE_o_struct_of_r_int64ANDr_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDENDEND_TO_o_tuple_of_r_int64ANDr_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 10: __C2etypeDecode.__m4INPLACE_DECODE_r_int64_TO_r_int64
2021-05-04 19:17:23 root: INFO: instruction count: 64: __C2etypeDecode.__m5INPLACE_DECODE_r_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 60: __C2etypeDecode.__m6INPLACE_DECODE_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 71: __C2etypeDecode.__m7INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_locusANDo_array_of_o_stringEND
2021-05-04 19:17:23 root: INFO: instruction count: 22: __C2etypeDecode.__m8INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_locus
2021-05-04 19:17:23 root: INFO: instruction count: 33: __C2etypeDecode.__m9INPLACE_DECODE_r_binary_TO_r_string
2021-05-04 19:17:23 root: INFO: instruction count: 10: __C2etypeDecode.__m10INPLACE_DECODE_r_int32_TO_r_int32
2021-05-04 19:17:23 root: INFO: instruction count: 120: __C2etypeDecode.__m11INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string
2021-05-04 19:17:23 root: INFO: instruction count: 33: __C2etypeDecode.__m12INPLACE_DECODE_o_binary_TO_o_string
2021-05-04 19:17:23 root: INFO: instruction count: 8: __C2etypeDecode.__m13INPLACE_DECODE_r_struct_of_END_TO_r_tuple_of_END
2021-05-04 19:17:23 root: INFO: decoder cache miss (0 hits, 2 misses, 0.000
2021-05-04 19:17:23 root: INFO: instruction count: 3: __C14FSContainer.<init>
2021-05-04 19:17:23 root: INFO: instruction count: 3: __C14FSContainer.<clinit>
2021-05-04 19:17:23 root: INFO: instruction count: 3: __C15etypeDecode.<init>
2021-05-04 19:17:23 root: INFO: instruction count: 5: __C15etypeDecode.apply
2021-05-04 19:17:23 root: INFO: instruction count: 23: __C15etypeDecode.__m16DECODE_o_struct_of_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDENDEND_TO_o_tuple_of_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 64: __C15etypeDecode.__m17INPLACE_DECODE_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 74: __C15etypeDecode.__m18INPLACE_DECODE_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND
2021-05-04 19:17:23 root: INFO: instruction count: 10: __C15etypeDecode.__m19INPLACE_DECODE_r_int64_TO_r_int64
2021-05-04 19:17:23 root: INFO: instruction count: 71: __C15etypeDecode.__m20INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_locusANDo_array_of_o_stringEND
2021-05-04 19:17:23 root: INFO: instruction count: 22: __C15etypeDecode.__m21INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_locus
2021-05-04 19:17:23 root: INFO: instruction count: 33: __C15etypeDecode.__m22INPLACE_DECODE_r_binary_TO_r_string
2021-05-04 19:17:23 root: INFO: instruction count: 10: __C15etypeDecode.__m23INPLACE_DECODE_r_int32_TO_r_int32
2021-05-04 19:17:23 root: INFO: instruction count: 120: __C15etypeDecode.__m24INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string
2021-05-04 19:17:23 root: INFO: instruction count: 33: __C15etypeDecode.__m25INPLACE_DECODE_o_binary_TO_o_string
2021-05-04 19:17:23 root: INFO: instruction count: 8: __C15etypeDecode.__m26INPLACE_DECODE_r_struct_of_END_TO_r_tuple_of_END
2021-05-04 19:17:23 root: INFO: RegionPool: REPORT_THRESHOLD: 320.0K allocated (128.0K blocks / 192.0K chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:23 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (320.0K blocks / 192.0K chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:23 root: INFO: Index reader cache queries: 4
2021-05-04 19:17:23 root: INFO: Index reader cache hit rate: 0.25
2021-05-04 19:17:25 Hail: INFO: Number of BGEN files parsed: 1
2021-05-04 19:17:25 Hail: INFO: Number of samples in BGEN files: 487409
2021-05-04 19:17:25 Hail: INFO: Number of variants across all BGEN files: 1255683
2021-05-04 19:17:25 MemoryStore: INFO: Block broadcast_1 stored as values in memory (estimated size 294.6 KiB, free 4.1 GiB)
2021-05-04 19:17:25 MemoryStore: INFO: Block broadcast_1_piece0 stored as bytes in memory (estimated size 27.1 KiB, free 4.1 GiB)
2021-05-04 19:17:25 BlockManagerInfo: INFO: Added broadcast_1_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 27.1 KiB, free: 4.1 GiB)
2021-05-04 19:17:25 SparkContext: INFO: Created broadcast 1 from broadcast at SparkBackend.scala:275
2021-05-04 19:17:25 root: INFO: decoder cache hit
2021-05-04 19:17:25 root: INFO: decoder cache hit
2021-05-04 19:17:27 root: INFO: Index reader cache queries: 1172
2021-05-04 19:17:27 root: INFO: Index reader cache hit rate: 0.7482935153583617
2021-05-04 19:17:27 root: INFO: RegionPool: FREE: 384.0K allocated (384.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:27 root: INFO: timing SparkBackend.parse_matrix_ir total 5.689s self 5.689s children 0.000ms %children 0.00%
2021-05-04 19:17:27 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:27 root: INFO: globbing path /broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised returned 1 files: 3_11_clust_akshaya_test_height_chrAll_5.best_noised
2021-05-04 19:17:28 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:28 root: INFO: timing SparkBackend.parse_table_ir total 174.775ms self 174.775ms children 0.000ms %children 0.00%
2021-05-04 19:17:28 Hail: INFO: Reading table to impute column types
2021-05-04 19:17:28 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:28 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:28 root: INFO: timing SparkBackend.parse_value_ir total 62.312ms self 62.312ms children 0.000ms %children 0.00%
2021-05-04 19:17:28 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:28 root: INFO: starting execution of query hail_query_1 of initial size 14
2021-05-04 19:17:28 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 14:
(TableAggregate
(TableRead None False
"{\"files\":[\"/broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised\"],\"typeMapStr\":{\"FID\":\"String\",\"IID\":\"String\",\"Noise_added\":\"Float64\"},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{},\"forceGZ\":false,\"name\":\"TextTableReader\"}")
(AggArrayPerElement __uid_3 unused False False
(MakeArray Array[String]
(GetField In_Regression (Ref row))
(GetField PRS (Ref row))
(GetField float_PRS (Ref row))
(GetField Noise (Ref row)))
(ApplyAggOp ImputeType () ((Ref __uid_3)))))
2021-05-04 19:17:28 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 14:
(TableAggregate
(TableRead
Table{global:Struct{},key:[],row:Struct{In_Regression:String,PRS:String,float_PRS:String,Noise:String}}
False
"{\"files\":[\"/broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised\"],\"typeMapStr\":{\"FID\":\"String\",\"IID\":\"String\",\"Noise_added\":\"Float64\"},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{},\"forceGZ\":false,\"name\":\"TextTableReader\"}")
(AggArrayPerElement __iruid_4 __iruid_5 False
False
(MakeArray Array[String]
(GetField In_Regression (Ref row))
(GetField PRS (Ref row))
(GetField float_PRS (Ref row))
(GetField Noise (Ref row)))
(ApplyAggOp ImputeType () ((Ref __iruid_4)))))
2021-05-04 19:17:28 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 14:
(TableAggregate
(TableRead
Table{global:Struct{},key:[],row:Struct{In_Regression:String,PRS:String,float_PRS:String,Noise:String}}
False
"{\"files\":[\"/broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised\"],\"typeMapStr\":{\"FID\":\"String\",\"IID\":\"String\",\"Noise_added\":\"Float64\"},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{},\"forceGZ\":false,\"name\":\"TextTableReader\"}")
(AggArrayPerElement __iruid_4 __iruid_5 False
False
(MakeArray Array[String]
(GetField In_Regression (Ref row))
(GetField PRS (Ref row))
(GetField float_PRS (Ref row))
(GetField Noise (Ref row)))
(ApplyAggOp ImputeType () ((Ref __iruid_4)))))
2021-05-04 19:17:28 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 14:
(TableAggregate
(TableRead
Table{global:Struct{},key:[],row:Struct{In_Regression:String,PRS:String,float_PRS:String,Noise:String}}
False
"{\"files\":[\"/broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised\"],\"typeMapStr\":{\"FID\":\"String\",\"IID\":\"String\",\"Noise_added\":\"Float64\"},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{},\"forceGZ\":false,\"name\":\"TextTableReader\"}")
(AggArrayPerElement __iruid_10 __iruid_11 False
False
(MakeArray Array[String]
(GetField In_Regression (Ref row))
(GetField PRS (Ref row))
(GetField float_PRS (Ref row))
(GetField Noise (Ref row)))
(ApplyAggOp ImputeType () ((Ref __iruid_10)))))
2021-05-04 19:17:28 root: INFO: interpreting non compilable node: TableAggregate
2021-05-04 19:17:28 root: INFO: encoder cache miss (0 hits, 1 misses, 0.000)
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C27FSContainer.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C27FSContainer.<clinit>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C28etypeEncode.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 5: __C28etypeEncode.apply
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C28etypeEncode.__m29ENCODE_r_tuple_of_END_TO_r_struct_of_END
2021-05-04 19:17:28 root: INFO: decoder cache miss (2 hits, 3 misses, 0.400
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C30FSContainer.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C30FSContainer.<clinit>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C31etypeDecode.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 5: __C31etypeDecode.apply
2021-05-04 19:17:28 root: INFO: instruction count: 16: __C31etypeDecode.__m32DECODE_r_struct_of_END_TO_r_tuple_of_END
2021-05-04 19:17:28 MemoryStore: INFO: Block broadcast_2 stored as values in memory (estimated size 2.1 KiB, free 4.1 GiB)
2021-05-04 19:17:28 MemoryStore: INFO: Block broadcast_2_piece0 stored as bytes in memory (estimated size 1233.0 B, free 4.1 GiB)
2021-05-04 19:17:28 BlockManagerInfo: INFO: Added broadcast_2_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 1233.0 B, free: 4.1 GiB)
2021-05-04 19:17:28 SparkContext: INFO: Created broadcast 2 from broadcast at SparkBackend.scala:275
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 4:
(Begin
(InitOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((Begin
(InitOp 0 (ImputeType (ImputeTypeStateSig))
())))))
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C33FSContainer.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C33FSContainer.<clinit>
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C34CompiledWithAggs.<init>
2021-05-04 19:17:28 root: INFO: instruction count: 65: __C34CompiledWithAggs.apply
2021-05-04 19:17:28 root: INFO: instruction count: 35: __C34CompiledWithAggs.newAggState
2021-05-04 19:17:28 root: INFO: instruction count: 81: __C34CompiledWithAggs.setAggState
2021-05-04 19:17:28 root: INFO: instruction count: 5: __C34CompiledWithAggs.getAggOffset
2021-05-04 19:17:28 root: INFO: instruction count: 35: __C34CompiledWithAggs.storeAggsToRegion
2021-05-04 19:17:28 root: INFO: instruction count: 5: __C34CompiledWithAggs.setNumSerialized
2021-05-04 19:17:28 root: INFO: instruction count: 6: __C34CompiledWithAggs.setSerializedAgg
2021-05-04 19:17:28 root: INFO: instruction count: 5: __C34CompiledWithAggs.getSerializedAgg
2021-05-04 19:17:28 root: INFO: instruction count: 9: __C34CompiledWithAggs.setPartitionIndex
2021-05-04 19:17:28 root: INFO: instruction count: 4: __C34CompiledWithAggs.addPartitionRegion
2021-05-04 19:17:28 root: INFO: instruction count: 4: __C34CompiledWithAggs.setPool
2021-05-04 19:17:28 root: INFO: instruction count: 3: __C34CompiledWithAggs.addFS
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 31:
(Begin
(Let __iruid_14
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_14))))
(StreamFor __iruid_15
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_14))
(I32 1))
(Let __iruid_10
(ArrayRef
(Ref __iruid_14)
(Ref __iruid_15)
(Str ""))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_15)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((Ref __iruid_10)))))))))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 29:
(Begin
(Let __iruid_22
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_22))))
(StreamFor __iruid_23
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_22))
(I32 1))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_23)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((ArrayRef
(Ref __iruid_22)
(Ref __iruid_23)
(Str "")))))))))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 29:
(Begin
(Let __iruid_22
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_22))))
(StreamFor __iruid_23
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_22))
(I32 1))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_23)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((ArrayRef
(Ref __iruid_22)
(Ref __iruid_23)
(Str "")))))))))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 29:
(Begin
(Let __iruid_28
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_28))))
(StreamFor __iruid_29
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_28))
(I32 1))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_29)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((ArrayRef
(Ref __iruid_28)
(Ref __iruid_29)
(Str "")))))))))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 29:
(Begin
(Let __iruid_28
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_28))))
(StreamFor __iruid_29
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_28))
(I32 1))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_29)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((ArrayRef
(Ref __iruid_28)
(Ref __iruid_29)
(Str "")))))))))))
2021-05-04 19:17:28 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 29:
(Begin
(Let __iruid_34
(MakeArray Array[String]
(GetField In_Regression
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField float_PRS
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1))
(GetField Noise
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{In_Regression:PCString,PRS:PCString,float_PRS:PCString,Noise:PCString}))
1)))
(Begin
(SeqOp 0
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))
((ArrayLen (Ref __iruid_34))))
(StreamFor __iruid_35
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_34))
(I32 1))
(SeqOp 0
(AggElements ((ImputeType (ImputeTypeStateSig))))
((Ref __iruid_35)
(Begin
(SeqOp 0 (ImputeType (ImputeTypeStateSig))
((ArrayRef
(Ref __iruid_34)
(Ref __iruid_35)
(Str "")))))))))))
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C48FSContainer.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C48FSContainer.<clinit>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C49CompiledWithAggs.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 455: __C49CompiledWithAggs.apply
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C49CompiledWithAggs.newAggState
2021-05-04 19:17:29 root: INFO: instruction count: 81: __C49CompiledWithAggs.setAggState
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C49CompiledWithAggs.getAggOffset
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C49CompiledWithAggs.storeAggsToRegion
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C49CompiledWithAggs.setNumSerialized
2021-05-04 19:17:29 root: INFO: instruction count: 6: __C49CompiledWithAggs.setSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C49CompiledWithAggs.getSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 42: __C49CompiledWithAggs.__m74arrayref_bounds_check
2021-05-04 19:17:29 root: INFO: instruction count: 9: __C49CompiledWithAggs.setPartitionIndex
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C49CompiledWithAggs.addPartitionRegion
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C49CompiledWithAggs.setPool
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C49CompiledWithAggs.addFS
2021-05-04 19:17:29 root: INFO: instruction count: 443: __C49CompiledWithAggs.applyregion24_68
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C77applySpills.<init>
2021-05-04 19:17:29 root: INFO: Aggregate: useTreeAggregate=true
2021-05-04 19:17:29 root: INFO: Aggregate: commutative=true
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1:
(DeserializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C82FSContainer.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C82FSContainer.<clinit>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C83CompiledWithAggs.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 192: __C83CompiledWithAggs.apply
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C83CompiledWithAggs.newAggState
2021-05-04 19:17:29 root: INFO: instruction count: 81: __C83CompiledWithAggs.setAggState
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C83CompiledWithAggs.getAggOffset
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C83CompiledWithAggs.storeAggsToRegion
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C83CompiledWithAggs.setNumSerialized
2021-05-04 19:17:29 root: INFO: instruction count: 6: __C83CompiledWithAggs.setSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C83CompiledWithAggs.getSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 9: __C83CompiledWithAggs.setPartitionIndex
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C83CompiledWithAggs.addPartitionRegion
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C83CompiledWithAggs.setPool
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C83CompiledWithAggs.addFS
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1:
(SerializeAggs 0 0
"{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}"
((ArrayAggStateSig ((ImputeTypeStateSig)))))
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C98FSContainer.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C98FSContainer.<clinit>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C99CompiledWithAggs.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 124: __C99CompiledWithAggs.apply
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C99CompiledWithAggs.newAggState
2021-05-04 19:17:29 root: INFO: instruction count: 81: __C99CompiledWithAggs.setAggState
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C99CompiledWithAggs.getAggOffset
2021-05-04 19:17:29 root: INFO: instruction count: 35: __C99CompiledWithAggs.storeAggsToRegion
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C99CompiledWithAggs.setNumSerialized
2021-05-04 19:17:29 root: INFO: instruction count: 6: __C99CompiledWithAggs.setSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 5: __C99CompiledWithAggs.getSerializedAgg
2021-05-04 19:17:29 root: INFO: instruction count: 9: __C99CompiledWithAggs.setPartitionIndex
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C99CompiledWithAggs.addPartitionRegion
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C99CompiledWithAggs.setPool
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C99CompiledWithAggs.addFS
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C115FSContainer.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C115FSContainer.<clinit>
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C116combOpF3.<init>
2021-05-04 19:17:29 root: INFO: instruction count: 186: __C116combOpF3.apply
2021-05-04 19:17:29 root: INFO: instruction count: 9: __C116combOpF3.setPartitionIndex
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C116combOpF3.addPartitionRegion
2021-05-04 19:17:29 root: INFO: instruction count: 4: __C116combOpF3.setPool
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C116combOpF3.addFS
2021-05-04 19:17:29 root: INFO: instruction count: 171: __C116combOpF3.applyregion29_43
2021-05-04 19:17:29 root: INFO: instruction count: 432: __C116combOpF3.applyregion18_69
2021-05-04 19:17:29 root: INFO: instruction count: 3: __C138applySpills.<init>
2021-05-04 19:17:29 SparkContext: INFO: Starting job: runJob at RVD.scala:725
2021-05-04 19:17:29 DAGScheduler: INFO: Got job 0 (runJob at RVD.scala:725) with 1 output partitions
2021-05-04 19:17:29 DAGScheduler: INFO: Final stage: ResultStage 0 (runJob at RVD.scala:725)
2021-05-04 19:17:29 DAGScheduler: INFO: Parents of final stage: List()
2021-05-04 19:17:29 DAGScheduler: INFO: Missing parents: List()
2021-05-04 19:17:29 DAGScheduler: INFO: Submitting ResultStage 0 (MapPartitionsRDD[3] at mapPartitions at ContextRDD.scala:168), which has no missing parents
2021-05-04 19:17:29 MemoryStore: INFO: Block broadcast_3 stored as values in memory (estimated size 30.1 KiB, free 4.1 GiB)
2021-05-04 19:17:29 MemoryStore: INFO: Block broadcast_3_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 4.1 GiB)
2021-05-04 19:17:29 BlockManagerInfo: INFO: Added broadcast_3_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 15.3 KiB, free: 4.1 GiB)
2021-05-04 19:17:29 SparkContext: INFO: Created broadcast 3 from broadcast at DAGScheduler.scala:1383
2021-05-04 19:17:29 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at mapPartitions at ContextRDD.scala:168) (first 15 tasks are for partitions Vector(0))
2021-05-04 19:17:29 TaskSchedulerImpl: INFO: Adding task set 0.0 with 1 tasks resource profile 0
2021-05-04 19:17:29 TaskSetManager: INFO: Starting task 0.0 in stage 0.0 (TID 0) (ugertmp-c021.broadinstitute.org, executor driver, partition 0, PROCESS_LOCAL, 4653 bytes) taskResourceAssignments Map()
2021-05-04 19:17:29 Executor: INFO: Running task 0.0 in stage 0.0 (TID 0)
2021-05-04 19:17:30 root: INFO: RegionPool: initialized for thread 53: Executor task launch worker for task 0.0 in stage 0.0 (TID 0)
2021-05-04 19:17:32 root: INFO: TaskReport: stage=0, partition=0, attempt=0, peakBytes=155648, peakBytesReadable=152.00 KiB
2021-05-04 19:17:32 root: INFO: RegionPool: FREE: 152.0K allocated (152.0K blocks / 0 chunks), regions.size = 5, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 0.0 (TID 0)
2021-05-04 19:17:32 Executor: INFO: Finished task 0.0 in stage 0.0 (TID 0). 917 bytes result sent to driver
2021-05-04 19:17:32 TaskSetManager: INFO: Finished task 0.0 in stage 0.0 (TID 0) in 3001 ms on ugertmp-c021.broadinstitute.org (executor driver) (1/1)
2021-05-04 19:17:32 TaskSchedulerImpl: INFO: Removed TaskSet 0.0, whose tasks have all completed, from pool
2021-05-04 19:17:32 DAGScheduler: INFO: ResultStage 0 (runJob at RVD.scala:725) finished in 3.130 s
2021-05-04 19:17:32 DAGScheduler: INFO: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job
2021-05-04 19:17:32 TaskSchedulerImpl: INFO: Killing all running tasks in stage 0: Stage finished
2021-05-04 19:17:32 DAGScheduler: INFO: Job 0 finished: runJob at RVD.scala:725, took 3.214916 s
2021-05-04 19:17:32 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 20:
(Let __iruid_12
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_16
(GetTupleElement 0 (Ref __iruid_12))
(ToArray
(StreamMap __iruid_11
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_16))
(I32 1))
(Let __iruid_13
(ArrayRef
(Ref __iruid_16)
(Ref __iruid_11)
(Str ""))
(GetTupleElement 0 (Ref __iruid_13))))))))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 18:
(Let __iruid_43
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_44
(GetTupleElement 0 (Ref __iruid_43))
(ToArray
(StreamMap __iruid_45
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_44))
(I32 1))
(GetTupleElement 0
(ArrayRef
(Ref __iruid_44)
(Ref __iruid_45)
(Str ""))))))))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 18:
(Let __iruid_43
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_44
(GetTupleElement 0 (Ref __iruid_43))
(ToArray
(StreamMap __iruid_45
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_44))
(I32 1))
(GetTupleElement 0
(ArrayRef
(Ref __iruid_44)
(Ref __iruid_45)
(Str ""))))))))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 18:
(Let __iruid_52
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_53
(GetTupleElement 0 (Ref __iruid_52))
(ToArray
(StreamMap __iruid_54
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_53))
(I32 1))
(GetTupleElement 0
(ArrayRef
(Ref __iruid_53)
(Ref __iruid_54)
(Str ""))))))))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 18:
(Let __iruid_52
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_53
(GetTupleElement 0 (Ref __iruid_52))
(ToArray
(StreamMap __iruid_54
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_53))
(I32 1))
(GetTupleElement 0
(ArrayRef
(Ref __iruid_53)
(Ref __iruid_54)
(Str ""))))))))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 18:
(Let __iruid_61
(ResultOp 0 (
(ArrayLen False (
(ImputeType (ImputeTypeStateSig))))))
(MakeTuple (0)
(Let __iruid_62
(GetTupleElement 0 (Ref __iruid_61))
(ToArray
(StreamMap __iruid_63
(StreamRange False
(I32 0)
(ArrayLen (Ref __iruid_62))
(I32 1))
(GetTupleElement 0
(ArrayRef
(Ref __iruid_62)
(Ref __iruid_63)
(Str ""))))))))
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C146FSContainer.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C146FSContainer.<clinit>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C147CompiledWithAggs.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 352: __C147CompiledWithAggs.apply
2021-05-04 19:17:33 root: INFO: instruction count: 35: __C147CompiledWithAggs.newAggState
2021-05-04 19:17:33 root: INFO: instruction count: 81: __C147CompiledWithAggs.setAggState
2021-05-04 19:17:33 root: INFO: instruction count: 5: __C147CompiledWithAggs.getAggOffset
2021-05-04 19:17:33 root: INFO: instruction count: 35: __C147CompiledWithAggs.storeAggsToRegion
2021-05-04 19:17:33 root: INFO: instruction count: 5: __C147CompiledWithAggs.setNumSerialized
2021-05-04 19:17:33 root: INFO: instruction count: 6: __C147CompiledWithAggs.setSerializedAgg
2021-05-04 19:17:33 root: INFO: instruction count: 5: __C147CompiledWithAggs.getSerializedAgg
2021-05-04 19:17:33 root: INFO: instruction count: 42: __C147CompiledWithAggs.__m174arrayref_bounds_check
2021-05-04 19:17:33 root: INFO: instruction count: 9: __C147CompiledWithAggs.setPartitionIndex
2021-05-04 19:17:33 root: INFO: instruction count: 4: __C147CompiledWithAggs.addPartitionRegion
2021-05-04 19:17:33 root: INFO: instruction count: 4: __C147CompiledWithAggs.setPool
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C147CompiledWithAggs.addFS
2021-05-04 19:17:33 root: INFO: instruction count: 321: __C147CompiledWithAggs.applyregion0_33
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C177applySpills.<init>
2021-05-04 19:17:33 root: INFO: took 4.690s
2021-05-04 19:17:33 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: before: IR size 1:
(Literal
Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: after: IR size 1:
(Literal
Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2:
(MakeTuple (0)
(Literal
Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]
<literal value>))
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1:
(Literal
Tuple[Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1:
(Literal
Tuple[Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1:
(Literal
Tuple[Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1:
(Literal
Tuple[Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]]
<literal value>)
2021-05-04 19:17:33 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1:
(Literal
Tuple[Array[Struct{anyNonMissing:Boolean,allDefined:Boolean,supportsBool:Boolean,supportsInt32:Boolean,supportsInt64:Boolean,supportsFloat64:Boolean}]]
<literal value>)
2021-05-04 19:17:33 root: INFO: encoder cache miss (0 hits, 2 misses, 0.000)
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C192FSContainer.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C192FSContainer.<clinit>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C193etypeEncode.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 5: __C193etypeEncode.apply
2021-05-04 19:17:33 root: INFO: instruction count: 9: __C193etypeEncode.__m194ENCODE_r_tuple_of_r_tuple_of_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDENDEND_TO_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDENDEND
2021-05-04 19:17:33 root: INFO: instruction count: 13: __C193etypeEncode.__m195ENCODE_r_tuple_of_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDEND_TO_r_struct_of_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDEND
2021-05-04 19:17:33 root: INFO: instruction count: 45: __C193etypeEncode.__m196ENCODE_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND_TO_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND
2021-05-04 19:17:33 root: INFO: instruction count: 63: __C193etypeEncode.__m197ENCODE_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND_TO_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND
2021-05-04 19:17:33 root: INFO: instruction count: 6: __C193etypeEncode.__m198ENCODE_r_bool_TO_r_bool
2021-05-04 19:17:33 MemoryStore: INFO: Block broadcast_4 stored as values in memory (estimated size 104.0 B, free 4.1 GiB)
2021-05-04 19:17:33 MemoryStore: INFO: Block broadcast_4_piece0 stored as bytes in memory (estimated size 82.0 B, free 4.1 GiB)
2021-05-04 19:17:33 BlockManagerInfo: INFO: Added broadcast_4_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 82.0 B, free: 4.1 GiB)
2021-05-04 19:17:33 SparkContext: INFO: Created broadcast 4 from broadcast at SparkBackend.scala:275
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C181FSContainer.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C181FSContainer.<clinit>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C182Compiled.<init>
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C182Compiled.apply
2021-05-04 19:17:33 root: INFO: instruction count: 9: __C182Compiled.setPartitionIndex
2021-05-04 19:17:33 root: INFO: instruction count: 4: __C182Compiled.addPartitionRegion
2021-05-04 19:17:33 root: INFO: instruction count: 4: __C182Compiled.setPool
2021-05-04 19:17:33 root: INFO: instruction count: 3: __C182Compiled.addFS
2021-05-04 19:17:33 root: INFO: instruction count: 32: __C182Compiled.addLiterals
2021-05-04 19:17:33 root: INFO: instruction count: 23: __C182Compiled.__m187DECODE_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDENDEND_TO_r_tuple_of_r_tuple_of_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDENDEND
2021-05-04 19:17:33 root: INFO: instruction count: 15: __C182Compiled.__m188INPLACE_DECODE_r_struct_of_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDEND_TO_r_tuple_of_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolENDEND
2021-05-04 19:17:33 root: INFO: instruction count: 64: __C182Compiled.__m189INPLACE_DECODE_r_array_of_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND_TO_r_array_of_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND
2021-05-04 19:17:33 root: INFO: instruction count: 50: __C182Compiled.__m190INPLACE_DECODE_r_struct_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND_TO_r_tuple_of_r_boolANDr_boolANDr_boolANDr_boolANDr_boolANDr_boolEND
2021-05-04 19:17:33 root: INFO: instruction count: 10: __C182Compiled.__m191INPLACE_DECODE_r_bool_TO_r_bool
2021-05-04 19:17:33 root: INFO: finished execution of query hail_query_1
2021-05-04 19:17:33 root: INFO: RegionPool: FREE: 168.0K allocated (168.0K blocks / 0 chunks), regions.size = 6, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON total 5.123s self 75.994ms children 5.047s %children 98.52%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR total 258.555ms self 1.072ms children 257.484ms %children 99.59%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.873ms self 0.873ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation total 256.545ms self 166.012ms children 90.533ms %children 35.29%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize total 90.533ms self 2.426ms children 88.107ms %children 97.32%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 6.627ms self 6.627ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 8.209ms self 8.209ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 18.710ms self 18.710ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 24.259ms self 24.259ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 2.414ms self 2.414ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 17.580ms self 17.580ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.675ms self 0.675ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.182ms self 0.182ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.423ms self 0.423ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.172ms self 2.172ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.210ms self 0.210ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.734ms self 1.734ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.524ms self 0.524ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.155ms self 0.155ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.443ms self 0.443ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.013ms self 2.013ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.183ms self 0.183ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.592ms self 1.592ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.066ms self 0.066ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable total 11.647ms self 0.021ms children 11.626ms %children 99.82%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.048ms self 0.048ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/LoweringTransformation total 11.493ms self 11.493ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.085ms self 0.085ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable total 23.153ms self 0.017ms children 23.137ms %children 99.93%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.048ms self 0.048ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation total 23.030ms self 6.647ms children 16.383ms %children 71.14%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 16.383ms self 0.248ms children 16.135ms %children 98.49%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.378ms self 0.378ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.212ms self 0.212ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.446ms self 0.446ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 2.221ms self 2.221ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.191ms self 0.191ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 1.666ms self 1.666ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.433ms self 0.433ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.154ms self 0.154ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.397ms self 0.397ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 3.666ms self 3.666ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.187ms self 0.187ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 1.546ms self 1.546ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.450ms self 0.450ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.159ms self 0.159ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.383ms self 0.383ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 1.965ms self 1.965ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.178ms self 0.178ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 1.502ms self 1.502ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.059ms self 0.059ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable total 4.697s self 0.032ms children 4.696s %children 100.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.058ms self 0.058ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation total 4.696s self 4.273s children 423.540ms %children 9.02%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 29.917ms self 0.025ms children 29.892ms %children 99.92%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.040ms self 0.040ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 29.826ms self 27.260ms children 2.566ms %children 8.60%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 2.566ms self 0.088ms children 2.478ms %children 96.55%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.185ms self 0.185ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.053ms self 0.053ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.122ms self 0.122ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.402ms self 1.402ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.068ms self 0.068ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.647ms self 0.647ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.027ms self 0.027ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.836ms self 0.017ms children 0.819ms %children 97.95%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.317ms self 0.317ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.470ms self 0.470ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.032ms self 0.032ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 3.277ms self 0.014ms children 3.264ms %children 99.58%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 3.231ms self 1.996ms children 1.234ms %children 38.21%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 1.234ms self 0.064ms children 1.170ms %children 94.80%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.107ms self 0.107ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.036ms self 0.036ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.090ms self 0.090ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.697ms self 0.697ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.067ms self 0.067ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.174ms self 0.174ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.019ms self 0.019ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 1.100ms self 0.014ms children 1.086ms %children 98.76%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.025ms self 0.025ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 0.769ms self 0.769ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.293ms self 0.293ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 8.792ms self 0.014ms children 8.779ms %children 99.84%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.015ms self 0.015ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 8.745ms self 7.131ms children 1.614ms %children 18.45%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 1.614ms self 0.065ms children 1.549ms %children 95.96%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.088ms self 0.088ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.038ms self 0.038ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.096ms self 0.096ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 1.071ms self 1.071ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.050ms self 0.050ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.205ms self 0.205ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.019ms self 0.019ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 45.198ms self 0.041ms children 45.156ms %children 99.91%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.224ms self 0.224ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 44.809ms self 9.863ms children 34.946ms %children 77.99%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 34.946ms self 0.152ms children 34.794ms %children 99.57%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.717ms self 0.717ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.229ms self 0.229ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.738ms self 0.738ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 7.293ms self 7.293ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.396ms self 0.396ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 2.425ms self 2.425ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.733ms self 0.733ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 5.162ms self 5.162ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.773ms self 0.773ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 4.397ms self 4.397ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.284ms self 0.284ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.356ms self 1.356ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.653ms self 0.653ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.214ms self 0.214ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.635ms self 0.635ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 7.055ms self 7.055ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.400ms self 0.400ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.334ms self 1.334ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.123ms self 0.123ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.714ms self 0.016ms children 0.698ms %children 97.76%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.138ms self 0.138ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.405ms self 0.405ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.156ms self 0.156ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 29.864ms self 0.018ms children 29.846ms %children 99.94%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.091ms self 0.091ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 29.650ms self 8.725ms children 20.925ms %children 70.57%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 20.925ms self 0.136ms children 20.789ms %children 99.35%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.444ms self 0.444ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.244ms self 0.244ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.708ms self 0.708ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 5.967ms self 5.967ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.251ms self 0.251ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.211ms self 1.211ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.590ms self 0.590ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.160ms self 0.160ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.533ms self 0.533ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 3.722ms self 3.722ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.164ms self 0.164ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.074ms self 1.074ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.395ms self 0.395ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.172ms self 0.172ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.542ms self 0.542ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 3.365ms self 3.365ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.162ms self 0.162ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.084ms self 1.084ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.105ms self 0.105ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 6.945ms self 0.025ms children 6.920ms %children 99.64%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.171ms self 0.171ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 6.595ms self 6.595ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.154ms self 0.154ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 24.172ms self 0.017ms children 24.155ms %children 99.93%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.093ms self 0.093ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 23.960ms self 6.414ms children 17.546ms %children 73.23%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 17.546ms self 0.134ms children 17.412ms %children 99.24%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.387ms self 0.387ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.196ms self 0.196ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.600ms self 0.600ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 3.461ms self 3.461ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.176ms self 0.176ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 1.182ms self 1.182ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.373ms self 0.373ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.147ms self 0.147ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.499ms self 0.499ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 3.045ms self 3.045ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.175ms self 0.175ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 1.073ms self 1.073ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.340ms self 0.340ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.148ms self 0.148ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.629ms self 0.629ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 3.714ms self 3.714ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.170ms self 0.170ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 1.098ms self 1.098ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.102ms self 0.102ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 51.703ms self 0.018ms children 51.685ms %children 99.97%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.013ms self 0.013ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 51.657ms self 51.094ms children 0.563ms %children 1.09%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 0.563ms self 0.052ms children 0.511ms %children 90.71%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.107ms self 0.107ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.034ms self 0.034ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.066ms self 0.066ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.185ms self 0.185ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.037ms self 0.037ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.082ms self 0.082ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.015ms self 0.015ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.052ms self 0.010ms children 0.042ms %children 80.57%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.020ms self 0.020ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.010ms self 0.010ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 7.536ms self 0.012ms children 7.525ms %children 99.85%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 7.511ms self 7.210ms children 0.301ms %children 4.01%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.301ms self 0.030ms children 0.271ms %children 90.05%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.040ms self 0.040ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.033ms self 0.033ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.114ms self 0.114ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.023ms self 0.023ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.049ms self 0.049ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.010ms self 0.010ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 0.153ms self 0.010ms children 0.143ms %children 93.53%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 0.120ms self 0.120ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 2.020ms self 0.011ms children 2.009ms %children 99.44%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 1.996ms self 1.680ms children 0.316ms %children 15.82%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.316ms self 0.031ms children 0.285ms %children 90.31%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.049ms self 0.049ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.015ms self 0.015ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.034ms self 0.034ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.113ms self 0.113ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.024ms self 0.024ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.051ms self 0.051ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 2.498ms self 0.017ms children 2.480ms %children 99.30%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.013ms self 0.013ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 2.457ms self 1.989ms children 0.469ms %children 19.07%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 0.469ms self 0.051ms children 0.417ms %children 89.10%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.094ms self 0.094ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.019ms self 0.019ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.050ms self 0.050ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.143ms self 0.143ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.035ms self 0.035ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.076ms self 0.076ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.048ms self 0.007ms children 0.041ms %children 84.96%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.019ms self 0.019ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.010ms self 0.010ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 1.890ms self 0.011ms children 1.879ms %children 99.41%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 1.866ms self 1.565ms children 0.301ms %children 16.11%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.301ms self 0.030ms children 0.271ms %children 90.14%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.039ms self 0.039ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.032ms self 0.032ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.113ms self 0.113ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.023ms self 0.023ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.051ms self 0.051ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 0.130ms self 0.009ms children 0.121ms %children 93.32%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.008ms self 0.008ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 0.102ms self 0.102ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 1.917ms self 0.012ms children 1.905ms %children 99.39%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 1.892ms self 1.594ms children 0.298ms %children 15.75%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.298ms self 0.032ms children 0.266ms %children 89.16%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.039ms self 0.039ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.033ms self 0.033ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.110ms self 0.110ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.023ms self 0.023ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.049ms self 0.049ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 131.013ms self 0.019ms children 130.994ms %children 99.99%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.126ms self 0.126ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 130.801ms self 104.099ms children 26.702ms %children 20.41%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 26.702ms self 0.100ms children 26.602ms %children 99.63%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 1.418ms self 1.418ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.146ms self 0.146ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.456ms self 0.456ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.377ms self 2.377ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.256ms self 0.256ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 5.377ms self 5.377ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.324ms self 0.324ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.095ms self 0.095ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.251ms self 0.251ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.971ms self 1.971ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.206ms self 0.206ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 9.448ms self 9.448ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.331ms self 0.331ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.098ms self 0.098ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.245ms self 0.245ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.970ms self 1.970ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.189ms self 0.189ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.444ms self 1.444ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.067ms self 0.067ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.380ms self 0.012ms children 0.367ms %children 96.76%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.093ms self 0.093ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.140ms self 0.140ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.135ms self 0.135ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 18.441ms self 0.014ms children 18.428ms %children 99.93%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.058ms self 0.058ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 18.301ms self 2.821ms children 15.481ms %children 84.59%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 15.481ms self 0.086ms children 15.395ms %children 99.45%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.322ms self 0.322ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.111ms self 0.111ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.252ms self 0.252ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 2.509ms self 2.509ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.183ms self 0.183ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.512ms self 1.512ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.310ms self 0.310ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.091ms self 0.091ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.223ms self 0.223ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 2.308ms self 2.308ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.179ms self 0.179ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.435ms self 1.435ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.361ms self 0.361ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.091ms self 0.091ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.219ms self 0.219ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 1.837ms self 1.837ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.191ms self 0.191ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 3.263ms self 3.263ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.068ms self 0.068ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 13.155ms self 0.015ms children 13.140ms %children 99.88%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.127ms self 0.127ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 12.913ms self 12.913ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.101ms self 0.101ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 41.790ms self 0.016ms children 41.774ms %children 99.96%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.060ms self 0.060ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 41.631ms self 2.813ms children 38.818ms %children 93.24%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 38.818ms self 0.094ms children 38.724ms %children 99.76%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.224ms self 0.224ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.123ms self 0.123ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.238ms self 0.238ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 2.144ms self 2.144ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.173ms self 0.173ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 10.864ms self 10.864ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 4.247ms self 4.247ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.109ms self 0.109ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.225ms self 0.225ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 2.780ms self 2.780ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.187ms self 0.187ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 1.561ms self 1.561ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.425ms self 0.425ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.161ms self 0.161ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 10.337ms self 10.337ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 2.228ms self 2.228ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.183ms self 0.183ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 2.515ms self 2.515ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.082ms self 0.082ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.034ms self 0.034ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable total 1.112ms self 0.012ms children 1.100ms %children 98.92%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation total 1.084ms self 0.521ms children 0.564ms %children 51.97%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize total 0.564ms self 0.045ms children 0.519ms %children 92.01%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.141ms self 0.141ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.027ms self 0.027ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.063ms self 0.063ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.182ms self 0.182ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.032ms self 0.032ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.073ms self 0.073ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile total 54.217ms self 48.599ms children 5.618ms %children 10.36%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR total 3.193ms self 0.013ms children 3.180ms %children 99.59%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation total 3.158ms self 1.685ms children 1.473ms %children 46.63%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 1.473ms self 0.043ms children 1.430ms %children 97.09%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.955ms self 0.955ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.025ms self 0.025ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.121ms self 0.121ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.027ms self 0.027ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.050ms self 0.050ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.060ms self 0.060ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.015ms self 0.015ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.093ms self 0.093ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.019ms self 0.019ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.046ms self 0.046ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.010ms self 0.010ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR total 0.055ms self 0.007ms children 0.049ms %children 87.94%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/LoweringTransformation total 0.023ms self 0.023ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR total 0.642ms self 0.010ms children 0.633ms %children 98.48%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.619ms self 0.361ms children 0.259ms %children 41.76%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.259ms self 0.021ms children 0.237ms %children 91.75%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.035ms self 0.035ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.017ms self 0.017ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.117ms self 0.117ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.021ms self 0.021ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.038ms self 0.038ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs total 1.098ms self 0.009ms children 1.088ms %children 99.14%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.008ms self 0.008ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/LoweringTransformation total 1.066ms self 1.066ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.629ms self 0.010ms children 0.619ms %children 98.38%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.606ms self 0.360ms children 0.246ms %children 40.64%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.246ms self 0.025ms children 0.221ms %children 89.73%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.036ms self 0.036ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.017ms self 0.017ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.097ms self 0.097ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.021ms self 0.021ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.040ms self 0.040ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/InitializeCompiledFunction total 1.724ms self 1.724ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/RunCompiledFunction total 0.008ms self 0.008ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: timing SparkBackend.executeJSON/convertRegionValueToAnnotation total 0.157ms self 0.157ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:33 root: INFO: globbing path /broad/jhlab_ukbiobank/users/aravikum/prs_sim/prsice_outputs/3_11_clust_akshaya_test_height_chrAll_5.best_noised returned 1 files: 3_11_clust_akshaya_test_height_chrAll_5.best_noised
2021-05-04 19:17:33 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:33 root: INFO: timing SparkBackend.parse_table_ir total 14.182ms self 14.182ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 Hail: INFO: Finished type imputation
Loading field 'FID' as type str (user-supplied type)
Loading field 'IID' as type str (user-supplied type)
Loading field 'In_Regression' as type str (imputed)
Loading field 'PRS' as type float64 (imputed)
Loading field 'float_PRS' as type float64 (imputed)
Loading field 'Noise' as type float64 (imputed)
Loading field 'Noise_added' as type float64 (user-supplied type)
2021-05-04 19:17:33 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:33 root: INFO: globbing path /broad/jhlab_ukbiobank/users/ebartell/z020_out_covariates.txt returned 1 files: z020_out_covariates.txt
2021-05-04 19:17:33 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:33 root: INFO: timing SparkBackend.parse_table_ir total 13.734ms self 13.734ms children 0.000ms %children 0.00%
2021-05-04 19:17:33 Hail: INFO: Reading table without type imputation
Loading field 'FID' as type str (user-supplied)
Loading field 'IID' as type str (user-supplied)
Loading field 'PC1' as type float64 (user-supplied)
Loading field 'PC2' as type float64 (user-supplied)
Loading field 'PC3' as type float64 (user-supplied)
Loading field 'PC4' as type float64 (user-supplied)
Loading field 'PC5' as type float64 (user-supplied)
Loading field 'PC6' as type float64 (user-supplied)
Loading field 'PC7' as type float64 (user-supplied)
Loading field 'PC8' as type float64 (user-supplied)
Loading field 'PC9' as type float64 (user-supplied)
Loading field 'PC10' as type float64 (user-supplied)
Loading field 'ArrayNum' as type int32 (user-supplied)
2021-05-04 19:17:33 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:34 BlockManagerInfo: INFO: Removed broadcast_3_piece0 on ugertmp-c021.broadinstitute.org:38861 in memory (size: 15.3 KiB, free: 4.1 GiB)
2021-05-04 19:17:34 BlockManagerInfo: INFO: Removed broadcast_2_piece0 on ugertmp-c021.broadinstitute.org:38861 in memory (size: 1233.0 B, free: 4.1 GiB)
2021-05-04 19:17:34 root: INFO: JSON: JObject(List((name,JString(ForceCountMatrixTable))))
2021-05-04 19:17:34 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:34 root: INFO: timing SparkBackend.parse_value_ir total 840.481ms self 840.481ms children 0.000ms %children 0.00%
2021-05-04 19:17:34 root: INFO: RegionPool: initialized for thread 15: Thread-5
2021-05-04 19:17:34 root: INFO: starting execution of query hail_query_2 of initial size 26
2021-05-04 19:17:34 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 26:
(MatrixToValueApply
"{\"name\":\"ForceCountMatrixTable\"}"
(MatrixRowsHead 1
(MatrixFilterCols
(MatrixMapRows
(MatrixRead None False False
"{\"name\":\"MatrixBGENReader\",\"files\":[\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\"],\"sampleFile\":\"/broad/jhlab_ukbiobank/PhenoData/LinkIds_ukb_500K_ImpData_v3_app11898_chrAUT.sample\",\"indexFileMap\":{\"/broad/ukbb/imputed_v3/ukb_imp_chr4_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr4_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr1_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr1_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr18_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr18_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr7_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr7_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr13_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr13_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr19_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr19_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr10_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr10_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr3_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr3_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr21_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr21_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr12_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr12_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr14_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr14_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr11_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr11_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr5_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr5_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr20_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr20_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr9_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr9_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr22_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr6_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr6_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr8_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr8_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr15_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr15_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr17_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr17_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr16_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr16_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr2_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr2_v3_hail.bgen.idx2\"},\"nPartitions\":null,\"blockSizeInMB\":128,\"includedVariants\":null}")
(SelectFields (locus alleles rsid varid)
(MakeStruct
(locus (GetField locus (Ref va)))
(alleles (GetField alleles (Ref va)))
(rsid (GetField rsid (Ref va)))
(varid (GetField varid (Ref va)))
(offset (GetField offset (Ref va)))
(file_idx (GetField file_idx (Ref va))))))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s (Ref sa))))
(False)))))
2021-05-04 19:17:34 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 21:
(MatrixToValueApply
"{\"name\":\"ForceCountMatrixTable\"}"
(MatrixRowsHead 1
(MatrixFilterCols
(MatrixMapRows
(MatrixRead
Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String},entry:Struct{GT:Call,GP:Array[Float64],dosage:Float64}}
False False
"{\"name\":\"MatrixBGENReader\",\"files\":[\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\"],\"sampleFile\":\"/broad/jhlab_ukbiobank/PhenoData/LinkIds_ukb_500K_ImpData_v3_app11898_chrAUT.sample\",\"indexFileMap\":{\"/broad/ukbb/imputed_v3/ukb_imp_chr4_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr4_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr1_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr1_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr18_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr18_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr7_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr7_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr13_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr13_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr19_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr19_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr10_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr10_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr3_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr3_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr21_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr21_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr12_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr12_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr14_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr14_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr11_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr11_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr5_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr5_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr20_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr20_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr9_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr9_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr22_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr6_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr6_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr8_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr8_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr15_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr15_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr17_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr17_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr16_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr16_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr2_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr2_v3_hail.bgen.idx2\"},\"nPartitions\":null,\"blockSizeInMB\":128,\"includedVariants\":null}")
(MakeStruct
(locus (GetField locus (Ref va)))
(alleles (GetField alleles (Ref va)))
(rsid (GetField rsid (Ref va)))
(varid (GetField varid (Ref va)))))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s (Ref sa))))
(False)))))
2021-05-04 19:17:34 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 79:
(TableToValueApply
"{\"name\":\"ForceCountTable\"}"
(TableHead 1
(TableMapGlobals
(TableMapRows
(TableMapGlobals
(TableMapRows
(TableRead
Table{global:Struct{__cols:Array[Struct{s:String}]},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,GP:Array[Float64],dosage:Float64}]}}
False
"{\"name\":\"MatrixBGENReader\",\"files\":[\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\"],\"sampleFile\":\"/broad/jhlab_ukbiobank/PhenoData/LinkIds_ukb_500K_ImpData_v3_app11898_chrAUT.sample\",\"indexFileMap\":{\"/broad/ukbb/imputed_v3/ukb_imp_chr4_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr4_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr1_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr1_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr18_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr18_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr7_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr7_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr13_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr13_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr19_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr19_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr10_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr10_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr3_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr3_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr21_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr21_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr12_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr12_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr14_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr14_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr11_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr11_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr5_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr5_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr20_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr20_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr9_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr9_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr22_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr6_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr6_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr8_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr8_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr15_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr15_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr17_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr17_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr16_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr16_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr2_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr2_v3_hail.bgen.idx2\"},\"nPartitions\":null,\"blockSizeInMB\":128,\"includedVariants\":null}")
(Let n_cols
(ArrayLen (GetField __cols (Ref global)))
(InsertFields
(Let __iruid_85
(MakeStruct)
(MakeStruct
(locus
(GetField locus
(SelectFields (locus alleles rsid varid)
(Ref row))))
(alleles
(GetField alleles
(SelectFields (locus alleles rsid varid)
(Ref row))))
(rsid
(GetField rsid
(SelectFields (locus alleles rsid varid)
(Ref row))))
(varid
(GetField varid
(SelectFields (locus alleles rsid varid)
(Ref row))))))
None
(`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(Ref row))))))
(InsertFields
(Ref global)
None
(newColIdx
(ToArray
(StreamFilter i
(ToStream False
(ToArray
(StreamRange False
(I32 0)
(ArrayLen (GetField __cols (Ref global)))
(I32 1))))
(Let sa
(ArrayRef
(GetField __cols (Ref global))
(Ref i)
(Str ""))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s (Ref sa))))
(False))))))))
(InsertFields
(Ref row)
None
(`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(ToArray
(StreamMap i
(ToStream False (GetField newColIdx (Ref global)))
(ArrayRef
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(Ref row))
(Ref i)
(Str "")))))))
(SelectFields (__cols)
(InsertFields
(Ref global)
None
(__cols
(ToArray
(StreamMap i
(ToStream False (GetField newColIdx (Ref global)))
(ArrayRef
(GetField __cols (Ref global))
(Ref i)
(Str ""))))))))))
2021-05-04 19:17:34 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 64:
(TableToValueApply
"{\"name\":\"ForceCountTable\"}"
(TableMapGlobals
(TableMapRows
(TableMapGlobals
(TableMapRows
(TableHead 1
(TableRead
Table{global:Struct{__cols:Array[Struct{s:String}]},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,GP:Array[Float64],dosage:Float64}]}}
False
"{\"name\":\"MatrixBGENReader\",\"files\":[\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\"],\"sampleFile\":\"/broad/jhlab_ukbiobank/PhenoData/LinkIds_ukb_500K_ImpData_v3_app11898_chrAUT.sample\",\"indexFileMap\":{\"/broad/ukbb/imputed_v3/ukb_imp_chr4_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr4_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr1_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr1_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr18_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr18_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr7_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr7_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr13_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr13_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr19_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr19_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr10_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr10_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr3_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr3_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr21_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr21_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr12_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr12_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr14_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr14_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr11_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr11_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr5_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr5_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr20_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr20_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr9_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr9_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr22_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr22_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr6_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr6_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr8_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr8_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr15_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr15_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr17_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr17_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr16_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr16_v3_hail.bgen.idx2\",\"/broad/ukbb/imputed_v3/ukb_imp_chr2_v3.bgen\":\"/broad/jhlab_ukbiobank/users/ebartell/qfam/subsetBgens/ukb_imp_chr2_v3_hail.bgen.idx2\"},\"nPartitions\":null,\"blockSizeInMB\":128,\"includedVariants\":null}"))
(MakeStruct
(locus (GetField locus (Ref row)))
(alleles (GetField alleles (Ref row)))
(rsid (GetField rsid (Ref row)))
(varid (GetField varid (Ref row)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(Ref row)))))
(InsertFields
(Ref global)
None
(newColIdx
(ToArray
(StreamFilter __iruid_119
(StreamRange False
(I32 0)
(ArrayLen (GetField __cols (Ref global)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s
(ArrayRef
(GetField __cols (Ref global))
(Ref __iruid_119)
(Str "")))))
(False)))))))
(InsertFields
(Ref row)
None
(`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(ToArray
(StreamMap __iruid_120
(ToStream False (GetField newColIdx (Ref global)))
(ArrayRef
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(Ref row))
(Ref __iruid_120)
(Str "")))))))
(InsertFields
(SelectFields (__cols) (Ref global))
("__cols")
(__cols
(ToArray
(StreamMap __iruid_121
(ToStream False (GetField newColIdx (Ref global)))
(ArrayRef
(GetField __cols (Ref global))
(Ref __iruid_121)
(Str ""))))))))
2021-05-04 19:17:34 root: INFO: interpreting non compilable node: TableToValueApply
2021-05-04 19:17:34 root: INFO: instruction count: 3: __C199FSContainer.<init>
2021-05-04 19:17:34 root: INFO: instruction count: 3: __C199FSContainer.<clinit>
2021-05-04 19:17:34 root: INFO: instruction count: 3: __C200bgen_rdd_decoder.<init>
2021-05-04 19:17:34 root: INFO: instruction count: 611: __C200bgen_rdd_decoder.apply
2021-05-04 19:17:34 root: INFO: instruction count: 16: __C200bgen_rdd_decoder.apply
2021-05-04 19:17:34 root: INFO: instruction count: 291: __C200bgen_rdd_decoder.__m214memoizeEntries
2021-05-04 19:17:34 root: INFO: instruction count: 9: __C200bgen_rdd_decoder.setPartitionIndex
2021-05-04 19:17:34 root: INFO: instruction count: 4: __C200bgen_rdd_decoder.addPartitionRegion
2021-05-04 19:17:34 root: INFO: instruction count: 4: __C200bgen_rdd_decoder.setPool
2021-05-04 19:17:34 root: INFO: instruction count: 3: __C200bgen_rdd_decoder.addFS
2021-05-04 19:17:34 root: INFO: instruction count: 568: __C200bgen_rdd_decoder.applyregion25_58
2021-05-04 19:17:34 root: INFO: instruction count: 3: __C218applySpills.<init>
2021-05-04 19:17:34 root: INFO: decoder cache hit
2021-05-04 19:17:34 root: INFO: decoder cache hit
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 3.8M allocated (128.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 3.9M allocated (192.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0M allocated (256.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0M allocated (320.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 4.1M allocated (384.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (4.3M blocks / 3.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:35 SparkContext: INFO: Starting job: runJob at ContextRDD.scala:362
2021-05-04 19:17:35 DAGScheduler: INFO: Got job 1 (runJob at ContextRDD.scala:362) with 1 output partitions
2021-05-04 19:17:35 DAGScheduler: INFO: Final stage: ResultStage 1 (runJob at ContextRDD.scala:362)
2021-05-04 19:17:35 DAGScheduler: INFO: Parents of final stage: List()
2021-05-04 19:17:35 DAGScheduler: INFO: Missing parents: List()
2021-05-04 19:17:35 DAGScheduler: INFO: Submitting ResultStage 1 (MapPartitionsRDD[6] at mapPartitions at ContextRDD.scala:250), which has no missing parents
2021-05-04 19:17:35 MemoryStore: INFO: Block broadcast_5 stored as values in memory (estimated size 65.8 KiB, free 4.1 GiB)
2021-05-04 19:17:35 MemoryStore: INFO: Block broadcast_5_piece0 stored as bytes in memory (estimated size 32.0 KiB, free 4.1 GiB)
2021-05-04 19:17:35 BlockManagerInfo: INFO: Added broadcast_5_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 32.0 KiB, free: 4.1 GiB)
2021-05-04 19:17:35 SparkContext: INFO: Created broadcast 5 from broadcast at DAGScheduler.scala:1383
2021-05-04 19:17:35 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[6] at mapPartitions at ContextRDD.scala:250) (first 15 tasks are for partitions Vector(0))
2021-05-04 19:17:35 TaskSchedulerImpl: INFO: Adding task set 1.0 with 1 tasks resource profile 0
2021-05-04 19:17:35 TaskSetManager: INFO: Starting task 0.0 in stage 1.0 (TID 1) (ugertmp-c021.broadinstitute.org, executor driver, partition 0, PROCESS_LOCAL, 5077 bytes) taskResourceAssignments Map()
2021-05-04 19:17:35 Executor: INFO: Running task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: initialized for thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 384.0K allocated (192.0K blocks / 192.0K chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (320.0K blocks / 192.0K chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 1.9M allocated (384.0K blocks / 1.5M chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (512.0K blocks / 1.5M chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 14.8M allocated (2.1M blocks / 12.7M chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: RegionPool: REPORT_THRESHOLD: 14.8M allocated (2.1M blocks / 12.7M chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 root: INFO: Index reader cache queries: 2
2021-05-04 19:17:35 root: INFO: Index reader cache hit rate: 0.0
2021-05-04 19:17:35 root: INFO: TaskReport: stage=1, partition=0, attempt=0, peakBytes=15559848, peakBytesReadable=14.84 MiB
2021-05-04 19:17:35 root: INFO: RegionPool: FREE: 2.1M allocated (2.1M blocks / 0 chunks), regions.size = 3, 0 current java objects, 0 max java objects, thread 53: Executor task launch worker for task 0.0 in stage 1.0 (TID 1)
2021-05-04 19:17:35 Executor: INFO: Finished task 0.0 in stage 1.0 (TID 1). 795 bytes result sent to driver
2021-05-04 19:17:35 TaskSetManager: INFO: Finished task 0.0 in stage 1.0 (TID 1) in 329 ms on ugertmp-c021.broadinstitute.org (executor driver) (1/1)
2021-05-04 19:17:35 TaskSchedulerImpl: INFO: Removed TaskSet 1.0, whose tasks have all completed, from pool
2021-05-04 19:17:35 DAGScheduler: INFO: ResultStage 1 (runJob at ContextRDD.scala:362) finished in 0.343 s
2021-05-04 19:17:35 DAGScheduler: INFO: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job
2021-05-04 19:17:35 TaskSchedulerImpl: INFO: Killing all running tasks in stage 1: Stage finished
2021-05-04 19:17:35 DAGScheduler: INFO: Job 1 finished: runJob at ContextRDD.scala:362, took 0.349507 s
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 14:
(Coalesce
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
(Die
Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,GP:Array[Float64],dosage:Float64}]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 11:
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 11:
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 11:
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 11:
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 11:
(MakeStruct
(locus
(GetField locus
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(alleles
(GetField alleles
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(rsid
(GetField rsid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(varid
(GetField varid
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1)))
(the entries! [877f12a8827e18f61222c6c8c5fb04a8]
(GetField
`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GT:PCCall,GP:+PCArray[+PFloat64],dosage:+PFloat64}]}))
1))))
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C233FSContainer.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C233FSContainer.<clinit>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C234Compiled.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 155: __C234Compiled.apply
2021-05-04 19:17:36 root: INFO: instruction count: 9: __C234Compiled.setPartitionIndex
2021-05-04 19:17:36 root: INFO: instruction count: 4: __C234Compiled.addPartitionRegion
2021-05-04 19:17:36 root: INFO: instruction count: 4: __C234Compiled.setPool
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C234Compiled.addFS
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 24:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_119
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_119)
(Str "")))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 24:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_155
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(ApplyIR contains () Boolean
(Literal Set[String] <literal value>)
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_155)
(Str "")))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 47:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_155
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(Let __iruid_159
(Literal Set[String] <literal value>)
(Let __iruid_160
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_155)
(Str "")))
(If
(IsNA (Ref __iruid_159))
(NA Boolean)
(Let __iruid_161
(LowerBoundOnOrderedCollection False
(Ref __iruid_159)
(Ref __iruid_160))
(If
(ApplyComparisonOp EQWithNA
(Ref __iruid_161)
(ArrayLen (CastToArray (Ref __iruid_159))))
(False)
(ApplyComparisonOp EQWithNA
(ArrayRef
(CastToArray (Ref __iruid_159))
(Ref __iruid_161)
(Str ""))
(Ref __iruid_160))))))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 38:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_169
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(Let __iruid_170
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_169)
(Str "")))
(Let __iruid_171
(LowerBoundOnOrderedCollection False
(Literal Set[String] <literal value>)
(Ref __iruid_170))
(If
(ApplyComparisonOp EQWithNA
(Ref __iruid_171)
(I32 33233))
(False)
(ApplyComparisonOp EQWithNA
(ArrayRef
(Literal Array[String] <literal value>)
(Ref __iruid_171)
(Str ""))
(Ref __iruid_170))))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 38:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_169
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(Let __iruid_170
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_169)
(Str "")))
(Let __iruid_171
(LowerBoundOnOrderedCollection False
(Literal Set[String] <literal value>)
(Ref __iruid_170))
(If
(ApplyComparisonOp EQWithNA
(Ref __iruid_171)
(I32 33233))
(False)
(ApplyComparisonOp EQWithNA
(ArrayRef
(Literal Array[String] <literal value>)
(Ref __iruid_171)
(Str ""))
(Ref __iruid_170))))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 38:
(Coalesce
(InsertFields
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)
None
(newColIdx
(ToArray
(StreamFilter __iruid_178
(StreamRange False
(I32 0)
(ArrayLen
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0)))
(I32 1))
(Coalesce
(ApplyUnaryPrimOp Bang
(Let __iruid_179
(GetField s
(ArrayRef
(GetField __cols
(In
SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{__cols:+PCArray[+PCStruct{s:+PCString}]}))
0))
(Ref __iruid_178)
(Str "")))
(Let __iruid_180
(LowerBoundOnOrderedCollection False
(Literal Set[String] <literal value>)
(Ref __iruid_179))
(If
(ApplyComparisonOp EQWithNA
(Ref __iruid_180)
(I32 33233))
(False)
(ApplyComparisonOp EQWithNA
(ArrayRef
(Literal Array[String] <literal value>)
(Ref __iruid_180)
(Str ""))
(Ref __iruid_179))))))
(False))))))
(Die
Struct{__cols:Array[Struct{s:String}],newColIdx:Array[Int32]}
-1
(Str "Internal e...")))
2021-05-04 19:17:36 root: INFO: encoder cache miss (0 hits, 3 misses, 0.000)
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C281FSContainer.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C281FSContainer.<clinit>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C282etypeEncode.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 5: __C282etypeEncode.apply
2021-05-04 19:17:36 root: INFO: instruction count: 33: __C282etypeEncode.__m283ENCODE_r_tuple_of_r_stringANDr_set_of_r_stringANDr_array_of_r_stringEND_TO_r_struct_of_r_binaryANDr_array_of_r_binaryANDr_array_of_r_binaryEND
2021-05-04 19:17:36 root: INFO: instruction count: 18: __C282etypeEncode.__m284ENCODE_r_string_TO_r_binary
2021-05-04 19:17:36 root: INFO: instruction count: 49: __C282etypeEncode.__m285ENCODE_r_set_of_r_string_TO_r_array_of_r_binary
2021-05-04 19:17:36 root: INFO: instruction count: 49: __C282etypeEncode.__m286ENCODE_r_array_of_r_string_TO_r_array_of_r_binary
2021-05-04 19:17:36 MemoryStore: INFO: Block broadcast_6 stored as values in memory (estimated size 714.1 KiB, free 4.1 GiB)
2021-05-04 19:17:36 MemoryStore: INFO: Block broadcast_6_piece0 stored as bytes in memory (estimated size 371.3 KiB, free 4.1 GiB)
2021-05-04 19:17:36 BlockManagerInfo: INFO: Added broadcast_6_piece0 in memory on ugertmp-c021.broadinstitute.org:38861 (size: 371.3 KiB, free: 4.1 GiB)
2021-05-04 19:17:36 SparkContext: INFO: Created broadcast 6 from broadcast at SparkBackend.scala:275
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C238FSContainer.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C238FSContainer.<clinit>
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C239Compiled.<init>
2021-05-04 19:17:36 root: INFO: instruction count: 545: __C239Compiled.apply
2021-05-04 19:17:36 root: INFO: instruction count: 42: __C239Compiled.__m250arrayref_bounds_check
2021-05-04 19:17:36 root: INFO: instruction count: 74: __C239Compiled.__m256findElt
2021-05-04 19:17:36 root: INFO: instruction count: 34: __C239Compiled.__m257ord_compare
2021-05-04 19:17:36 root: INFO: instruction count: 5: __C239Compiled.__m258ord_compareNonnull
2021-05-04 19:17:36 root: INFO: instruction count: 92: __C239Compiled.__m259ord_compareNonnull
2021-05-04 19:17:36 root: INFO: instruction count: 34: __C239Compiled.__m262ord_equiv
2021-05-04 19:17:36 root: INFO: instruction count: 12: __C239Compiled.__m263ord_equivNonnull
2021-05-04 19:17:36 root: INFO: instruction count: 42: __C239Compiled.__m267arrayref_bounds_check
2021-05-04 19:17:36 root: INFO: instruction count: 34: __C239Compiled.__m268ord_equiv
2021-05-04 19:17:36 root: INFO: instruction count: 15: __C239Compiled.__m269ord_equivNonnull
2021-05-04 19:17:36 root: INFO: instruction count: 12: __C239Compiled.__m272setup_iab
2021-05-04 19:17:36 root: INFO: instruction count: 9: __C239Compiled.setPartitionIndex
2021-05-04 19:17:36 root: INFO: instruction count: 4: __C239Compiled.addPartitionRegion
2021-05-04 19:17:36 root: INFO: instruction count: 4: __C239Compiled.setPool
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C239Compiled.addFS
2021-05-04 19:17:36 root: INFO: instruction count: 82: __C239Compiled.addLiterals
2021-05-04 19:17:36 root: INFO: instruction count: 37: __C239Compiled.__m277DECODE_r_struct_of_r_binaryANDr_array_of_r_binaryANDr_array_of_r_binaryEND_TO_r_tuple_of_r_stringANDr_set_of_r_stringANDr_array_of_r_stringEND
2021-05-04 19:17:36 root: INFO: instruction count: 33: __C239Compiled.__m278INPLACE_DECODE_r_binary_TO_r_string
2021-05-04 19:17:36 root: INFO: instruction count: 130: __C239Compiled.__m279INPLACE_DECODE_r_array_of_r_binary_TO_r_set_of_r_string
2021-05-04 19:17:36 root: INFO: instruction count: 64: __C239Compiled.__m280INPLACE_DECODE_r_array_of_r_binary_TO_r_array_of_r_string
2021-05-04 19:17:36 root: INFO: instruction count: 3: __C287applySpills.<init>
2021-05-04 19:17:36 root: INFO: RegionPool: REPORT_THRESHOLD: 16.2M allocated (6.4M blocks / 9.8M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:36 root: INFO: RegionPool: REPORT_THRESHOLD: 32.2M allocated (6.4M blocks / 25.8M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:36 root: INFO: RegionPool: REPORT_THRESHOLD: 64.2M allocated (6.4M blocks / 57.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:36 root: INFO: RegionPool: REPORT_THRESHOLD: 128.1M allocated (6.4M blocks / 121.6M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:37 root: INFO: RegionPool: REPORT_THRESHOLD: 256.1M allocated (6.4M blocks / 249.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:37 root: INFO: RegionPool: REPORT_THRESHOLD: 512.2M allocated (6.4M blocks / 505.8M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:37 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0G allocated (6.4M blocks / 1017.7M chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:38 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0G allocated (6.4M blocks / 2.0G chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:40 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0G allocated (6.4M blocks / 4.0G chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:43 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0G allocated (6.4M blocks / 8.0G chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
2021-05-04 19:17:50 root: INFO: RegionPool: REPORT_THRESHOLD: 16.0G allocated (6.4M blocks / 16.0G chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 15: Thread-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment