Skip to content

Instantly share code, notes, and snippets.

@tamird

tamird/testrace.log

Created September 8, 2016 20:35
Show Gist options
  • Save tamird/b36a7cd82e749ca2dd70e4dde69b899d to your computer and use it in GitHub Desktop.
Save tamird/b36a7cd82e749ca2dd70e4dde69b899d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
testrace.log
=== RUN TestErrorOnCommitResultsInRollback
I160901 19:37:59.725452 18579 storage/engine/rocksdb.go:354 opening in memory rocksdb instance
I160901 19:37:59.741085 18579 gossip/gossip.go:230 [n?] initial resolvers: []
W160901 19:37:59.741227 18579 gossip/gossip.go:1041 [n?] no resolvers found; use --join to specify a connected node
W160901 19:37:59.744218 18579 server/status/runtime.go:109 Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I160901 19:37:59.748327 18579 server/node.go:413 [n?] store store=0:0 not bootstrapped
I160901 19:37:59.755629 18558 storage/replica.go:2139 [s1,r1{/Min-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:37:59.756229 18558 storage/replica.go:2174 [s1,r1{/Min-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
I160901 19:37:59.756702 18558 storage/replica_trigger.go:308 new range lease replica {1 1 1} 1970-01-01 00:00:00 +0000 UTC 409099h38m8.754454868s following replica {0 0 0} 1970-01-01 00:00:00 +0000 UTC 0s [physicalTime=2016-09-01 19:37:59.756601992 +0000 UTC]
I160901 19:37:59.759786 18579 server/node.go:345 [n?] **** cluster {b42171bb-fb35-451a-9907-a0d1e9a3ccbf} has been created
I160901 19:37:59.759859 18579 server/node.go:346 [n?] **** add additional nodes by specifying --join=127.0.0.1:54663
I160901 19:37:59.763677 18579 gossip/gossip.go:264 [n?] NodeID set to 1
I160901 19:37:59.783099 18579 server/node.go:426 [n?] initialized store store=1:1: {Capacity:536870912 Available:536870912 RangeCount:0}
I160901 19:37:59.783284 18579 server/node.go:318 [n?] node ID 1 initialized
I160901 19:37:59.783550 18579 gossip/gossip.go:278 [n?] NodeDescriptor set to node_id:1 address:<network_field:"tcp" address_field:"127.0.0.1:54663" > attrs:<>
I160901 19:37:59.784026 18579 storage/stores.go:313 read 0 node addresses from persistent storage
I160901 19:37:59.784384 18579 server/node.go:554 [n?] connecting to gossip network to verify cluster ID...
I160901 19:37:59.785017 18612 storage/replica.go:2747 [n?,s1,r1{/Min-/Max}] gossiping cluster id {"b42171bb-fb35-451a-9907-a0d1e9a3ccbf"} from store 1, range 1
I160901 19:37:59.785543 18612 storage/replica.go:2769 [n?,s1,r1{/Min-/Max}] gossiping sentinel from store 1, range 1
I160901 19:37:59.786018 18612 storage/replica.go:2776 [n?,s1,r1{/Min-/Max}] gossiping first range from store 1, range 1: [{1 1 1}]
I160901 19:37:59.786616 18619 kv/dist_sender.go:223 [n?] gossipped first range descriptor: [{NodeID:1 StoreID:1 ReplicaID:1}]
I160901 19:37:59.789097 18579 server/node.go:575 [n?] node connected via gossip and verified as part of cluster {"b42171bb-fb35-451a-9907-a0d1e9a3ccbf"}
I160901 19:37:59.789711 18579 server/node.go:364 [n?] node=1: started with [[]=] engine(s) and attributes []
I160901 19:37:59.790564 18579 server/server.go:488 [n1] starting https server at 127.0.0.1:38807
I160901 19:37:59.790743 18579 server/server.go:489 [n1] starting grpc/postgres server at 127.0.0.1:54663
I160901 19:37:59.791028 18579 rpc/context.go:199 dialing 127.0.0.1:54663
I160901 19:37:59.792688 18621 storage/split_queue.go:99 splitting store=1:1 1{/Min-/Max} at keys [/Table/11/0 /Table/12/0 /Table/13/0 /Table/14/0]
I160901 19:37:59.796531 18631 sql/event_log.go:109 [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:{NetworkField:tcp AddressField:127.0.0.1:54663} Attrs:} ClusterID:{UUID:b42171bb-fb35-451a-9907-a0d1e9a3ccbf} StartedAt:1472758679789155200}
I160901 19:37:59.832108 18610 storage/replica.go:2139 [n1,s1,r1{/Min-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:37:59.833197 18610 storage/replica.go:2174 [n1,s1,r1{/Min-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
I160901 19:37:59.866232 18610 storage/replica.go:1890 [n1,s1,r1{/Min-/Max}] pending commands: refurbished 0, reproposing 1 (at 16.3); reasonNewLeader
I160901 19:37:59.876303 18559 storage/replica.go:2139 [n1,s1,r1{/Min-/Max}] applying command with forced error: storage/replica.go:2095: command observed at lease index 4, but required < 4
E160901 19:37:59.877511 18559 storage/replica.go:2174 [n1,s1,r1{/Min-/Max}] error executing raft command: storage/replica.go:2095: command observed at lease index 4, but required < 4
I160901 19:37:59.878069 18621 storage/replica_command.go:2213 [n1,s1,r1{/Min-/Max}] initiating a split of this range at key /Table/11
I160901 19:37:59.882664 18621 storage/log.go:85 Range Event: "split", range: 1, info: {"UpdatedDesc":{"range_id":1,"end_key":"kw==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":2,"start_key":"kw==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:37:59.889263 18631 kv/dist_sender.go:1051 [n1] application error: retry txn "server/node.go:739 (*Node).recordJoinEvent.func1" id=7b9e07e0 key=/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/0 rw=false pri=0.02096025 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758679.814285135,1 orig=1472758679.797371467,0 max=1472758679.797371467,0 wto=false rop=false
I160901 19:37:59.900349 18631 kv/dist_sender.go:786 [n1] reply error BeginTransaction [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/0,/Min), ConditionalPut [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/0,/Min), ConditionalPut [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/2/1,/Min), ConditionalPut [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/3/1,/Min), ConditionalPut [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/4/1,/Min), ConditionalPut [/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/5/1,/Min): retry txn "server/node.go:739 (*Node).recordJoinEvent.func1" id=7b9e07e0 key=/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/0 rw=false pri=0.02096025 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758679.814285135,1 orig=1472758679.797371467,0 max=1472758679.797371467,0 wto=false rop=false
I160901 19:37:59.900845 18631 internal/client/db.go:493 failed batch: retry txn "server/node.go:739 (*Node).recordJoinEvent.func1" id=7b9e07e0 key=/Table/12/1/2016-09-01T19:37:59.796592Z/"\x14pJ7bH~\x89\x00"/0 rw=false pri=0.05709892 iso=SERIALIZABLE stat=PENDING epo=1 ts=1472758679.814285135,1 orig=1472758679.814285135,1 max=1472758679.847371467,0 wto=false rop=false
I160901 19:37:59.901052 18631 sql/event_log.go:109 [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:{NetworkField:tcp AddressField:127.0.0.1:54663} Attrs:} ClusterID:{UUID:b42171bb-fb35-451a-9907-a0d1e9a3ccbf} StartedAt:1472758679789155200}
I160901 19:37:59.926756 18561 storage/replica_command.go:1380 [n1,s1,r1{/Min-/Max}] <nil> pushed 6eeb550f: priority
W160901 19:37:59.954672 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/11 /Meta2/Max]: resolved? false
I160901 19:37:59.959703 18621 kv/dist_sender.go:1051 application error: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.01859199 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758679.923895574,1 orig=1472758679.878479660,0 max=1472758679.878479660,0 wto=false rop=false
I160901 19:37:59.959968 18621 kv/dist_sender.go:786 reply error EndTransaction [/Local/Range/""/RangeDescriptor,/Min): retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.01859199 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758679.923895574,1 orig=1472758679.878479660,0 max=1472758679.878479660,0 wto=false rop=false
I160901 19:37:59.960533 18621 internal/client/db.go:493 failed batch: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.05067866 iso=SERIALIZABLE stat=PENDING epo=1 ts=1472758679.923895574,1 orig=1472758679.923895574,1 max=1472758679.928479660,0 wto=false rop=false
I160901 19:37:59.966923 18621 storage/log.go:85 Range Event: "split", range: 1, info: {"UpdatedDesc":{"range_id":1,"end_key":"kw==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":2,"start_key":"kw==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:37:59.976174 18610 storage/replica_command.go:1380 [n1,s1,r1{/Min-/Max}] <nil> pushed 6eeb550f: priority
W160901 19:37:59.985747 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/11 /Meta2/Max]: resolved? false
I160901 19:37:59.988788 18621 kv/dist_sender.go:1051 application error: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.05067866 iso=SERIALIZABLE stat=PENDING epo=1 ts=1472758679.971336499,1 orig=1472758679.878479660,0 max=1472758679.878479660,0 wto=false rop=false
I160901 19:37:59.988988 18621 kv/dist_sender.go:786 reply error EndTransaction [/Local/Range/""/RangeDescriptor,/Min): retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.05067866 iso=SERIALIZABLE stat=PENDING epo=1 ts=1472758679.971336499,1 orig=1472758679.878479660,0 max=1472758679.878479660,0 wto=false rop=false
I160901 19:37:59.989459 18621 internal/client/db.go:493 failed batch: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=6eeb550f key=/Local/Range/""/RangeDescriptor rw=true pri=0.05067866 iso=SERIALIZABLE stat=PENDING epo=2 ts=1472758679.971336499,1 orig=1472758679.971336499,1 max=1472758679.928479660,0 wto=false rop=false
I160901 19:37:59.993714 18621 storage/log.go:85 Range Event: "split", range: 1, info: {"UpdatedDesc":{"range_id":1,"end_key":"kw==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":2,"start_key":"kw==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
W160901 19:38:00.008001 18621 kv/dist_sender.go:697 1 retries for an RPC at sequence 20, last error was: <nil>, remaining key ranges {/Min "\x00"}: EndTransaction [/Local/Range/""/RangeDescriptor,/Min)
I160901 19:38:00.018445 18559 storage/queue.go:298 [replicate] store=1:1 2{/Table/11-/Max}: split needed; not adding
I160901 19:38:00.021003 18621 kv/dist_sender.go:1051 application error: key range /Table/12/0-/Min outside of bounds of range /Min-/Table/11
I160901 19:38:00.021213 18621 kv/dist_sender.go:786 reply error AdminSplit [/Table/12/0,/Min): key range /Table/12/0-/Min outside of bounds of range /Min-/Table/11
W160901 19:38:00.021556 18621 kv/dist_sender.go:831 key range /Table/12/0-/Min outside of bounds of range /Min-/Table/11
I160901 19:38:00.022532 18621 storage/replica_command.go:2213 [n1,s1,r2{/Table/11-/Max}] initiating a split of this range at key /Table/12
I160901 19:38:00.028403 18561 storage/replica.go:2139 [n1,s1,r2{/Table/11-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.029226 18561 storage/replica.go:2174 [n1,s1,r2{/Table/11-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
I160901 19:38:00.033594 18621 storage/log.go:85 Range Event: "split", range: 2, info: {"UpdatedDesc":{"range_id":2,"start_key":"kw==","end_key":"lA==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":3,"start_key":"lA==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.051879 18561 storage/queue.go:298 [replicate] store=1:1 3{/Table/12-/Max}: split needed; not adding
I160901 19:38:00.054122 18559 storage/replica.go:2139 [n1,s1,r3{/Table/12-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.055250 18559 storage/replica.go:2174 [n1,s1,r3{/Table/12-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
W160901 19:38:00.055864 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/11 /Meta2/Max]: resolved? false
I160901 19:38:00.059514 18621 kv/dist_sender.go:1051 application error: key range /Table/13/0-/Min outside of bounds of range /Table/11-/Table/12
I160901 19:38:00.059731 18621 kv/dist_sender.go:786 reply error AdminSplit [/Table/13/0,/Min): key range /Table/13/0-/Min outside of bounds of range /Table/11-/Table/12
W160901 19:38:00.060222 18621 kv/dist_sender.go:831 key range /Table/13/0-/Min outside of bounds of range /Table/11-/Table/12
I160901 19:38:00.061386 18621 storage/replica_command.go:2213 [n1,s1,r3{/Table/12-/Max}] initiating a split of this range at key /Table/13
W160901 19:38:00.089545 18621 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Max]: resolved? false
W160901 19:38:00.096536 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/12 /Meta2/Max]: resolved? false
I160901 19:38:00.102821 18642 rpc/clock_offset.go:147 0 of 0 nodes are within the maximum offset of 50ms
I160901 19:38:00.106291 18621 storage/log.go:85 Range Event: "split", range: 3, info: {"UpdatedDesc":{"range_id":3,"start_key":"lA==","end_key":"lQ==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":4,"start_key":"lQ==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.116756 18610 storage/replica_command.go:1380 [n1,s1,r3{/Table/12-/Max}] <nil> pushed 77fd43b7: priority
W160901 19:38:00.122748 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/13 /Meta2/Max]: resolved? false
I160901 19:38:00.129909 18621 kv/dist_sender.go:1051 application error: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=77fd43b7 key=/Local/Range/"\x94"/RangeDescriptor rw=true pri=0.04338264 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758680.067307768,1 orig=1472758680.062659947,0 max=1472758680.062659947,0 wto=false rop=false
I160901 19:38:00.130112 18621 kv/dist_sender.go:786 reply error EndTransaction [/Local/Range/"\x94"/RangeDescriptor,/Min): retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=77fd43b7 key=/Local/Range/"\x94"/RangeDescriptor rw=true pri=0.04338264 iso=SERIALIZABLE stat=PENDING epo=0 ts=1472758680.067307768,1 orig=1472758680.062659947,0 max=1472758680.062659947,0 wto=false rop=false
I160901 19:38:00.130772 18621 internal/client/db.go:493 failed batch: retry txn "storage/replica_command.go:2259 (*Replica).AdminSplit" id=77fd43b7 key=/Local/Range/"\x94"/RangeDescriptor rw=true pri=0.04338264 iso=SERIALIZABLE stat=PENDING epo=1 ts=1472758680.067307768,1 orig=1472758680.067307768,1 max=1472758680.112659947,0 wto=false rop=false
I160901 19:38:00.137339 18621 storage/log.go:85 Range Event: "split", range: 3, info: {"UpdatedDesc":{"range_id":3,"start_key":"lA==","end_key":"lQ==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":4,"start_key":"lQ==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.161541 18559 storage/queue.go:298 [replicate] store=1:1 4{/Table/13-/Max}: split needed; not adding
I160901 19:38:00.164659 18610 storage/replica.go:2139 [n1,s1,r4{/Table/13-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.165494 18610 storage/replica.go:2174 [n1,s1,r4{/Table/13-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
I160901 19:38:00.166873 18621 kv/dist_sender.go:1051 application error: key range /Table/14/0-/Min outside of bounds of range /Table/12-/Table/13
I160901 19:38:00.167056 18621 kv/dist_sender.go:786 reply error AdminSplit [/Table/14/0,/Min): key range /Table/14/0-/Min outside of bounds of range /Table/12-/Table/13
W160901 19:38:00.167453 18621 kv/dist_sender.go:831 key range /Table/14/0-/Min outside of bounds of range /Table/12-/Table/13
I160901 19:38:00.168473 18621 storage/replica_command.go:2213 [n1,s1,r4{/Table/13-/Max}] initiating a split of this range at key /Table/14
W160901 19:38:00.182996 18579 storage/store.go:2122 [n1,s1] conflicting intents on [/Meta2/Table/13 /Meta2/Max]: resolved? false
I160901 19:38:00.184662 18621 storage/log.go:85 Range Event: "split", range: 4, info: {"UpdatedDesc":{"range_id":4,"start_key":"lQ==","end_key":"lg==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":5,"start_key":"lg==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.203347 18561 storage/replica.go:2139 [n1,s1,r5{/Table/14-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.204159 18561 storage/replica.go:2174 [n1,s1,r5{/Table/14-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
W160901 19:38:00.324797 18654 sql/pgwire/v3.go:172 unrecognized configuration parameter "client_encoding"
W160901 19:38:00.324919 18654 sql/pgwire/v3.go:172 unrecognized configuration parameter "datestyle"
W160901 19:38:00.324997 18654 sql/pgwire/v3.go:172 unrecognized configuration parameter "extra_float_digits"
I160901 19:38:00.338084 18654 sql/event_log.go:109 [n1] Event: "create_database", target: 50, info: {DatabaseName:t Statement:CREATE DATABASE t User:root}
I160901 19:38:00.374402 18621 storage/split_queue.go:99 splitting store=1:1 5{/Table/14-/Max} at keys [/Table/50/0]
I160901 19:38:00.376363 18621 kv/dist_sender.go:1051 application error: key range /Table/50/0-/Min outside of bounds of range /Table/13-/Table/14
I160901 19:38:00.376542 18621 kv/dist_sender.go:786 reply error AdminSplit [/Table/50/0,/Min): key range /Table/50/0-/Min outside of bounds of range /Table/13-/Table/14
W160901 19:38:00.377067 18621 kv/dist_sender.go:831 key range /Table/50/0-/Min outside of bounds of range /Table/13-/Table/14
I160901 19:38:00.378332 18621 storage/replica_command.go:2213 [n1,s1,r5{/Table/14-/Max}] initiating a split of this range at key /Table/50
I160901 19:38:00.394633 18621 storage/log.go:85 Range Event: "split", range: 5, info: {"UpdatedDesc":{"range_id":5,"start_key":"lg==","end_key":"ug==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":6,"start_key":"ug==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.430281 18654 sql/event_log.go:109 [n1] Event: "create_table", target: 51, info: {TableName:t.test Statement:CREATE TABLE t.test (k INT PRIMARY KEY, v TEXT) User:root}
I160901 19:38:00.485677 18559 storage/replica.go:2139 [n1,s1,r6{/Table/50-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.486742 18559 storage/replica.go:2174 [n1,s1,r6{/Table/50-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
I160901 19:38:00.513370 18621 storage/split_queue.go:99 splitting store=1:1 6{/Table/50-/Max} at keys [/Table/51/0]
I160901 19:38:00.515285 18621 kv/dist_sender.go:1051 application error: key range /Table/51/0-/Min outside of bounds of range /Table/14-/Table/50
I160901 19:38:00.515527 18621 kv/dist_sender.go:786 reply error AdminSplit [/Table/51/0,/Min): key range /Table/51/0-/Min outside of bounds of range /Table/14-/Table/50
W160901 19:38:00.516042 18621 kv/dist_sender.go:831 key range /Table/51/0-/Min outside of bounds of range /Table/14-/Table/50
I160901 19:38:00.517330 18621 storage/replica_command.go:2213 [n1,s1,r6{/Table/50-/Max}] initiating a split of this range at key /Table/51
I160901 19:38:00.529708 18621 storage/log.go:85 Range Event: "split", range: 6, info: {"UpdatedDesc":{"range_id":6,"start_key":"ug==","end_key":"uw==","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2},"NewDesc":{"range_id":7,"start_key":"uw==","end_key":"//8=","replicas":[{"node_id":1,"store_id":1,"replica_id":1}],"next_replica_id":2}}
I160901 19:38:00.553131 18560 storage/replica.go:2139 [n1,s1,r7{/Table/51-/Max}] applying command with forced error: storage/replica.go:2064: no-op on empty Raft entry
E160901 19:38:00.553992 18560 storage/replica.go:2174 [n1,s1,r7{/Table/51-/Max}] error executing raft command: storage/replica.go:2064: no-op on empty Raft entry
panic: test timed out after 8m0s
goroutine 18682 [running]:
panic(0x1a92e00, 0xc421a827f0)
/usr/local/go/src/runtime/panic.go:500 +0x1ae
testing.startAlarm.func1()
/usr/local/go/src/testing/testing.go:918 +0x14e
created by time.goFunc
/usr/local/go/src/time/sleep.go:154 +0x78
goroutine 1 [chan receive]:
testing.(*T).Run(0xc42026ac00, 0x1c56238, 0x22, 0x1f291d0, 0x910901)
/usr/local/go/src/testing/testing.go:647 +0x56e
testing.RunTests.func1(0xc42026ac00)
/usr/local/go/src/testing/testing.go:793 +0xba
testing.tRunner(0xc42026ac00, 0xc42003bd78)
/usr/local/go/src/testing/testing.go:610 +0xca
testing.RunTests(0x1f2a7b0, 0x280a960, 0x64, 0x64, 0x0)
/usr/local/go/src/testing/testing.go:799 +0x4bb
testing.(*M).Run(0xc42003bf00, 0x3511580)
/usr/local/go/src/testing/testing.go:743 +0x130
github.com/cockroachdb/cockroach/sql_test.TestMain(0xc42003bf00)
/go/src/github.com/cockroachdb/cockroach/sql/main_test.go:199 +0x13a
main.main()
github.com/cockroachdb/cockroach/sql/_test/_testmain.go:620 +0x1b6
goroutine 17 [syscall, 7 minutes, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1
goroutine 8 [chan receive]:
github.com/cockroachdb/cockroach/util/log.(*loggingT).flushDaemon(0x2b55760)
/go/src/github.com/cockroachdb/cockroach/util/log/clog.go:1032 +0x85
created by github.com/cockroachdb/cockroach/util/log.init.1
/go/src/github.com/cockroachdb/cockroach/util/log/clog.go:613 +0xc3
goroutine 18659 [IO wait]:
net.runtime_pollWait(0x7f04797a2c30, 0x72, 0x5ef973)
/usr/local/go/src/runtime/netpoll.go:160 +0x5e
net.(*pollDesc).wait(0xc42021e530, 0x72, 0x5ec228, 0x1b067c0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
net.(*pollDesc).waitRead(0xc42021e530, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
net.(*netFD).Read(0xc42021e4d0, 0xc421281800, 0x400, 0x400, 0x0, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_unix.go:243 +0x1d9
net.(*conn).Read(0xc420144008, 0xc421281800, 0x400, 0x400, 0x9ddaeb41e68d2c1c, 0x0, 0x1f2afb8)
/usr/local/go/src/net/net.go:173 +0x97
net.(*TCPConn).Read(0xc420144008, 0xc421281800, 0x400, 0x400, 0x1, 0x2811f80, 0xc420012150)
<autogenerated>:69 +0x73
github.com/cockroachdb/cmux.(*MuxConn).Read(0xc4210de000, 0xc421281800, 0x400, 0x400, 0xc420773318, 0x12a6e39, 0xc420773318)
/go/src/github.com/cockroachdb/cmux/cmux.go:218 +0x183
crypto/tls.(*block).readFromUntil(0xc42157c120, 0x7f04797a43f8, 0xc4210de000, 0x5, 0xc4210de000, 0x140)
/usr/local/go/src/crypto/tls/conn.go:472 +0x10d
crypto/tls.(*Conn).readRecord(0xc4202f2000, 0x1f2b217, 0xc4202f2100, 0x5f1ccf)
/usr/local/go/src/crypto/tls/conn.go:574 +0x13d
crypto/tls.(*Conn).Read(0xc4202f2000, 0xc421956000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1109 +0x186
bufio.(*Reader).fill(0xc4213ef800)
/usr/local/go/src/bufio/bufio.go:97 +0x197
bufio.(*Reader).Read(0xc4213ef800, 0xc420d4c878, 0x9, 0x9, 0x10, 0x10, 0xc421dbce20)
/usr/local/go/src/bufio/bufio.go:209 +0x6ac
io.ReadAtLeast(0x2811a00, 0xc4213ef800, 0xc420d4c878, 0x9, 0x9, 0x9, 0x61a5fa, 0xc4201a8d00, 0xc420d3b298)
/usr/local/go/src/io/io.go:307 +0xb2
io.ReadFull(0x2811a00, 0xc4213ef800, 0xc420d4c878, 0x9, 0x9, 0x0, 0xc4216f4f50, 0xc420773a00)
/usr/local/go/src/io/io.go:325 +0x73
golang.org/x/net/http2.readFrameHeader(0xc420d4c878, 0x9, 0x9, 0x2811a00, 0xc4213ef800, 0x0, 0xc400000000, 0xc420d3b298, 0x2815b00)
/go/src/golang.org/x/net/http2/frame.go:237 +0xa3
golang.org/x/net/http2.(*Framer).ReadFrame(0xc420d4c840, 0x0, 0x2811f80, 0xc420012150, 0xc42322fee0)
/go/src/golang.org/x/net/http2/frame.go:464 +0x105
google.golang.org/grpc/transport.(*framer).readFrame(0xc420d3ac00, 0xc420d3b380, 0xc420d3b380, 0xc420773cc0, 0x1)
/go/src/google.golang.org/grpc/transport/http_util.go:505 +0x53
google.golang.org/grpc/transport.(*http2Server).HandleStreams(0xc4209eb320, 0xc420d3ac90)
/go/src/google.golang.org/grpc/transport/http2_server.go:260 +0x30a
google.golang.org/grpc.(*Server).serveStreams(0xc4212ce960, 0x282bae0, 0xc4209eb320)
/go/src/google.golang.org/grpc/server.go:424 +0x1d6
google.golang.org/grpc.(*Server).serveNewHTTP2Transport(0xc4212ce960, 0x282bd80, 0xc4202f2000, 0x2818980, 0xc4215aed10)
/go/src/google.golang.org/grpc/server.go:411 +0x4f1
google.golang.org/grpc.(*Server).handleRawConn(0xc4212ce960, 0x282bde0, 0xc4210de000)
/go/src/google.golang.org/grpc/server.go:388 +0x5c3
created by google.golang.org/grpc.(*Server).Serve
/go/src/google.golang.org/grpc/server.go:357 +0x20a
goroutine 18626 [select]:
github.com/cockroachdb/cockroach/storage.(*replicaScanner).waitAndProcess(0xc420097380, 0xecf5a7a97, 0xc42f4742e7, 0x2b545c0, 0xc42239c100, 0xc42103d0e0, 0xc4200a4480, 0xc420c1bb20)
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:201 +0x299
github.com/cockroachdb/cockroach/storage.(*replicaScanner).scanLoop.func1.1(0xc4200a4480, 0xc423364600)
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:259 +0x124
github.com/cockroachdb/cockroach/storage.(*storeRangeSet).Visit(0xc4233646c0, 0xc420c46680)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:242 +0x453
github.com/cockroachdb/cockroach/storage.(*replicaScanner).scanLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:261 +0x3a5
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aae0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18585 [select]:
github.com/cockroachdb/cockroach/sql.(*LeaseManager).RefreshLeases.func1()
/go/src/github.com/cockroachdb/cockroach/sql/lease.go:1143 +0x1204
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc422696f30)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18661 [select]:
google.golang.org/grpc/transport.(*http2Server).controller(0xc4209eb320)
/go/src/google.golang.org/grpc/transport/http2_server.go:673 +0xa56
created by google.golang.org/grpc/transport.newHTTP2Server
/go/src/google.golang.org/grpc/transport/http2_server.go:139 +0xa91
goroutine 18599 [select]:
github.com/cockroachdb/cockroach/storage.(*bookie).start.func1()
/go/src/github.com/cockroachdb/cockroach/storage/reservation.go:274 +0x4c8
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc420ba8240)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18639 [runnable]:
github.com/cockroachdb/cockroach/kv.grpcTransportFactory.func1()
/go/src/github.com/cockroachdb/cockroach/kv/transport_race.go:80 +0x3eb
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc421c96bc0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18610 [runnable]:
github.com/cockroachdb/cockroach/storage/engine._Cfunc_DBIterSeek(0x7f046ce35278, 0xc423206bc0, 0x1b, 0x0, 0x0, 0x1a59560, 0x7f046ce35278, 0x3, 0x623310, 0x3, ...)
??:0
github.com/cockroachdb/cockroach/storage/engine.(*rocksDBIterator).Seek(0xc420a3c870, 0xc423206bc0, 0x1b, 0x20, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/storage/engine/rocksdb.go:1101 +0x4bc
github.com/cockroachdb/cockroach/storage/engine.mvccGetMetadata(0x282fa00, 0xc420a3c870, 0xc423206bc0, 0x1b, 0x20, 0x0, 0x0, 0xc420c12400, 0xc420a3c870, 0x1a59501, ...)
/go/src/github.com/cockroachdb/cockroach/storage/engine/mvcc.go:618 +0x98
github.com/cockroachdb/cockroach/storage/engine.mvccGetUsingIter(0x7f04797a17e0, 0xc42117c630, 0x282fa00, 0xc420a3c870, 0xc423206bc0, 0x1b, 0x20, 0x0, 0x0, 0x1, ...)
/go/src/github.com/cockroachdb/cockroach/storage/engine/mvcc.go:558 +0x29f
github.com/cockroachdb/cockroach/storage/engine.MVCCGet(0x7f04797a17e0, 0xc42117c630, 0x7f04797a2f30, 0xc423849d40, 0xc423206bc0, 0x1b, 0x20, 0x0, 0x0, 0xc420958501, ...)
/go/src/github.com/cockroachdb/cockroach/storage/engine/mvcc.go:539 +0x175
github.com/cockroachdb/cockroach/storage/engine.MVCCGetProto(0x7f04797a17e0, 0xc42117c630, 0x7f04797a2f30, 0xc423849d40, 0xc423206bc0, 0x1b, 0x20, 0x0, 0x7f0400000000, 0x1, ...)
/go/src/github.com/cockroachdb/cockroach/storage/engine/mvcc.go:450 +0xd0
github.com/cockroachdb/cockroach/storage.(*AbortCache).Get(0xc423b946b8, 0x7f04797a17e0, 0xc42117c630, 0x7f04797a2f30, 0xc423849d40, 0xc421bfc5d0, 0xc42117c9c0, 0x0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/storage/abort_cache.go:101 +0x162
github.com/cockroachdb/cockroach/storage.(*Replica).checkIfTxnAborted(0xc420debd40, 0x7f04797a17e0, 0xc42117c630, 0x7f04797a2f30, 0xc423849d40, 0xc421bfc5d0, 0x0, 0xc421bfc5e0, 0x4, 0x8, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:2362 +0x17c
github.com/cockroachdb/cockroach/storage.(*Replica).applyRaftCommandInBatch(0xc420debd40, 0x7f04797a17e0, 0xc42117c630, 0xc421bfc578, 0x8, 0x100000001, 0x1, 0x14704a3788e90994, 0x0, 0x100000001, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:2313 +0x6de
github.com/cockroachdb/cockroach/storage.(*Replica).applyRaftCommand(0xc420debd40, 0xc421bfc578, 0x8, 0x7f04797a17e0, 0xc42117c630, 0xf, 0x4, 0x100000001, 0x1, 0x14704a3788e90994, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:2227 +0xc1f
github.com/cockroachdb/cockroach/storage.(*Replica).processRaftCommand(0xc420debd40, 0xc421bfc578, 0x8, 0xf, 0x6, 0x100000001, 0x1, 0x14704a3788e90994, 0x0, 0x100000001, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:2143 +0x3d9
github.com/cockroachdb/cockroach/storage.(*Replica).handleRaftReady(0xc420debd40, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:1750 +0x9cd
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1.4()
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2602 +0xa4
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1.1(0xc423fab740)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2522 +0x74
created by github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2524 +0xd8
goroutine 18642 [select]:
github.com/cockroachdb/cockroach/rpc.(*Context).runHeartbeat(0xc42136ac80, 0xc420937000, 0xc420e2d170, 0xf, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/rpc/context.go:282 +0x813
github.com/cockroachdb/cockroach/rpc.(*Context).GRPCDial.func1.1.1()
/go/src/github.com/cockroachdb/cockroach/rpc/context.go:205 +0xb3
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc423b581b0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18580 [chan receive]:
github.com/cockroachdb/cockroach/storage/engine.(*RocksDB).Open.func1(0xc423849d40)
/go/src/github.com/cockroachdb/cockroach/storage/engine/rocksdb.go:388 +0x64
created by github.com/cockroachdb/cockroach/storage/engine.(*RocksDB).Open
/go/src/github.com/cockroachdb/cockroach/storage/engine/rocksdb.go:389 +0xb61
goroutine 18625 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aaa0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18623 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aa60)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18587 [chan receive]:
github.com/cockroachdb/cockroach/util/netutil.MakeServer.func2()
/go/src/github.com/cockroachdb/cockroach/util/netutil/net.go:100 +0x8e
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bae0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18592 [chan receive]:
github.com/cockroachdb/cmux.(*muxListener).Accept(0xc42117bd80, 0xc4228f97d0, 0xc422697b00, 0xc422697b00, 0x281bb00)
<autogenerated>:9 +0xa4
crypto/tls.(*listener).Accept(0xc42117be00, 0xc4228f97a0, 0x1ad5fe0, 0x27f83f0, 0x1b5ef60)
/usr/local/go/src/crypto/tls/tls.go:52 +0x56
net/http.(*Server).Serve(0xc420a4a380, 0x281dbc0, 0xc42117be00, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2273 +0x249
github.com/cockroachdb/cockroach/server.(*Server).Start.func5()
/go/src/github.com/cockroachdb/cockroach/server/server.go:401 +0x8f
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117be40)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18654 [select]:
github.com/cockroachdb/cockroach/storage.(*Replica).addWriteCmd(0xc420debd40, 0x7f04797a17e0, 0xc42117c630, 0x14704a3788e90994, 0x0, 0x100000001, 0x1, 0x6, 0xbff0000000000000, 0xc423b08cc0, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:1408 +0x5a5
github.com/cockroachdb/cockroach/storage.(*Replica).Send(0xc420debd40, 0x7f04797a17e0, 0xc42117c630, 0x14704a3788e90994, 0x0, 0x100000001, 0x1, 0x6, 0xbff0000000000000, 0xc423b08cc0, ...)
/go/src/github.com/cockroachdb/cockroach/storage/replica.go:969 +0x2a5
github.com/cockroachdb/cockroach/storage.(*Store).Send(0xc420c1b900, 0x7f04797a17e0, 0xc42117c570, 0x14704a3788e90994, 0x0, 0x100000001, 0x1, 0x6, 0xbff0000000000000, 0xc423b08cc0, ...)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2082 +0x8ee
github.com/cockroachdb/cockroach/storage.(*Stores).Send(0xc42239c8c0, 0x7f04797a17e0, 0xc42117c4b0, 0x0, 0x0, 0x100000001, 0x1, 0x6, 0xbff0000000000000, 0xc423b08cc0, ...)
/go/src/github.com/cockroachdb/cockroach/storage/stores.go:182 +0x24b
github.com/cockroachdb/cockroach/server.(*Node).Batch.func3()
/go/src/github.com/cockroachdb/cockroach/server/node.go:823 +0x3d4
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunTask(0xc42103d0e0, 0xc423666c20, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:215 +0x10e
github.com/cockroachdb/cockroach/server.(*Node).Batch(0xc420176d80, 0x7f04797a3ea8, 0xc42084c180, 0xc421f60168, 0x0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/server/node.go:835 +0x2cf
github.com/cockroachdb/cockroach/kv.(*grpcTransport).SendNext(0xc420a3c280, 0xc42084c0c0)
/go/src/github.com/cockroachdb/cockroach/kv/transport.go:187 +0x3ec
github.com/cockroachdb/cockroach/kv.(*DistSender).sendToReplicas(0xc4207dbe60, 0x7f04797a17e0, 0xc42117c2d0, 0x2540be400, 0xb2d05e00, 0x0, 0x6, 0xc423206460, 0x1, 0x1, ...)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:1025 +0x203
github.com/cockroachdb/cockroach/kv.(*DistSender).sendRPC(0xc4207dbe60, 0x7f04797a17e0, 0xc42117c2d0, 0x6, 0xc423206460, 0x1, 0x1, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:379 +0x27c
github.com/cockroachdb/cockroach/kv.(*DistSender).sendSingleRange(0xc4207dbe60, 0x7f04797a17e0, 0xc42117c2d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbff0000000000000, 0xc423b08480, ...)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:495 +0x1d9
github.com/cockroachdb/cockroach/kv.(*DistSender).sendChunk.func2(0xc423667c40, 0xc423667ad8, 0xc4236681e0, 0xc4207dbe60, 0xc4236681d0, 0x4, 0xc420ba1430)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:778 +0x2eb
github.com/cockroachdb/cockroach/kv.(*DistSender).sendChunk(0xc4207dbe60, 0x7f04797a17e0, 0xc42117c2d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbff0000000000000, 0xc423b08480, ...)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:779 +0x89d
github.com/cockroachdb/cockroach/kv.(*DistSender).Send(0xc4207dbe60, 0x7f04797a17e0, 0xc42117c2d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbff0000000000000, 0xc423b083c0, ...)
/go/src/github.com/cockroachdb/cockroach/kv/dist_sender.go:606 +0x1a1
github.com/cockroachdb/cockroach/kv.(*TxnCoordSender).Send(0xc420ebef20, 0x7f04797a17e0, 0xc42117c2d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbff0000000000000, 0xc42015d238, ...)
/go/src/github.com/cockroachdb/cockroach/kv/txn_coord_sender.go:391 +0x68a
github.com/cockroachdb/cockroach/internal/client.(*txnSender).Send(0xc42015d1e0, 0x7f04797a17a0, 0xc420012828, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbff0000000000000, 0xc42015d238, ...)
/go/src/github.com/cockroachdb/cockroach/internal/client/txn.go:52 +0x121
github.com/cockroachdb/cockroach/internal/client.(*DB).send(0xc42015d1e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/internal/client/db.go:490 +0x2bf
github.com/cockroachdb/cockroach/internal/client.(*Txn).send(0xc42015d1e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/internal/client/txn.go:675 +0x676
github.com/cockroachdb/cockroach/internal/client.(*Txn).(github.com/cockroachdb/cockroach/internal/client.send)-fm(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/internal/client/txn.go:359 +0x7d
github.com/cockroachdb/cockroach/internal/client.sendAndFill(0xc423669630, 0xc420d9a000, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/internal/client/db.go:417 +0x1ac
github.com/cockroachdb/cockroach/internal/client.(*Txn).Run(0xc42015d1e0, 0xc420d9a000, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/internal/client/txn.go:359 +0xfe
github.com/cockroachdb/cockroach/sql.(*tableInserter).finalize(0xc420d0e120, 0x7f04797a17a0, 0xc420012828, 0x0, 0xc420012828)
/go/src/github.com/cockroachdb/cockroach/sql/tablewriter.go:116 +0x189
github.com/cockroachdb/cockroach/sql.(*insertNode).Next(0xc4227b8180, 0x282dc01, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/insert.go:265 +0x1087
github.com/cockroachdb/cockroach/sql.countRowsAffected(0x282dcc0, 0xc4227b8180, 0x6, 0xc423669b00, 0x282dcc0)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:607 +0xda
github.com/cockroachdb/cockroach/sql.(*Executor).execStmt(0xc42281b680, 0x2825bc0, 0xc42043b2c0, 0xc4213b10a0, 0xc42043b200, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:1091 +0x937
github.com/cockroachdb/cockroach/sql.(*Executor).execStmtInOpenTxn(0xc42281b680, 0x2825bc0, 0xc42043b2c0, 0xc4213b10a0, 0x0, 0xc4213b1028, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:962 +0x331
github.com/cockroachdb/cockroach/sql.(*Executor).execStmtsInCurrentTxn(0xc42281b680, 0xc421c61530, 0x1, 0x1, 0xc4213b10a0, 0xc4213b1028, 0xc423660000, 0xc42056e900, 0x2, 0x2, ...)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:709 +0x511
github.com/cockroachdb/cockroach/sql.runTxnAttempt(0xc42281b680, 0xc4213b10a0, 0x1, 0xc4213b1028, 0xc4227c02a0, 0xc421c61530, 0x1, 0x1, 0xc42366ae38, 0x765e875e27a624e, ...)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:634 +0x202
github.com/cockroachdb/cockroach/sql.(*Executor).execRequest.func2(0xc42015d1e0, 0xc4227c02a0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:518 +0x19a
github.com/cockroachdb/cockroach/internal/client.(*Txn).Exec(0xc42015d1e0, 0xc423970000, 0x0, 0x0, 0xc42397d0e0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/internal/client/txn.go:553 +0x231
github.com/cockroachdb/cockroach/sql.(*Executor).execRequest(0xc42281b680, 0xc4213b1000, 0xc422115087, 0x31, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:523 +0x5d3
github.com/cockroachdb/cockroach/sql.(*Executor).ExecuteStatements(0xc42281b680, 0xc4213b1000, 0xc422115087, 0x31, 0x0, 0x32, 0xf79, 0xc42366b500, 0x31)
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:392 +0x13f
github.com/cockroachdb/cockroach/sql/pgwire.(*v3Conn).executeStatements(0xc4227b9500, 0x7f04797a18a0, 0xc420334380, 0xc422115087, 0x31, 0x0, 0x0, 0x0, 0x0, 0x10000c4220a0101, ...)
/go/src/github.com/cockroachdb/cockroach/sql/pgwire/v3.go:641 +0xba
github.com/cockroachdb/cockroach/sql/pgwire.(*v3Conn).handleSimpleQuery(0xc4227b9500, 0x7f04797a18a0, 0xc420334380, 0xc4227b9528, 0x36, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/pgwire/v3.go:320 +0x10a
github.com/cockroachdb/cockroach/sql/pgwire.(*v3Conn).serve(0xc4227b9500, 0xc4209a6840, 0xc4209a6840, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/pgwire/v3.go:275 +0xa20
github.com/cockroachdb/cockroach/sql/pgwire.(*Server).ServeConn(0xc42239c880, 0x282bd80, 0xc420a6c380, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/sql/pgwire/server.go:240 +0x98a
github.com/cockroachdb/cockroach/server.(*Server).Start.func8.1(0x282bde0, 0xc4209651e0)
/go/src/github.com/cockroachdb/cockroach/server/server.go:417 +0x89
github.com/cockroachdb/cockroach/util/netutil.(*Server).ServeWith.func1(0xc42103d0e0, 0xc420144dc8, 0x282bde0, 0xc4209651e0, 0xc421620430)
/go/src/github.com/cockroachdb/cockroach/util/netutil/net.go:138 +0xc9
created by github.com/cockroachdb/cockroach/util/netutil.(*Server).ServeWith
/go/src/github.com/cockroachdb/cockroach/util/netutil/net.go:140 +0x2f9
goroutine 18635 [select]:
github.com/cockroachdb/cockroach/sql.(*SchemaChangeManager).Start.func1()
/go/src/github.com/cockroachdb/cockroach/sql/schema_changer.go:704 +0x1a60
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158acc0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18622 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aa20)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18652 [chan receive]:
database/sql.(*DB).connectionOpener(0xc420964f20)
/usr/local/go/src/database/sql/sql.go:730 +0x6e
created by database/sql.Open
/usr/local/go/src/database/sql/sql.go:493 +0x2af
goroutine 18589 [chan receive]:
github.com/cockroachdb/cockroach/server.(*Server).Start.func2()
/go/src/github.com/cockroachdb/cockroach/server/server.go:378 +0xae
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bd40)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18612 [select]:
github.com/cockroachdb/cockroach/storage.(*Store).startGossip.func1()
/go/src/github.com/cockroachdb/cockroach/storage/store.go:1035 +0x34a
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc421621cc0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18598 [select]:
github.com/cockroachdb/cockroach/gossip.(*Gossip).manage.func1()
/go/src/github.com/cockroachdb/cockroach/gossip/gossip.go:948 +0x416
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc420e2d220)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18607 [semacquire]:
sync.runtime_Semacquire(0xc421bfc51c)
/usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*WaitGroup).Wait(0xc421bfc510)
/usr/local/go/src/sync/waitgroup.go:131 +0xbf
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1()
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2608 +0x793
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc421c7d310)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18579 [IO wait]:
net.runtime_pollWait(0x7f04797a2630, 0x72, 0x5ef973)
/usr/local/go/src/runtime/netpoll.go:160 +0x5e
net.(*pollDesc).wait(0xc42194c370, 0x72, 0x5ec228, 0x1b067c0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
net.(*pollDesc).waitRead(0xc42194c370, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
net.(*netFD).Read(0xc42194c310, 0xc42106f400, 0x400, 0x400, 0x0, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_unix.go:243 +0x1d9
net.(*conn).Read(0xc420028e20, 0xc42106f400, 0x400, 0x400, 0x102, 0xc42106f40d, 0xc42106f405)
/usr/local/go/src/net/net.go:173 +0x97
net.(*TCPConn).Read(0xc420028e20, 0xc42106f400, 0x400, 0x400, 0xc421459100, 0x12a6e39, 0xc421459100)
<autogenerated>:69 +0x73
crypto/tls.(*block).readFromUntil(0xc4228192c0, 0x7f04797a3ee8, 0xc420028e20, 0x5, 0xc420028e20, 0x0)
/usr/local/go/src/crypto/tls/conn.go:472 +0x10d
crypto/tls.(*Conn).readRecord(0xc420a6c700, 0x1f2b217, 0xc420a6c800, 0xc423dbb700)
/usr/local/go/src/crypto/tls/conn.go:574 +0x13d
crypto/tls.(*Conn).Read(0xc420a6c700, 0xc4213e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1109 +0x186
bufio.(*Reader).fill(0xc4203e9e60)
/usr/local/go/src/bufio/bufio.go:97 +0x197
bufio.(*Reader).Read(0xc4203e9e60, 0xc4208822a0, 0x5, 0x200, 0xc4214596f8, 0x61a5fa, 0xc4203e29c0)
/usr/local/go/src/bufio/bufio.go:209 +0x6ac
io.ReadAtLeast(0x2811a00, 0xc4203e9e60, 0xc4208822a0, 0x5, 0x200, 0x5, 0x1f2b240, 0x4, 0x61a277)
/usr/local/go/src/io/io.go:307 +0xb2
io.ReadFull(0x2811a00, 0xc4203e9e60, 0xc4208822a0, 0x5, 0x200, 0x1abd7a0, 0x0, 0x30000626dbd)
/usr/local/go/src/io/io.go:325 +0x73
github.com/cockroachdb/pq.(*conn).recvMessage(0xc420882280, 0xc4227c0040, 0x0, 0x1a98420, 0x1)
/go/src/github.com/cockroachdb/pq/conn.go:953 +0xde
github.com/cockroachdb/pq.(*conn).recv1Buf(0xc420882280, 0xc4227c0040, 0xc4208822a0)
/go/src/github.com/cockroachdb/pq/conn.go:1003 +0x47
github.com/cockroachdb/pq.(*conn).recv1(0xc420882280, 0xc421459990, 0x1fb)
/go/src/github.com/cockroachdb/pq/conn.go:1024 +0xb0
github.com/cockroachdb/pq.(*conn).simpleExec(0xc420882280, 0x1c6e632, 0x31, 0xc421459a60, 0xc421459a10, 0x5ec949, 0x1af8f40, 0x1c21300, 0xfb8601)
/go/src/github.com/cockroachdb/pq/conn.go:614 +0x2c0
github.com/cockroachdb/pq.(*conn).Exec(0xc420882280, 0x1c6e632, 0x31, 0x35119e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/cockroachdb/pq/conn.go:864 +0x158
database/sql.(*Tx).Exec(0xc420a3c000, 0x1c6e632, 0x31, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/database/sql/sql.go:1382 +0x529
github.com/cockroachdb/cockroach/sql_test.TestErrorOnCommitResultsInRollback(0xc420e0f140)
/go/src/github.com/cockroachdb/cockroach/sql/txn_restart_test.go:582 +0x311
testing.tRunner(0xc420e0f140, 0x1f291d0)
/usr/local/go/src/testing/testing.go:610 +0xca
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:646 +0x530
goroutine 18586 [select]:
github.com/cockroachdb/cockroach/sql.NewExecutor.func1()
/go/src/github.com/cockroachdb/cockroach/sql/executor.go:270 +0x1c7
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc4238b20c0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18611 [select]:
github.com/cockroachdb/cockroach/storage.(*Store).Start.func3()
/go/src/github.com/cockroachdb/cockroach/storage/store.go:959 +0x19d
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158a160)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18648 [select]:
google.golang.org/grpc/transport.(*http2Client).controller(0xc421bb0f00)
/go/src/google.golang.org/grpc/transport/http2_client.go:999 +0x8c9
created by google.golang.org/grpc/transport.newHTTP2Client
/go/src/google.golang.org/grpc/transport/http2_client.go:233 +0x14bd
goroutine 18634 [select]:
github.com/cockroachdb/cockroach/server.(*Node).startWriteSummaries.func1()
/go/src/github.com/cockroachdb/cockroach/server/node.go:667 +0x32d
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aca0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18658 [select]:
github.com/cockroachdb/cockroach/storage.(*idAllocator).start.func1()
/go/src/github.com/cockroachdb/cockroach/storage/id_alloc.go:124 +0xacc
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc4227701e0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18594 [chan receive]:
github.com/cockroachdb/cmux.(*muxListener).Accept(0xc42117bc60, 0x1f2a500, 0xc4212ce960, 0x282bde0, 0xc4210de000)
<autogenerated>:9 +0xa4
google.golang.org/grpc.(*Server).Serve(0xc4212ce960, 0x2824040, 0xc42117bc60, 0x0, 0x0)
/go/src/google.golang.org/grpc/server.go:348 +0x1b8
github.com/cockroachdb/cockroach/server.(*Server).Start.func7()
/go/src/github.com/cockroachdb/cockroach/server/server.go:412 +0x8d
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117be80)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18591 [chan receive]:
github.com/cockroachdb/cmux.(*muxListener).Accept(0xc42117bd60, 0xc4228f9710, 0x1ad5fe0, 0x27f83f0, 0x1b5ef60)
<autogenerated>:9 +0xa4
net/http.(*Server).Serve(0xc420a4a400, 0x2824040, 0xc42117bd60, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2273 +0x249
github.com/cockroachdb/cockroach/server.(*Server).Start.func4()
/go/src/github.com/cockroachdb/cockroach/server/server.go:394 +0x54
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bde0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18621 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aa00)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18633 [select]:
github.com/cockroachdb/cockroach/ts.(*poller).start.func1()
/go/src/github.com/cockroachdb/cockroach/ts/db.go:81 +0x1c7
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc421c964a0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18632 [select]:
github.com/cockroachdb/cockroach/server.(*Server).startSampleEnvironment.func1()
/go/src/github.com/cockroachdb/cockroach/server/server.go:623 +0x1b4
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158ac80)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18624 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158aa80)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18595 [chan receive]:
github.com/cockroachdb/cmux.(*muxListener).Accept(0xc42117bc40, 0x1f298a0, 0xc42103d0e0, 0xc420144dc8, 0x282bde0)
<autogenerated>:9 +0xa4
github.com/cockroachdb/cockroach/util/netutil.(*Server).ServeWith(0xc420144dc8, 0xc42103d0e0, 0x2824040, 0xc42117bc40, 0xc421620430, 0xd94d3d, 0xc421810788)
/go/src/github.com/cockroachdb/cockroach/util/netutil/net.go:117 +0x71
github.com/cockroachdb/cockroach/server.(*Server).Start.func8()
/go/src/github.com/cockroachdb/cockroach/server/server.go:420 +0x112
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc422697b90)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18605 [select]:
github.com/cockroachdb/cockroach/storage.(*bookie).start.func1()
/go/src/github.com/cockroachdb/cockroach/storage/reservation.go:274 +0x4c8
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc4209181e0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18590 [IO wait]:
net.runtime_pollWait(0x7f04797a2db0, 0x72, 0x5ef973)
/usr/local/go/src/runtime/netpoll.go:160 +0x5e
net.(*pollDesc).wait(0xc4217be3e0, 0x72, 0x5ec228, 0x1b067c0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
net.(*pollDesc).waitRead(0xc4217be3e0, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
net.(*netFD).accept(0xc4217be380, 0x0, 0x2816940, 0xc421d93300)
/usr/local/go/src/net/fd_unix.go:419 +0x2b8
net.(*TCPListener).accept(0xc420144e00, 0xc420027ed8, 0x60a543, 0xc420027ea8)
/usr/local/go/src/net/tcpsock_posix.go:132 +0x51
net.(*TCPListener).Accept(0xc420144e00, 0x1f27400, 0xc42239cb00, 0xc421620420, 0x180001)
/usr/local/go/src/net/tcpsock.go:222 +0x50
github.com/cockroachdb/cmux.(*cMux).Serve(0xc42239cb00, 0x0, 0x0)
/go/src/github.com/cockroachdb/cmux/cmux.go:124 +0xdd
github.com/cockroachdb/cockroach/server.(*Server).Start.func3()
/go/src/github.com/cockroachdb/cockroach/server/server.go:390 +0x47
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bdc0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18560 [chan receive]:
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1.1(0xc423fab740)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2521 +0x58
created by github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2524 +0xd8
goroutine 18636 [IO wait]:
net.runtime_pollWait(0x7f046c43b630, 0x72, 0x5ef973)
/usr/local/go/src/runtime/netpoll.go:160 +0x5e
net.(*pollDesc).wait(0xc4217be370, 0x72, 0x5ec228, 0x1b067c0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
net.(*pollDesc).waitRead(0xc4217be370, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
net.(*netFD).accept(0xc4217be310, 0x0, 0x2816940, 0xc423c92180)
/usr/local/go/src/net/fd_unix.go:419 +0x2b8
net.(*TCPListener).accept(0xc420144dd0, 0xc420028e10, 0xc421c9eec8, 0x61486f)
/usr/local/go/src/net/tcpsock_posix.go:132 +0x51
net.(*TCPListener).Accept(0xc420144dd0, 0x1f27408, 0xc42239cac0, 0x282c020, 0xc420028e10)
/usr/local/go/src/net/tcpsock.go:222 +0x50
github.com/cockroachdb/cmux.(*cMux).Serve(0xc42239cac0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cmux/cmux.go:124 +0xdd
github.com/cockroachdb/cockroach/server.(*Server).Start.func11()
/go/src/github.com/cockroachdb/cockroach/server/server.go:495 +0x47
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158ad20)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18584 [select]:
github.com/cockroachdb/cockroach/storage.NewRaftTransport.func1()
/go/src/github.com/cockroachdb/cockroach/storage/raft_transport.go:167 +0x12f6
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc420e2c9a0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18613 [select]:
github.com/cockroachdb/cockroach/storage.(*Store).startGossip.func2()
/go/src/github.com/cockroachdb/cockroach/storage/store.go:1055 +0x34a
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc421621cd0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18593 [chan receive]:
github.com/cockroachdb/cockroach/server.(*Server).Start.func6()
/go/src/github.com/cockroachdb/cockroach/server/server.go:405 +0xa6
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117be60)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18647 [IO wait]:
net.runtime_pollWait(0x7f046c43b7b0, 0x72, 0x5ef973)
/usr/local/go/src/runtime/netpoll.go:160 +0x5e
net.(*pollDesc).wait(0xc421a52a00, 0x72, 0x5ec228, 0x1b067c0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
net.(*pollDesc).waitRead(0xc421a52a00, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
net.(*netFD).Read(0xc421a529a0, 0xc42080f000, 0x400, 0x400, 0x0, 0x2819740, 0xc4200120a8)
/usr/local/go/src/net/fd_unix.go:243 +0x1d9
net.(*conn).Read(0xc4200283a8, 0xc42080f000, 0x400, 0x400, 0x102, 0xc42080f00d, 0xc42080f005)
/usr/local/go/src/net/net.go:173 +0x97
net.(*TCPConn).Read(0xc4200283a8, 0xc42080f000, 0x400, 0x400, 0xc423d71540, 0x12a6e39, 0xc423d71540)
<autogenerated>:69 +0x73
crypto/tls.(*block).readFromUntil(0xc423498810, 0x7f04797a3ee8, 0xc4200283a8, 0x5, 0xc4200283a8, 0x0)
/usr/local/go/src/crypto/tls/conn.go:472 +0x10d
crypto/tls.(*Conn).readRecord(0xc420061180, 0x1f2b217, 0xc420061280, 0xc42080f00d)
/usr/local/go/src/crypto/tls/conn.go:574 +0x13d
crypto/tls.(*Conn).Read(0xc420061180, 0xc421eb8000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1109 +0x186
bufio.(*Reader).fill(0xc42031fe60)
/usr/local/go/src/bufio/bufio.go:97 +0x197
bufio.(*Reader).Read(0xc42031fe60, 0xc4215b44b8, 0x9, 0x9, 0xc4206fe000, 0x1040100000030, 0x0)
/usr/local/go/src/bufio/bufio.go:209 +0x6ac
io.ReadAtLeast(0x2811a00, 0xc42031fe60, 0xc4215b44b8, 0x9, 0x9, 0x9, 0x14c9d3d, 0x281b700, 0xc420d3b740)
/usr/local/go/src/io/io.go:307 +0xb2
io.ReadFull(0x2811a00, 0xc42031fe60, 0xc4215b44b8, 0x9, 0x9, 0xc420d3b830, 0x4, 0x0)
/usr/local/go/src/io/io.go:325 +0x73
golang.org/x/net/http2.readFrameHeader(0xc4215b44b8, 0x9, 0x9, 0x2811a00, 0xc42031fe60, 0x0, 0xc400000000, 0x0, 0x0)
/go/src/golang.org/x/net/http2/frame.go:237 +0xa3
golang.org/x/net/http2.(*Framer).ReadFrame(0xc4215b4480, 0x281b8c0, 0xc420d3b830, 0x0, 0xc423d71e20)
/go/src/golang.org/x/net/http2/frame.go:464 +0x105
google.golang.org/grpc/transport.(*framer).readFrame(0xc42388be00, 0xc420d3b830, 0xc420d3b830, 0xc423d71eb0, 0x1)
/go/src/google.golang.org/grpc/transport/http_util.go:505 +0x53
google.golang.org/grpc/transport.(*http2Client).reader(0xc421bb0f00)
/go/src/google.golang.org/grpc/transport/http2_client.go:921 +0x154
created by google.golang.org/grpc/transport.newHTTP2Client
/go/src/google.golang.org/grpc/transport/http2_client.go:203 +0xc1a
goroutine 18638 [select]:
google.golang.org/grpc.(*addrConn).transportMonitor(0xc4210fac80)
/go/src/google.golang.org/grpc/clientconn.go:720 +0x7d7
google.golang.org/grpc.(*ClientConn).resetAddrConn.func1(0xc4210fac80)
/go/src/google.golang.org/grpc/clientconn.go:479 +0x26e
created by google.golang.org/grpc.(*ClientConn).resetAddrConn
/go/src/google.golang.org/grpc/clientconn.go:480 +0x647
goroutine 18588 [chan receive]:
github.com/cockroachdb/cockroach/util/netutil.MakeServer.func2()
/go/src/github.com/cockroachdb/cockroach/util/netutil/net.go:100 +0x8e
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bb60)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18582 [select]:
github.com/cockroachdb/cockroach/storage.(*StorePool).start.func1()
/go/src/github.com/cockroachdb/cockroach/storage/store_pool.go:340 +0x519
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117b600)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18559 [chan receive]:
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1.1(0xc423fab740)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2521 +0x58
created by github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2524 +0xd8
goroutine 18597 [select]:
github.com/cockroachdb/cockroach/gossip.(*Gossip).bootstrap.func1()
/go/src/github.com/cockroachdb/cockroach/gossip/gossip.go:911 +0x621
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc420e2d210)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18583 [select]:
github.com/cockroachdb/cockroach/kv.(*TxnCoordSender).startStats(0xc420ebef20)
/go/src/github.com/cockroachdb/cockroach/kv/txn_coord_sender.go:208 +0xde7
github.com/cockroachdb/cockroach/kv.(*TxnCoordSender).(github.com/cockroachdb/cockroach/kv.startStats)-fm()
/go/src/github.com/cockroachdb/cockroach/kv/txn_coord_sender.go:193 +0x42
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc420e2c950)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18627 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158ab00)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18620 [select]:
github.com/cockroachdb/cockroach/storage.(*baseQueue).processLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/queue.go:418 +0x395
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158a9e0)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18629 [select]:
github.com/cockroachdb/cockroach/server.(*Node).startComputePeriodicMetrics.func1()
/go/src/github.com/cockroachdb/cockroach/server/node.go:631 +0x2f3
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158ab60)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18581 [chan receive]:
github.com/cockroachdb/cockroach/rpc.NewContext.func1()
/go/src/github.com/cockroachdb/cockroach/rpc/context.go:122 +0x95
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117a840)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18561 [chan receive]:
github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1.1(0xc423fab740)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2521 +0x58
created by github.com/cockroachdb/cockroach/storage.(*Store).processRaft.func1
/go/src/github.com/cockroachdb/cockroach/storage/store.go:2524 +0xd8
goroutine 18630 [select]:
github.com/cockroachdb/cockroach/server.(*Node).startGossip.func1()
/go/src/github.com/cockroachdb/cockroach/server/node.go:591 +0x545
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc423366f30)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18596 [chan receive]:
github.com/cockroachdb/cockroach/gossip.(*server).start.func3()
/go/src/github.com/cockroachdb/cockroach/gossip/server.go:370 +0x94
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42117bf40)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
goroutine 18628 [select]:
github.com/cockroachdb/cockroach/storage.(*replicaScanner).waitAndProcess(0xc420097400, 0xecf5a7a97, 0xc42f48bb46, 0x2b545c0, 0xc42239c100, 0xc42103d0e0, 0xc4200a4480, 0xc420c1bb20)
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:201 +0x299
github.com/cockroachdb/cockroach/storage.(*replicaScanner).scanLoop.func1.1(0xc4200a4480, 0xc423364600)
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:259 +0x124
github.com/cockroachdb/cockroach/storage.(*storeRangeSet).Visit(0xc423364b70, 0xc420c46900)
/go/src/github.com/cockroachdb/cockroach/storage/store.go:242 +0x453
github.com/cockroachdb/cockroach/storage.(*replicaScanner).scanLoop.func1()
/go/src/github.com/cockroachdb/cockroach/storage/scanner.go:261 +0x3a5
github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc42103d0e0, 0xc42158ab20)
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x8b
created by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker
/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x74
FAIL github.com/cockroachdb/cockroach/sql 480.549s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment