rally.log

2019-11-19 00:39:26,617 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [8/32] drivers reached join point [5/5].
2019-11-19 00:39:26,617 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [9/32] drivers reached join point [5/5].
2019-11-19 00:39:26,617 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [10/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [11/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [12/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [13/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [14/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [15/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [16/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [17/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [18/32] drivers reached join point [5/5].
2019-11-19 00:39:26,618 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [19/32] drivers reached join point [5/5].
2019-11-19 00:39:26,619 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [20/32] drivers reached join point [5/5].
2019-11-19 00:39:26,619 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [21/32] drivers reached join point [5/5].
2019-11-19 00:39:26,619 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [22/32] drivers reached join point [5/5].
2019-11-19 00:39:26,620 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [23/32] drivers reached join point [5/5].
2019-11-19 00:39:26,623 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [24/32] drivers reached join point [5/5].
2019-11-19 00:39:26,988 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [25/32] drivers reached join point [5/5].
2019-11-19 00:39:27,163 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [26/32] drivers reached join point [5/5].
2019-11-19 00:39:27,165 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [27/32] drivers reached join point [5/5].
2019-11-19 00:39:27,338 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [28/32] drivers reached join point [5/5].
2019-11-19 00:39:27,339 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [29/32] drivers reached join point [5/5].
2019-11-19 00:39:27,341 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [30/32] drivers reached join point [5/5].
2019-11-19 00:39:27,343 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [31/32] drivers reached join point [5/5].
2019-11-19 00:40:26,461 -not-actor-/PID:17499 elasticsearch WARNING POST http://xxxxxx:80/_all/_forcemerge [status:N/A request:59.870s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1213, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1213, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
2019-11-19 00:42:49,937 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [14] has exited prematurely. Aborting benchmark.
2019-11-19 00:42:49,947 ActorAddr-(T|:16144)/PID:17499 esrally.driver.driver INFO Iteration-count-based schedule stopped after [0] warmup iterations and [1] iterations.
2019-11-19 00:43:20,418 ActorAddr-(T|:16144)/PID:17499 esrally.actor INFO LoadGenerator[0] is ready for the next task.
2019-11-19 00:43:20,419 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [7] has exited prematurely. Aborting benchmark.
2019-11-19 00:43:20,418 ActorAddr-(T|:16144)/PID:17499 esrally.actor INFO LoadGenerator[0] reached join point [JoinPoint(5)].
2019-11-19 00:43:20,421 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO [32/32] drivers reached join point [5/5].
2019-11-19 00:43:20,421 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO All drivers completed their tasks until join point [5/5].
2019-11-19 00:43:20,431 ActorAddr-(T|:24632)/PID:17497 esrally.driver.driver INFO All steps completed.
2019-11-19 00:43:20,431 ActorAddr-(T|:24632)/PID:17497 esrally.metrics INFO Closing metrics store.
2019-11-19 00:44:14,631 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [30] has exited prematurely. Aborting benchmark.
2019-11-19 00:47:35,881 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [12] has exited prematurely. Aborting benchmark.
2019-11-19 00:50:54,555 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [13] has exited prematurely. Aborting benchmark.
2019-11-19 00:51:44,63 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [26] has exited prematurely. Aborting benchmark.
2019-11-19 00:53:11,377 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [21] has exited prematurely. Aborting benchmark.
2019-11-19 00:56:51,958 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [19] has exited prematurely. Aborting benchmark.
2019-11-19 00:57:09,446 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [28] has exited prematurely. Aborting benchmark.
2019-11-19 00:57:10,199 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [17] has exited prematurely. Aborting benchmark.
2019-11-19 00:57:14,316 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [18] has exited prematurely. Aborting benchmark.
2019-11-19 00:58:00,329 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [8] has exited prematurely. Aborting benchmark.
2019-11-19 00:58:01,971 ActorAddr-(T|:24632)/PID:17497 esrally.actor ERROR Load generator [24] has exited prematurely. Aborting benchmark.
2019-11-19 00:58:09,131 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:24632)] (ignoring).
2019-11-19 00:58:09,133 -not-actor-/PID:17486 esrally.racecontrol ERROR A benchmark failure has occurred
2019-11-19 00:58:09,133 -not-actor-/PID:17486 esrally.racecontrol INFO Telling benchmark actor to exit.
2019-11-19 00:58:09,132 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:24632)] and will forward it now.
2019-11-19 00:58:09,133 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:24632)] and will forward it now.
2019-11-19 00:58:09,133 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:24632)] and will forward it now.
2019-11-19 00:58:09,134 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:24632)] and will forward it now.
2019-11-19 00:58:09,135 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO BenchmarkActor received unknown message [ActorExitRequest] (ignoring).
2019-11-19 00:58:09,136 ActorAddr-(T|:11776)/PID:17495 esrally.actor INFO MechanicActor#receiveMessage unrecognized(msg = [<class 'thespian.actors.ActorExitRequest'>] sender = [ActorAddr-(T|:2378)])
2019-11-19 00:58:09,137 ActorAddr-(T|:28983)/PID:17496 esrally.actor INFO Stopping nodes [search-es-0-79f54c7d49-ckvjj] (due to ActorExitRequest)
2019-11-19 00:58:09,138 ActorAddr-(T|:11776)/PID:17495 esrally.actor ERROR Child actor exited with [ChildActorExited:ActorAddr-(T|:28983)] while in status [benchmark_started].
2019-11-19 00:58:09,139 ActorAddr-(T|:2378)/PID:17494 esrally.actor INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:11776)] (ignoring).
2019-11-19 00:58:12,138 -not-actor-/PID:17486 esrally.rally INFO Attempting to shutdown internal actor system.
2019-11-19 00:58:12,139 -not-actor-/PID:17493 root INFO ActorSystem Logging Shutdown
2019-11-19 00:58:12,160 -not-actor-/PID:17492 root INFO ---- Actor System shutdown
2019-11-19 00:58:12,160 -not-actor-/PID:17486 esrally.rally INFO Actor system is still running. Waiting...
2019-11-19 00:58:13,161 -not-actor-/PID:17486 esrally.rally INFO Shutdown completed.
2019-11-19 00:58:13,161 -not-actor-/PID:17486 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/esrally/rally.py", line 515, in dispatch_sub_command
    race(cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/rally.py", line 433, in race
    with_actor_system(lambda c: racecontrol.run(c), cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/rally.py", line 461, in with_actor_system
    runnable(cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/rally.py", line 433, in <lambda>
    with_actor_system(lambda c: racecontrol.run(c), cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/racecontrol.py", line 369, in run
    raise e
  File "/usr/local/lib/python3.5/dist-packages/esrally/racecontrol.py", line 366, in run
    pipeline(cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/racecontrol.py", line 78, in __call__
    self.target(cfg)
  File "/usr/local/lib/python3.5/dist-packages/esrally/racecontrol.py", line 302, in benchmark_only
    return race(cfg, external=True)
  File "/usr/local/lib/python3.5/dist-packages/esrally/racecontrol.py", line 254, in race
    raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: ('Load generator [14] has exited prematurely.', None)