Skip to content

Instantly share code, notes, and snippets.

@nesou2
Created July 17, 2019 10:19
Show Gist options
  • Save nesou2/6ce985283c3e460c64b000951edbd066 to your computer and use it in GitHub Desktop.
Save nesou2/6ce985283c3e460c64b000951edbd066 to your computer and use it in GitHub Desktop.
2019-07-17 11:17:39,061 INFO function_runner.py:255 -- tune.track signature detected.
2019-07-17 11:17:39,065 INFO tune.py:61 -- Tip: to resume incomplete experiments, pass resume='prompt' or resume=True to run()
2019-07-17 11:17:39,065 INFO tune.py:233 -- Starting a new experiment.
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 0/8 CPUs, 0/0 GPUs
Memory usage on this node: 13.8/16.7 GB
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 1/8 CPUs, 0/0 GPUs
Memory usage on this node: 13.8/16.7 GB
Result logdir: /home/sebastian/ray_results/train_mnist
Number of trials: 1 ({'RUNNING': 1})
RUNNING trials:
- train_mnist_0_lr=0.21459,momentum=0.26263: RUNNING
2019-07-17 11:17:46,068 ERROR trial_runner.py:487 -- Error processing event.
Traceback (most recent call last):
File "/home/sebastian/anaconda3/envs/mlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 447, in _process_trial
self._total_time += result[TIME_THIS_ITER_S]
KeyError: 'time_this_iter_s'
2019-07-17 11:17:46,071 INFO ray_trial_executor.py:187 -- Destroying actor for trial train_mnist_0_lr=0.21459,momentum=0.26263. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-618be857c33b> in <module>
11
12 # Note: use `ray.init(redis_address=...)` to enable distributed execution
---> 13 analysis = tune.run(train_mnist, config=search_space, **experiment_config)
~/anaconda3/envs/mlearning/lib/python3.7/site-packages/ray/tune/tune.py in run(run_or_experiment, name, stop, config, resources_per_trial, num_samples, local_dir, upload_dir, trial_name_creator, loggers, sync_function, checkpoint_freq, checkpoint_at_end, export_formats, max_failures, restore, search_alg, scheduler, with_server, server_port, verbose, resume, queue_trials, reuse_actors, trial_executor, raise_on_failed_trial, return_trials, ray_auto_init)
256 if time.time() - last_debug > DEBUG_PRINT_INTERVAL:
257 if verbose:
--> 258 print(runner.debug_string())
259 last_debug = time.time()
260
~/anaconda3/envs/mlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py in debug_string(self, max_debug)
379 for t in sorted_trials:
380 messages.append(" - {}:\t{}".format(
--> 381 t, t.progress_string()))
382
383 return "\n".join(messages) + "\n"
~/anaconda3/envs/mlearning/lib/python3.7/site-packages/ray/tune/trial.py in progress_string(self)
474 self.last_result.get(HOSTNAME),
475 self.last_result.get(PID))), "{} s".format(
--> 476 int(self.last_result.get(TIME_TOTAL_S)))
477 ]
478
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment