Skip to content

Instantly share code, notes, and snippets.

Avatar
🌳
wash warm, tumble dry

Paco Nathan ceteri

🌳
wash warm, tumble dry
View GitHub Profile
@ceteri
ceteri / pycharm.errors.txt
Created Jul 13, 2020
PyCharm errors when attempting `pip install ray`
View pycharm.errors.txt
vm is not compatible with the npm config "prefix" option: currently set to "/usr/local"
Run `npm config delete prefix` or `nvm use --delete-prefix v8.11.2 --silent` to unset it.
(venv) (base) derwen:~/src$ pip install ray --no-binary multidict
Collecting ray
Using cached ray-0.8.6-cp37-cp37m-macosx_10_13_intel.whl (53.4 MB)
Processing /Users/paco/Library/Caches/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653/PyYAML-5.3.1-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting google
Using cached google-3.0.0-py2.py3-none-any.whl (45 kB)
Requirement already satisfied: numpy>=1.16 in /Users/paco/venv/lib/python3.7/site-packages (from ray) (1.19.0)
Collecting colorama
View rl28.txt
1 reward 0.00/ 0.02/ 1.00 len 7.83 saved tmp/ppo/froz/checkpoint_1/checkpoint-1
2 reward 0.00/ 0.02/ 1.00 len 7.40 saved tmp/ppo/froz/checkpoint_2/checkpoint-2
3 reward 0.00/ 0.02/ 1.00 len 7.21 saved tmp/ppo/froz/checkpoint_3/checkpoint-3
4 reward 0.00/ 0.03/ 1.00 len 7.36 saved tmp/ppo/froz/checkpoint_4/checkpoint-4
5 reward 0.00/ 0.03/ 1.00 len 7.26 saved tmp/ppo/froz/checkpoint_5/checkpoint-5
6 reward 0.00/ 0.05/ 1.00 len 7.57 saved tmp/ppo/froz/checkpoint_6/checkpoint-6
7 reward 0.00/ 0.05/ 1.00 len 7.82 saved tmp/ppo/froz/checkpoint_7/checkpoint-7
8 reward 0.00/ 0.07/ 1.00 len 7.42 saved tmp/ppo/froz/checkpoint_8/checkpoint-8
9 reward 0.00/ 0.07/ 1.00 len 7.87 saved tmp/ppo/froz/checkpoint_9/checkpoint-9
10 reward 0.00/ 0.09/ 1.00 len 8.84 saved tmp/ppo/froz/checkpoint_10/checkpoint-10
View rl27.txt
1 reward -902.00/-751.75/-345.00 len 194.80 saved tmp/ppo/taxi/checkpoint_1/checkpoint-1
2 reward -902.00/-751.85/-345.00 len 193.70 saved tmp/ppo/taxi/checkpoint_2/checkpoint-2
3 reward -902.00/-725.72/-340.00 len 193.00 saved tmp/ppo/taxi/checkpoint_3/checkpoint-3
4 reward -902.00/-705.04/-151.00 len 192.59 saved tmp/ppo/taxi/checkpoint_4/checkpoint-4
5 reward -902.00/-682.85/-151.00 len 192.62 saved tmp/ppo/taxi/checkpoint_5/checkpoint-5
6 reward -902.00/-643.69/-128.00 len 190.27 saved tmp/ppo/taxi/checkpoint_6/checkpoint-6
7 reward -902.00/-585.58/-78.00 len 185.95 saved tmp/ppo/taxi/checkpoint_7/checkpoint-7
8 reward -794.00/-524.43/-21.00 len 176.76 saved tmp/ppo/taxi/checkpoint_8/checkpoint-8
9 reward -794.00/-482.32/-21.00 len 172.15 saved tmp/ppo/taxi/checkpoint_9/checkpoint-9
10 reward -713.00/-443.42/-21.00 len 166.61 saved tmp/ppo/taxi/checkpoint_10/checkpoint-10
View rl26.txt
1 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_1/checkpoint-1
2 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_2/checkpoint-2
3 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_3/checkpoint-3
4 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_4/checkpoint-4
5 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_5/checkpoint-5
6 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_6/checkpoint-6
7 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_7/checkpoint-7
8 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_8/checkpoint-8
9 reward -200.00/-200.00/-200.00 len 200.00 saved tmp/ppo/moun/checkpoint_9/checkpoint-9
View rl25.txt
1 reward 9.00/ 22.65/ 63.00 len 22.65 saved tmp/ppo/cart/checkpoint_1/checkpoint-1
2 reward 12.00/ 42.72/151.00 len 42.72 saved tmp/ppo/cart/checkpoint_2/checkpoint-2
3 reward 12.00/ 68.17/322.00 len 68.17 saved tmp/ppo/cart/checkpoint_3/checkpoint-3
4 reward 13.00/ 97.87/408.00 len 97.87 saved tmp/ppo/cart/checkpoint_4/checkpoint-4
5 reward 13.00/131.53/500.00 len 131.53 saved tmp/ppo/cart/checkpoint_5/checkpoint-5
6 reward 13.00/165.24/500.00 len 165.24 saved tmp/ppo/cart/checkpoint_6/checkpoint-6
7 reward 13.00/202.48/500.00 len 202.48 saved tmp/ppo/cart/checkpoint_7/checkpoint-7
8 reward 22.00/233.83/500.00 len 233.83 saved tmp/ppo/cart/checkpoint_8/checkpoint-8
9 reward 22.00/271.82/500.00 len 271.82 saved tmp/ppo/cart/checkpoint_9/checkpoint-9
10 reward 22.00/302.99/500.00 len 302.99 saved tmp/ppo/cart/checkpoint_10/checkpoint-10
View rl24.sh
rllib rollout \
 tmp/ppo/moun/checkpoint_40/checkpoint-40 \
 - config "{\"env\": \"MountainCar-v0\"}" \
 - run PPO \
 - steps 2000
View rl23.txt
_____________________________________________________________________________
Layer (type) Output Shape Param # Connected to
=============================================================================
observations (InputLayer) [(None, 2)] 0 
_____________________________________________________________________________
fc_1 (Dense) (None, 256) 768 observations[0][0]
_____________________________________________________________________________
fc_value_1 (Dense) (None, 256) 768 observations[0][0]
_____________________________________________________________________________
fc_2 (Dense) (None, 256) 65792 fc_1[0][0] 
View rl22.py
SELECT_ENV = "MountainCar-v0"
config = ppo.DEFAULT_CONFIG.copy()
config["log_level"] = "WARN"
config["num_workers"] = 4 # default = 2
config["train_batch_size"] = 10000 # default = 4000
config["sgd_minibatch_size"] = 256 # default = 128
config["evaluation_num_episodes"] = 50 # default = 10
View rl21.py
CHECKPOINT_ROOT = "tmp/ppo/moun"
shutil.rmtree(CHECKPOINT_ROOT, ignore_errors=True, onerror=None)
ray_results = os.getenv("HOME") + "/ray_results/"
shutil.rmtree(ray_results, ignore_errors=True, onerror=None)
View rl20.sh
rllib rollout \
 tmp/ppo/cart/checkpoint_40/checkpoint-40 \
 - config "{\"env\": \"CartPole-v1\"}" \
 - run PPO \
 - steps 2000
You can’t perform that action at this time.