mfitton

Dashboard Development Guide

Overview

The dashboard roughly consists of two parts, a front end single-page web-application build with Typescript and React, and a back-end web server built with Python. These two parts are located within the directory ray/dashboard.

You can install the dependencies of the dashboard by running npm install in the ray/dashboard/client directory, and generate a production build of the dashboard by running npm run build.

For general purpose development, the best setup is to use the hot-reloading server provided by the command npm run start in the ray/dashboard/client directory. This command will start the web application on port 3000 and recompile / reload front-end file changes as you make them. This also allows browser debuggers to be much more useful, as you can set breakpoints in your typescript code, as opposed to working with minified javascript in a production build.

In order to connect this development server up to a live running ray cluster, the cluster m

mfitton

"""
The purpose of this module is to show how one could implement data
caching using Ray. If you tried to use multiprocessing to implement
this from scratch, you would need to deal with race conditions, 
thread-safety, and sharing memory between multiple processes. With
Ray, you can write this cache roughly like you would write a
single-threaded Python cache.
"""
import time
import requests

mfitton

"""
The purpose of this module is to show how one could implement data
caching using Ray. If you tried to use multiprocessing to implement
this from scratch, you would need to deal with race conditions, 
thread-safety, and sharing memory between multiple processes. With
Ray, you can write this cache roughly like you would write a
single-threaded Python cache.
"""
import ray
import flask

mfitton

Things we need to add to API

Machine View

We’re missing the cpu_percent calculation for the CPU.tsx feature

We need to add errorCounts for each worker. It looks like currently we only give the count of errors for the whole node and not for each worker.

Same for logCounts

In addition to pulling in the new GPU monitoring code, we will also need to supply the full resource slots rather than a mapping from type of resource to quantity as we do now.

mfitton

Fixing the node removed from cluster bug

Repro Script

import ray
from ray.cluster_utils import Cluster

cluster = Cluster()
cluster.add_node()
cluster.add_node()

mfitton

mfitton

for i, x in enumerate(np.linspace(S[0], S[2], w)):
    for j, y in enumerate(np.linspace(S[1], S[3], h)):
        coords.append((i, j))
        next_task_xys.append((x, y))
        if len(next_task_xys) == CHUNK_SIZE:
            results.append(
                trace_rays_with_bounces.remote(next_task_xys)
            )
            next_task_xys = []
if next_task_xys:

mfitton

@ray.remote
def trace_rays_with_bounces(xys):
    results = []
    for (x, y) in xys:
        # ... Snipped, same code as before ...
        results.append(np.clip(col, 0, 1))
    return results

mfitton

for i, x in enumerate(np.linspace(S[0], S[2], w)):
    for j, y in enumerate(np.linspace(S[1], S[3], h)):
        ...
        # Loop through initial and secondary rays.
        while depth < depth_max:
            traced = trace_ray(rayO, rayD)

mfitton

for i, x in enumerate(np.linspace(S[0], S[2], w)):
    for j, y in enumerate(np.linspace(S[1], S[3], h)):
        ...
        # Loop through initial and secondary rays.
        while depth < depth_max:
            traced = trace_ray(rayO, rayD)