joshlk

joshlk

NVIDIA GPU P2P Benchmark bandwidth/throughput and latency

Using https://github.com/NVIDIA/cuda-samples

You can also view the GPU topology using nvidia-smi topo -m

1. Download repo git clone https://github.com/NVIDIA/cuda-samples.git
2. Checkout the tag that corresponds with the right CUDA version: git checkout tags/v11.1
3. You might need to install some additional packages sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev libglfw3-dev libgles2-mesa-dev
4. Either build everything by just execting make in root dir. Or cd Samples/p2pBandwidthLatencyTest; make

joshlk

joshlk

from math import ceil
from typing import Optional, Tuple, Union

import torch
from torch import Tensor, nn
from torch.distributed._tensor import DTensor, Replicate, Shard, distribute_module, distribute_tensor
from torch.distributed._tensor.op_schema import OpSchema, OutputSharding
from torch.distributed._tensor.ops.embedding_ops import embedding_rules
from torch.distributed._tensor.ops.utils import register_prop_rule
from torch.distributed._tensor.placement_types import DTensorSpec, _Partial

joshlk

joshlk

#!/usr/bin/env bash
# peak_memory [command ...]
# Run the given command line in the background and kill if script exits
trap 'kill $(jobs -p) 2&> /dev/null' EXIT
"$@" & 
pid=$! phys_peak=0 not_avail_peak=0
# while command still running
while ps -p $pid &>/dev/null; do
  sleep 1
  mem_total="$(cat /proc/meminfo | grep  'MemTotal:' | grep -oe '\([0-9.]*\)')"

joshlk

import trio
import tqdm

class TrioProgress(trio.abc.Instrument):

    def __init__(self, total, notebook_mode=False, **kwargs):
        if notebook_mode:
            from tqdm import tqdm_notebook as tqdm
        else:
            from tqdm import tqdm

joshlk

import pandas as pd

def _map_to_pandas(rdds):
    """ Needs to be here due to pickling issues """
    return [pd.DataFrame(list(rdds))]

def toPandas(df, n_partitions=None):
    """
    Returns the contents of `df` as a local `pandas.DataFrame` in a speedy fashion. The DataFrame is
    repartitioned if `n_partitions` is passed.

joshlk

1. Install Cannon printer drivers sudo apt update; sudo apt install software-properties-common cups-backend-bjnp
2. Make sure user has password (add password using sudo passwd $user)
3. Follow instructions to install server https://pimylifeup.com/raspberry-pi-print-server/
4. Chrome doesn't work to add a printer - worked on Safari

joshlk