TengdaHan

Multi-node-training on slurm with PyTorch

What's this?

• A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
• Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
• Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
• Warning: might need to re-factor your own code.
• Warning: might be secretly condemned by your colleagues because using too many GPUs.

mingfeima

(Internal Tranining Material)

Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:

• How to find the bottleneck operator?
• How to trace source file of a particular operator?
• How do I indentify threading issues? (oversubscription)
• How do I tell a specific operator is running efficiently or not?

This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.

YashasSamaga

g++ -I/usr/local/include/opencv4/ benchmark.cpp -lopencv_core -lopencv_imgproc -lopencv_dnn -lopencv_imgcodecs -O3 -std=c++17

zhensongren

To list all python versions in default locations

ls /usr/bin/python*

To remove just python3 package

sudo apt-get remove python3.5

plus it's dependent packages

sudo apt-get remove --auto-remove python3.5

plus configuration and/or data files of python3

sudo apt-get purge python3.5

simecek

c-bata

name: release
on:
  push:
    branches:
    - master

jobs:
  release:
    name: Build
    runs-on: ubuntu-latest

endolith

raulqf

How to install OpenCV 4.5.2 with CUDA 11.2 and CUDNN 8.2 in Ubuntu 22.04

First of all install update and upgrade your system:

    $ sudo apt update
    $ sudo apt upgrade

Then, install required libraries:

bernardomig

How to setup nvidia-container-runtime and podman/runc

Podman is such a cool project! However, there is no easy way to setup the nvidia-container-runtime and podman so that we can run unprivileged container in a gpu host. This is specially interesting in environments with multiple people accessing the same host (strong isolation between containers!!!).

Steps to setup the whole system

1. Install podman and friends (buildah and skopeo)

Ubuntu: add-apt-repository -y ppa:projectatomic/ppa && apt install podman buildah skopeo

dayne

ssh-agent checker/launcher

about

A bash script for managing ssh-agent that I use for Ubuntu and OSX.

Started as a variation from this classic on Stack Overflow How to check if ssh-agent is already running and then adapted it to align with behaivor I want.

This version does the following:

• checks for ssh agent forward (remote login) and if so does not setup ssh-agent or any keys