Emin Orhan eminorhan

## printarr
Pretty print tables summarizing properties of tensor arrays in numpy, pytorch, jax, etc.

Now on pip! `pip install arrgh` https://github.com/nmwsharp/arrgh

## specific_gpt.py
"""Stream a response from the OpenAI completion API."""
import os
import re
import sys
import time
import random

import openai
openai.api_key = open(os.path.expanduser("~/.openai")).read().strip()

## ddp_notes.md

      
              1 file
            
          
              25 forks
            
          
              18 comments
            
          
              177 stars
            
          
                TengdaHan
                / ddp_notes.md
            
            
              Last active
              May 21, 2024 07:02
            
              
                Multi-node-training on slurm with PyTorch
              
          
    Multi-node-training on slurm with PyTorch

What's this?


A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated,
or you need more than 4 GPUs for a single job.
Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
Warning: might need to re-factor your own code.
Warning: might be secretly condemned by your colleagues because using too many GPUs.
	Pretty print tables summarizing properties of tensor arrays in numpy, pytorch, jax, etc.

	Now on pip! `pip install arrgh` https://github.com/nmwsharp/arrgh
	"""Stream a response from the OpenAI completion API."""
	import os
	import re
	import sys
	import time
	import random

	import openai
	openai.api_key = open(os.path.expanduser("~/.openai")).read().strip()