crazyguitar

import cPickle
import pickle
import json
import random
from time import time
from hashlib import md5

test_runs = 1000

def float_list():

crazyguitar

Ordinary launch commands (no profiling):

Single-process:

python main_amp.py -a resnet50 --b 224 --deterministic --workers 4 --opt-level O1 ./bare_metal_train_val/

Multi-process:

python -m torch.distributed.launch  --nproc_per_node=2 main_amp.py -a resnet50 --b 224 --deterministic --workers 4 --opt-level O1 ./bare_metal_train_val/

crazyguitar

Often, there's a misconception about the Global Interpreter Lock (GIL) in Python, where it's believed that the Python interpreter only permits one thread to run at any given time. Infact, Python Interpreter allows threads running in parallel when GIL is released in a function. For instance, functions like time.sleep leverage I/O blocking within Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS, enabling the Python interpreter to handle multiple threads simultaneously in such cases.

reference:

1. pysleep

The provided reference demonstrates how Python allows sleep operations to run in parallel:

crazyguitar

Installing CUDA 12.1.1 + PyTorch nightly + Python 3.10 on Ubuntu 22.10

Should you keep your NVIDIA driver?

CUDA 12.1.1 toolkit is gonna offer to install Nvidia driver 530 for us. It's from New Feature branch. It's likely to be newer than the default Nvidia driver you would've installed via apt-get (apt would prefer to give you 525, i.e. Production Branch).

If you're confident that you already have a new enough Nvidia driver for CUDA 12.1.1, and you'd like to keep your driver: feel free to skip this "uninstall driver" step.

But if you're not sure, or you know your driver is too old: let's uninstall it. CUDA will install a new driver for us later.

crazyguitar

#include <rdma/fabric.h>
#include <rdma/fabric.h>
#include <rdma/fi_endpoint.h>
#include <rdma/fi_cm.h>
#include <rdma/fi_errno.h>
#include <rdma/fi_rma.h>

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

crazyguitar

• RDMA Aware Networks Programming User Manual
• On the Impact of Cluster Configuration on RoCE Application Design
• RDMA over Commodity Ethernet at Scale
• Design Guidelines for High Performance RDMA Systems
• FaSST: Fast, Scalable and Simple Distributed Transactions with Two-sided (RDMA) Datagram RPCs
• RDMA [1]: A short history of remote DMA networking
• [Slide] RDMA Tutorial
• Understanding the concepts and mechanisms of RDMA
• [InfiniBand RDMA over PCI Express Networ

crazyguitar

# files/env:
Defaults env_keep += "SSH_AUTH_SOCK"

# tasks/main.yml
- name: ensure sudo keeps SSH_AUTH_SOCK in environment
  copy: src=env
        dest=/etc/sudoers.d/env
        mode=0440
        owner=root
        group=root

crazyguitar

Using the ssh agent

The following would show a similar output if a key is already added to your agent, and you are good to go.

$ ssh-add -l
2048 d7:8e:3d:03:9c:4f:f8:9d:04:0f:11:c5:24:e1:2f:3a rsa w/o comment (RSA)

The following will show if no agent is running.

crazyguitar

SSH Forwarding on MacOS X

Forwarding SSH agent

Configure ~/.ssh/config

# Allow Agent forwarding for a specific host (by security)
Host remotehost
  ForwardAgent yes

crazyguitar

Changing playback sound speed in realtime with python and multiprocessing

This is a script to change the sound speed in real time like for example background music in pygame.

The script generates the raw data of a sound file each time there is space in the queue. The sample will scale to the desired speed which you can modify with sound_Factor.

To play the music, you just need to get buffers from the queue and pass them to your music stream. Exemple for pygame :

# musique update
if channel.get_queue() == None: