mehdidc

#!/bin/bash -x
#SBATCH --account={account}
#SBATCH --nodes={nodes}
#SBATCH --gres=gpu:4
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=24
#SBATCH --time=06:00:00
#SBATCH --partition={partition}
#SBATCH --output={output_file}
echo "Job Id:$SLURM_JOB_ID"

mehdidc

job_id_regexp: "Job Id:(\\d+)"
cmd: "sbatch {sbatch_script}"
check_interval_secs: 600
partition: booster
account: laionize
experiments:
  small:
    model_scale:
        model: [ViT-B-32]
    samples_seen_scale:        

mehdidc

"""
Implementation of 'Maximum Likelihood Estimation of Intrinsic Dimension' by Elizaveta Levina and Peter J. Bickel

how to use
----------

The goal is to estimate intrinsic dimensionality of data, the estimation of dimensionality is scale dependent
(depending on how much you zoom into the data distribution you can find different dimesionality), so they
propose to average it over different scales, the interval of the scales [k1, k2] are the only parameters of the algorithm.

mehdidc

Disable Laptop Keyboard in Ubuntu

Taken From :- https://blog.hostonnet.com/laptop-keyboard-ubuntu

To disable laptop built in keyboard in Ubuntu, first you need to identify the device id for keyboard.

Now run xinput –list command to get list of all input devices

hostonnet@pc4:~$ xinput --list
⎡ Virtual core pointer                      id=2    [master pointer  (3)]

mehdidc

import ast
import json
import logging
import math
import os
import random
import sys
import braceexpand
from dataclasses import dataclass
from multiprocessing import Value

mehdidc

language,acc1,model,mean_per_class_recall,dataset,pretrained,image_retrieval_recall@5,acc,mean_average_precision,acc5,task,model_fullname,text_retrieval_recall@5
en,0.18418107200596942,RN50-quickgelu,0.18626162135734856,cars,cc12m,,,,0.5200845665961945,zeroshot_classification,RN50-quickgelu cc12m,
en,0.17945529163039423,RN50,0.1817410995349502,cars,cc12m,,,,0.5139907971645318,zeroshot_classification,RN50 cc12m,
en,0.7211789578410646,ViT-B-16,0.7212942008721658,cars,commonpool_l_basic_s1b_b8k,,,,0.9759980101977366,zeroshot_classification,ViT-B-16 commonpool_l_basic_s1b_b8k,
en,0.8166894664842681,ViT-B-16,0.8155803538003477,cars,commonpool_l_clip_s1b_b8k,,,,0.9907971645317747,zeroshot_classification,ViT-B-16 commonpool_l_clip_s1b_b8k,
en,0.6628528789951499,ViT-B-16,0.6613039344249736,cars,commonpool_l_image_s1b_b8k,,,,0.9621937569953986,zeroshot_classification,ViT-B-16 commonpool_l_image_s1b_b8k,
en,0.6775276706877255,ViT-B-16,0.6744836925516351,cars,commonpool_l_laion_s1b_b8k,,,,0.9602039547319985,zeroshot_cla

mehdidc

import sys
fd = open(sys.argv[2], "w")
lines = open(sys.argv[1]).readlines()
for l in lines:
    toks = l.split(" ")
    t = toks[0]
    image_id = t.split("#")[0]
    caption = " ".join(toks[1:])
    L = f"{image_id}.jpg,{caption}"
    fd.write(L)

mehdidc

import io
import tarfile
import random
from collections import defaultdict
from lxml import etree
import uuid
from PIL import Image, ImageDraw
from glob import glob
import time
import os

mehdidc

(Internal Tranining Material)

Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:

• How to find the bottleneck operator?
• How to trace source file of a particular operator?
• How do I indentify threading issues? (oversubscription)
• How do I tell a specific operator is running efficiently or not?

This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.

mehdidc

model_fullname,model_fullname_pretty,model_arch,samples_seen,gmacs_per_sample,gmacs_total,upstream_dataset,downstream_dataset,acc1,acc5,mean_per_class_recall,image_retrieval_recall@5,text_retrieval_recall@5
ViT-g-14 /fsx/rom1504/open_clip/good_models/g_90.pt,g/14 2B,ViT-g-14,12208147020,290.74,3549396664594.8003,LAION-2B,vtab+,0.5654112282297443,0.8329414582676622,0.56279878057792,,
ViT-g-14 /fsx/rom1504/open_clip/good_models/g_90.pt,g/14 2B,ViT-g-14,12208147020,290.74,3549396664594.8003,LAION-2B,vtab/caltech101,0.8522353714661407,0.963346482577252,0.944284654839904,,
ViT-g-14 /fsx/rom1504/open_clip/good_models/g_90.pt,g/14 2B,ViT-g-14,12208147020,290.74,3549396664594.8003,LAION-2B,imagenet1k,0.76664,0.9485,0.76656,,
ViT-g-14 /fsx/rom1504/open_clip/good_models/g_90.pt,g/14 2B,ViT-g-14,12208147020,290.74,3549396664594.8003,LAION-2B,vtab/cifar100,0.8391,0.9729,0.8388,,
ViT-g-14 /fsx/rom1504/open_clip/good_models/g_90.pt,g/14 2B,ViT-g-14,12208147020,290.74,3549396664594.8003,LAION-2B,imagenetv2,0.6961,0.9086,0.6