Skip to content

Instantly share code, notes, and snippets.

View toanbku's full-sized avatar
:fishsticks:
Try hard

Toan Ho toanbku

:fishsticks:
Try hard
16 followers · 9 following
View GitHub Profile
@toanbku
toanbku / renameKey.ts
Last active May 16, 2024 02:57
Rename key of object in TypeScript while keeping the object immutable
export function renameKey<T extends object, K extends keyof T, N extends string>(
obj: T,
oldProp: K,
newProp: N
): Omit<T, K> & { [P in N]: T[K] } {
const { [oldProp]: oldValue, ...rest } = obj;
return {
...rest,
[newProp]: oldValue,
} as Omit<T, K> & { [P in N]: T[K] };
@toanbku
toanbku / gist:e7f6620ce7146e8847751fbf9927065f
Created July 25, 2023 01:53
(base) [RedmondAI] ubuntu@oa-server-8:~/OA/model/model_training$ CUDA_VISIBLE_DEVICES=0 OMP_NUM_THREADS=1 accelerate launch --main_process_port 29501 --config_file configs/accelerate_config.yaml --num_processes 1 trainer_rl.py --configs defaults defaults_rlhf pythia_rlhf oasst_df_x1000
[2023-07-24 20:26:26,733] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-24 20:26:30,673] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
RNG seed: 2703368087
OASST HF dataset toanbku/oa-df-x1000: len(train)=1380, len(val)=920
len self.tokenizer 50282
[RANK 0] Initializing model: toanbku/oa-pythia-12b-sft-df
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████| 3/3 [00:17<00:00, 5.85s/it]
Resizing embeddings to 50282
Number of trainable parameters: 11841M
@toanbku
toanbku / gist:760b3c71e5ae44364e52769995b2f2d4
Created July 21, 2023 03:54
(base) [RedmondAI] ubuntu@oa-server-8:~/OA/model/model_training$ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 OMP_NUM_THREADS=1 accelerate launch --main_process_port 29501 --config_file configs/accelerate_config.yaml --num_processes 6 trainer_rl.py --configs defaults defaults_rlhf pythia_rlhf oasst_df_x1000
[2023-07-21 03:19:15,948] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-21 03:19:19,845] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-21 03:19:19,845] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-21 03:19:19,864] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-21 03:19:19,865] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-21 03:19:19,873] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-0
@toanbku
toanbku / gist:538d4482640923d179aad2e5acd41976
Created July 18, 2023 15:43
(cuda118) [RedmondAI] ubuntu@oa-server-4:~/OA/model/model_training$ CUDA_VISIBLE_DEVICES=1 OMP_NUM_THREADS=1 accelerate launch --main_process_port 29501 --config_file configs/accelerate_config.yaml --num_processes 1 trainer_rl.py --configs defaults defaults_rlhf pythia_rlhf oa_df
[2023-07-18 15:22:12,144] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-07-18 15:22:15,570] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
RNG seed: 2703368087
Found cached dataset json (/home/ubuntu/.cache/huggingface/datasets/toanbku___json/toanbku--oa-df-811abf2c8473a2c5/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
OASST HF dataset toanbku/oa-df: len(train)=19, len(val)=4
len self.tokenizer 50282
[2023-07-18 15:22:19,161] [WARNING] [comm.py:152:init_deepspeed_backend] NCCL backend in DeepSpeed not yet implemented
[2023-07-18 15:22:19,161] [INFO] [comm.py:594:init_distributed] cdb=None
[2023-07-18 15:22:19
@toanbku
toanbku / gist:72fbe26f9a9e4d0b2029aa5f086944ed
Last active June 15, 2022 07:51
Use system font on Discord (MacOS)