Skip to content

Instantly share code, notes, and snippets.

View MustafaAlahmid's full-sized avatar
Focusing on Bitcoin

Mustafa Faris MustafaAlahmid

Focusing on Bitcoin
View GitHub Profile
@MustafaAlahmid
MustafaAlahmid / sft_trainer.py
Created December 17, 2023 20:03 — forked from lewtun/sft_trainer.py
Fine-tuning Mistral 7B with TRL & DeepSpeed ZeRO-3
# This is a modified version of TRL's `SFTTrainer` example (https://github.com/huggingface/trl/blob/main/examples/scripts/sft_trainer.py),
# adapted to run with DeepSpeed ZeRO-3 and Mistral-7B-V1.0. The settings below were run on 1 node of 8 x A100 (80GB) GPUs.
#
# Usage:
# - Install the latest transformers & accelerate versions: `pip install -U transformers accelerate`
# - Install deepspeed: `pip install deepspeed==0.9.5`
# - Install TRL from main: pip install git+https://github.com/huggingface/trl.git
# - Clone the repo: git clone github.com/huggingface/trl.git
# - Copy this Gist into trl/examples/scripts
# - Run from root of trl repo with: accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero3.yaml --gradient_accumulation_steps 8 examples/scripts/sft_trainer.py
@MustafaAlahmid
MustafaAlahmid / Udemy...MyContract.sol
Created April 1, 2021 16:18
Created using remix-ide: Realtime Ethereum Contract Compiler and Runtime. Load this file by pasting this gists URL or ID at https://remix.ethereum.org/#version=soljson-v0.6.0+commit.26b70077.js&optimize=false&runs=200&gist=
pragma solidity 0.6.0;
contract myContract{
string public myString = "Hello Blockchain";
}
@MustafaAlahmid
MustafaAlahmid / Udemy...MyContract.sol
Created April 1, 2021 10:16
Created using remix-ide: Realtime Ethereum Contract Compiler and Runtime. Load this file by pasting this gists URL or ID at https://remix.ethereum.org/#version=soljson-v0.6.0+commit.26b70077.js&optimize=false&runs=200&gist=
pragma solidity 0.6.0;
contract myContract{
string public myString = "Hello Blockchain";
}