| # This is a modified version of TRL's `SFTTrainer` example (https://github.com/huggingface/trl/blob/main/examples/scripts/sft_trainer.py), | |
| # adapted to run with DeepSpeed ZeRO-3 and Mistral-7B-V1.0. The settings below were run on 1 node of 8 x A100 (80GB) GPUs. | |
| # | |
| # Usage: | |
| # - Install the latest transformers & accelerate versions: `pip install -U transformers accelerate` | |
| # - Install deepspeed: `pip install deepspeed==0.9.5` | |
| # - Install TRL from main: pip install git+https://github.com/huggingface/trl.git | |
| # - Clone the repo: git clone github.com/huggingface/trl.git | |
| # - Copy this Gist into trl/examples/scripts | |
| # - Run from root of trl repo with: accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero3.yaml --gradient_accumulation_steps 8 examples/scripts/sft_trainer.py |
This guide likely applies to other models and, potentially, even laptops from other OEMs that have NVME drives. However, I've only tested this on my Dell XPS 15 (9560) with the OEM Windows installation from the Signature Edition model.
Switching from RAID to AHCI is significantly simpler than switching from AHCI to RAID. All that's needed is a successful boot to Safe Mode.
msconfig.exe or open an admin cmd/PowerShell window and run:
React Fiber is an ongoing reimplementation of React's core algorithm. It is the culmination of over two years of research by the React team.