HorizonML explores decentralized model training by implementing and benchmarking data-parallel, layer-parallel, and tensor-parallel strategies across simulated nodes. The project aims to challenge centralized AI training pipelines by laying the groundwork for a modular, trustless training framework.
https://github.com/Deeptanshu-sankhwar/horizonml/blob/main/assets/HorizonML.pdf
https://www.loom.com/share/cfda4a569b7049ab82cf2999bc885cdc?sid=04d39fc5-da1f-4c10-8d5e-6eccbb066c54
http://github.com/Deeptanshu-sankhwar/horizonml
https://github.com/Deeptanshu-sankhwar/horizonml/tree/main/benchmark_results/all