wookim3/ddp3.py

Created October 19, 2020 14:30

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/wookim3/9f3d44d92683f3785416c9162234dc2f.js"></script>
Save wookim3/9f3d44d92683f3785416c9162234dc2f to your computer and use it in GitHub Desktop.

Download ZIP

Raw

ddp3.py

	model = torch.nn.parallel.DistributedDataParallel(
	model,
	device_ids=[rank],
	output_device=rank,
	gradient_as_bucket_view=True
	)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment