DataParallel hang on machine with two K80 on board

[QuantScientist (Solomon K)]

See:

How to use DataParallel in backward?

if you notice the examples, DataParallel is not applied to the entire network + loss. It is only applied to part of the network. before adding DataParallel: network = features (conv layers) -> classifier (linear layers) error = loss_function(network(input), target) error.backward() After adding DataParallel: network = DataParallel(features (conv layers)) -> classifier (linear layers) error = loss_function(network(input), target) error.backward()