Set channels_last to be default in PyTorch

Is it possible to set memory_format=torch.channels_last in the notebook code?

No, you cannot set the memory format globally (as it wouldn’t make sense to create all intermediates/internal tensors in this format) and would need to call it on the model and the input tensors.

Yeah, thank you @ptrblck.

  1. Interesting decision, since I read recently (y-day) channel last format may have some faster execution on GPU and TensorFlow also uses this channel last format probable for speed but I don’t know if this is really a huge gain. Maybe just some 20ish percents.

  2. But you know other imaging libraries such as PIL (and numpy) use channel first format so maybe this is the reason.

The speedup depends on the actual model, the used precision, the device, and especially the used libraries.
While channels_last should give you a speedup for CNNs in mixed-precision, the actual performance depends on the kernel availability and the heuristics. E.g. if you are using the latest cuDNN (with the beta v8 API) you should see better performance in channels_last+amp for a variety of models compared to older releases. However, some configs such as grouped convs could still run into performance cliffs, so it’s unfortunately still not a clear decision.

The preference of the memory layout changes from time to time, so it really comes down to which format is now the “best” one :wink:

Yeah, I was reading y-day the Efficient PyTorch: Tensor Memory Format Matters and the conclusion is:

The Memory Layout of an input tensor can significantly impact a model’s running time. For Vision Models, prefer a Channels Last memory format to get the most out of your PyTorch models.

But how you can define the Vision Models nowadays when Transformer is probable today also a vision model.
:wink:

And now just one simple ask. If I work with input tensor in channel-last memory format do other tensors need to be channel-last as well?