GPU usage for different ways of initialization of models

Hi, I am processing high-resolution images on GPU with limited memory.So I want to reduce the GPU memory consumption as much as possible. I am wondering which way to initialize model is better. One way is to put all the modules in one list and then use nn.Sequential(modules). The other way is to use self.conv1 and self.conv2 …So every component have their own module. I am wondering which way is better and how much difference it is gonna make for GPU consumption here?

Thank you so much and best regards

You won’t save any memory if you prefer one approach to the other. While the latter might give you more flexibility in creating the model, both will yield the same result and use the same amount of memory in case you have a simple sequential architecture.

Have a look at torch.utils.checkpoint to add checkpointing to your model and save some memory.