I’m running a version of StarGAN where the ConvTranspose2d layers which up-sample the signal have been replaced by a resize convolution as described below. For some reason, the forwards pass through the Upsample layers are very slow in comparison to the rest of the network, which is surprising to me, considering that the process doesn’t depend on any matrices of parameters.
layers.append(nn.ConvTranspose2d(curr_dim, curr_dim//2, kernel_size=4, stride=2, padding=1, bias=False))
is replaced by
layers.append(nn.Upsample(scale_factor=2, mode='nearest'))
layers.append(nn.ReflectionPad2d(1))
layers.append(nn.Conv2d(curr_dim, curr_dim // 2, kernel_size=3, stride=1, padding=0, bias=False))
The first pass through the Upsample layer has an input size of (256, 256, 256) and takes roughly 9.5s.
The second pass has an input size of (128, 512, 512) and takes roughly 17.5s.
When the same up-sampling is performed by a script I wrote using NumPy, it takes 0.3s and 0.5s respectively.
A colleague suggested that this could be caused by the fact that a GPU is designed to process images with fewer channels than the signals processed in this network, but I find this unsatisfying as the rest of the layers in the network run in a fraction of a second. I’m wondering if anyone else has any thoughts as to the cause?