In the past, I have been using Theano where I could select the convolution strategy (see: Theano convolution strategies.). When I reimplement my model in PyTorch the memory usage is significantly higher, suggesting the convolution strategy is different.
Is there a way to determine which of these strategies is selected by the benchmark tool? We have tried that benchmark, but our application requires a lot of memory and in the first convolutional layer we already run out of memory. A similar implementation in Theano did not have this problem, which is why we want to figure out which strategy is used.
Where can we find how the benchmark algorithm works? Does it also take memory into account or just computational efficiency? Thanks for the effort.