Hello!
I’m trying to understand why I’m seeing such a performance gap between my regular conv2d C implementation and torch.conv2d on CPU.
A single torch.conv2d call with a 512x512 matrix and a 256x256 kernel as arguments takes 59.159 secs to execute whereas a very basic C implementation runs in only 0.650 sec !
I expected it to be the other way around but no: the regular C implementation is about 91 times faster despite running on a single core and having no particular optimization (I only rearranged the for loops to improve locality and compiled with -O2 -ffastmath).
What are the causes of such a performance gap on such a small example?
For reference, here’s the code:
- used to benchmark
torch.conv2d
import torch
import time
x = torch.rand(1, 1, 512, 512)
k = torch.rand(1, 1, 256, 256)
with torch.inference_mode():
start = time.time()
torch.conv2d(x, k)
end = time.time()
print(end - start)
- used to benchmark my implementation (compiled with
-O0)
#include <stdio.h>
#include <time.h>
int main(void) {
struct Tensor *a = tensor_init_random(512, 512);
struct Tensor *b = tensor_init_random(256, 256);
struct Tensor *c;
clock_t start, end;
start = clock();
c = conv2d(a, b, 0, 1);
end = clock();
tensor_free(c);
printf("%f\n", difftime(end, start)/CLOCKS_PER_SEC);
return (0);
}
- The C implementation of conv2d (compiled with
-O2 -ffast-math)
struct Tensor *conv2d(struct Tensor *tensor, struct Tensor *kernel,
float const bias, uint32_t const stride) {
__assert_non_zero_stride(stride);
__assert_conv_kernel_size(tensor, kernel);
uint32_t const h = ((tensor->shape[0] - (kernel->shape[0] - 1) - 1) / stride) + 1;
uint32_t const w = ((tensor->shape[1] - (kernel->shape[1] - 1) - 1) / stride) + 1;
struct Tensor *out = tensor_init_constant(h, w, bias);
for (uint32_t i = 0; i < h; i++)
for (uint32_t k = 0; k < kernel->shape[0]; k++)
for (uint32_t l = 0; l < kernel->shape[1]; l++)
for (uint32_t j = 0; j < w; j++)
out->data[i * w + j] += kernel->data[k * kernel->shape[1] + l] * tensor->data[(i + k) * tensor->shape[1] + l + j];
return (out);
}
- Some env informations
PyTorch version: 1.10.0
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 12.0.1 (arm64)
Clang version: 12.0.5 (clang-1205.0.22.9)
Python platform: macOS-12.0.1-arm64-arm-64bit
Python version: 3.9.7
CUDA runtime version: No CUDA