Training speed slowdown problem when using gpu

When I’m training using univariate time-series data, when I use gpu in the SamePadConv layer, I get 1.33 seconds and use cpu in the SamePadConv layer, I get 0.0002 seconds.

class SamePadConv(Module):
    def __init__(self, in_channels, out_channels, kernel_size, dilation=1, groups=1):
        super().__init__()
        self.receptive_field = (kernel_size - 1) * dilation + 1
        padding = self.receptive_field // 2
        self.conv = nn.Conv1d(
            in_channels, out_channels, kernel_size,
            padding=padding,
            dilation=dilation,
            groups=groups
        )
        self.remove = 1 if self.receptive_field % 2 == 0 else 0
        
    def forward(self, x):
        out = self.conv(x)
        if self.remove > 0:
            out = out[:, :, : -self.remove]
        return out

As for as I know, if I use gpu, it should be faster, so why is this cause occurring?