CUDA out of memory for a tiny network

Memory Format
1

I have a tiny CNN model that gives this summary:

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 32, 222, 222]             896
              ReLU-2         [-1, 32, 222, 222]               0
         MaxPool2d-3         [-1, 32, 111, 111]               0
            Conv2d-4         [-1, 32, 109, 109]           9,248
              ReLU-5         [-1, 32, 109, 109]               0
         MaxPool2d-6           [-1, 32, 54, 54]               0
            Conv2d-7           [-1, 32, 52, 52]           9,248
              ReLU-8           [-1, 32, 52, 52]               0
         MaxPool2d-9           [-1, 32, 26, 26]               0
          Flatten-10                [-1, 21632]               0
           Linear-11                    [-1, 1]          21,633
          Sigmoid-12                    [-1, 1]               0
================================================================
Total params: 41,025
Trainable params: 41,025
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 35.24
Params size (MB): 0.16
Estimated Total Size (MB): 35.97
----------------------------------------------------------------

The input images are 3x224x224 and the batch size is 16. When I start training the model (with torch.optim.SGD), I get this:

RuntimeError: CUDA out of memory. Tried to allocate 11.89 GiB (GPU 0; 8.00 GiB total capacity; 1.14 GiB already allocated; 4.83 GiB free; 1.14 GiB reserved in total by PyTorch)

Why does it try to allocate so much memory for such a small network? Thanks.

2

I found the bug which was trivial: I was loading all the validation images into a single batch which was too big. I would delete this post but don’t see any option to do that.

3

Click the ‘show more’ button on the right bottom of your post. Then you can see a trash can.