Memory usage is increasing when use cpu to inference resnet101 network model

2020-10-26 16:51:25.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1600.jpg
torch.Size([1, 3, 1024, 724])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    506.9 MiB    506.9 MiB           1       @profile
   141                                             def forward(self, x):
   142    553.1 MiB     46.1 MiB           1           x = self.conv1(x)
   143    598.6 MiB     45.5 MiB           1           x = self.bn1(x)
   144    598.6 MiB      0.1 MiB           1           x = self.relu(x)
   145    612.9 MiB     14.3 MiB           1           x = self.maxpool(x)
   146                                         
   147    681.8 MiB     68.9 MiB           1           x = self.layer1(x)
   148    707.1 MiB     25.4 MiB           1           x = self.layer2(x)
   149    755.9 MiB     48.7 MiB           1           x = self.layer3(x)
   150    788.4 MiB     32.5 MiB           1           x = self.layer4(x)
   151    788.4 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153    788.4 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:29.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1701.jpg
torch.Size([1, 3, 768, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    685.6 MiB    685.6 MiB           1       @profile
   141                                             def forward(self, x):
   142    733.6 MiB     48.0 MiB           1           x = self.conv1(x)
   143    781.6 MiB     48.0 MiB           1           x = self.bn1(x)
   144    781.6 MiB      0.0 MiB           1           x = self.relu(x)
   145    781.6 MiB      0.0 MiB           1           x = self.maxpool(x)
   146                                         
   147    854.1 MiB     72.4 MiB           1           x = self.layer1(x)
   148    940.2 MiB     86.1 MiB           1           x = self.layer2(x)
   149    985.1 MiB     45.0 MiB           1           x = self.layer3(x)
   150   1022.6 MiB     37.4 MiB           1           x = self.layer4(x)
   151   1022.6 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1022.6 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:32.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1728.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    817.4 MiB    817.4 MiB           1       @profile
   141                                             def forward(self, x):
   142    853.4 MiB     36.0 MiB           1           x = self.conv1(x)
   143    889.5 MiB     36.0 MiB           1           x = self.bn1(x)
   144    889.5 MiB      0.0 MiB           1           x = self.relu(x)
   145    898.5 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147    952.8 MiB     54.3 MiB           1           x = self.layer1(x)
   148   1027.0 MiB     74.2 MiB           1           x = self.layer2(x)
   149   1034.5 MiB      7.5 MiB           1           x = self.layer3(x)
   150   1064.4 MiB     29.9 MiB           1           x = self.layer4(x)
   151   1064.4 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1064.4 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:35.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1729.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    960.9 MiB    960.9 MiB           1       @profile
   141                                             def forward(self, x):
   142    996.9 MiB     36.0 MiB           1           x = self.conv1(x)
   143   1032.8 MiB     35.9 MiB           1           x = self.bn1(x)
   144   1032.8 MiB      0.0 MiB           1           x = self.relu(x)
   145   1041.8 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1095.6 MiB     53.7 MiB           1           x = self.layer1(x)
   148   1149.4 MiB     53.8 MiB           1           x = self.layer2(x)
   149   1149.4 MiB      0.0 MiB           1           x = self.layer3(x)
   150   1151.5 MiB      2.1 MiB           1           x = self.layer4(x)
   151   1144.8 MiB     -6.8 MiB           1           x = self.avgpool(x)
   152                                         
   153   1144.8 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:37.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1730.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    960.4 MiB    960.4 MiB           1       @profile
   141                                             def forward(self, x):
   142    996.4 MiB     36.0 MiB           1           x = self.conv1(x)
   143   1032.3 MiB     35.9 MiB           1           x = self.bn1(x)
   144   1032.3 MiB      0.0 MiB           1           x = self.relu(x)
   145   1041.3 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1095.1 MiB     53.7 MiB           1           x = self.layer1(x)
   148   1148.9 MiB     53.8 MiB           1           x = self.layer2(x)
   149   1148.9 MiB      0.0 MiB           1           x = self.layer3(x)
   150   1151.0 MiB      2.1 MiB           1           x = self.layer4(x)
   151   1148.1 MiB     -2.9 MiB           1           x = self.avgpool(x)
   152                                         
   153   1148.1 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:40.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1873
torch.Size([1, 3, 835, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    969.1 MiB    969.1 MiB           1       @profile
   141                                             def forward(self, x):
   142   1021.4 MiB     52.3 MiB           1           x = self.conv1(x)
   143   1073.6 MiB     52.3 MiB           1           x = self.bn1(x)
   144   1073.6 MiB      0.0 MiB           1           x = self.relu(x)
   145   1089.9 MiB     16.3 MiB           1           x = self.maxpool(x)
   146                                         
   147   1168.6 MiB     78.7 MiB           1           x = self.layer1(x)
   148   1223.2 MiB     54.6 MiB           1           x = self.layer2(x)
   149   1272.3 MiB     49.1 MiB           1           x = self.layer3(x)
   150   1280.7 MiB      8.4 MiB           1           x = self.layer4(x)
   151   1280.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1280.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:43.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1874
torch.Size([1, 3, 961, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1127.2 MiB   1127.2 MiB           1       @profile
   141                                             def forward(self, x):
   142   1187.4 MiB     60.2 MiB           1           x = self.conv1(x)
   143   1247.5 MiB     60.1 MiB           1           x = self.bn1(x)
   144   1247.5 MiB      0.0 MiB           1           x = self.relu(x)
   145   1262.5 MiB     15.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1413.0 MiB    150.4 MiB           1           x = self.layer1(x)
   148   1542.6 MiB    129.6 MiB           1           x = self.layer2(x)
   149   1568.0 MiB     25.5 MiB           1           x = self.layer3(x)
   150   1613.7 MiB     45.7 MiB           1           x = self.layer4(x)
   151   1613.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1613.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:48.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1875
torch.Size([1, 3, 788, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1424.6 MiB   1424.6 MiB           1       @profile
   141                                             def forward(self, x):
   142   1473.8 MiB     49.3 MiB           1           x = self.conv1(x)
   143   1523.1 MiB     49.3 MiB           1           x = self.bn1(x)
   144   1523.1 MiB      0.0 MiB           1           x = self.relu(x)
   145   1535.4 MiB     12.3 MiB           1           x = self.maxpool(x)
   146                                         
   147   1609.6 MiB     74.2 MiB           1           x = self.layer1(x)
   148   1661.2 MiB     51.6 MiB           1           x = self.layer2(x)
   149   1707.8 MiB     46.6 MiB           1           x = self.layer3(x)
   150   1727.7 MiB     19.8 MiB           1           x = self.layer4(x)
   151   1727.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1727.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:51.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1876
torch.Size([1, 3, 744, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1561.1 MiB   1561.1 MiB           1       @profile
   141                                             def forward(self, x):
   142   1607.6 MiB     46.5 MiB           1           x = self.conv1(x)
   143   1654.1 MiB     46.5 MiB           1           x = self.bn1(x)
   144   1654.1 MiB      0.0 MiB           1           x = self.relu(x)
   145   1665.7 MiB     11.6 MiB           1           x = self.maxpool(x)
   146                                         
   147   1735.8 MiB     70.1 MiB           1           x = self.layer1(x)
   148   1836.8 MiB    100.9 MiB           1           x = self.layer2(x)
   149   1869.1 MiB     32.3 MiB           1           x = self.layer3(x)
   150   1869.2 MiB      0.1 MiB           1           x = self.layer4(x)
   151   1869.2 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1869.2 MiB      0.0 MiB           1           return x

This is memory usage log, tracked with “Memory-Profile” python package
I have tried "self.net.eval() " and “with torch.no_grad()”, but It was not worked.
And I find a weird phenomenon, When input varied size images, that Memory increasing is bigger than same size input.
Does anyone have encountered the same problem?

Main environment:
Mac OS
Python 3.7
PyTorch 1.2(also tried with PyTorch 1.5)

see this: