AttributeError: 'NoneType' object has no attribute 'in_channels'

1

I am trying to use prune.py in python 3.7.1 and pytorch 1.0, but I have the following error:

[phung@archlinux SqueezeNet-Pruning]$ python finetune.py --prune
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
warnings.warn("The use of the transforms.Scale transform is deprecated, " +
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:562: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
warnings.warn("The use of the transforms.RandomSizedCrop transform is deprecated, " +
Accuracy : 1.0
Number of prunning iterations to reduce 67% filters 13
Ranking filters…
Layers that will be prunned {3: 2, 6: 9, 12: 77, 9: 10, 10: 8, 11: 14, 7: 7, 0: 1}
Prunning filters…
Traceback (most recent call last):
File “finetune.py”, line 306, in
fine_tuner.prune()
File “finetune.py”, line 242, in prune
model = prune_squeezenet_conv_layer(model, layer_index, filter_index)
File “/home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/pruning/Pruning-CNN/SqueezeNet-Pruning/prune.py”, line 146, in prune_squeezenet_conv_layer
torch.nn.Conv2d(in_channels = old_conv_layer.in_channels-1,
AttributeError: ‘NoneType’ object has no attribute ‘in_channels’
[phung@archlinux SqueezeNet-Pruning]$

2

old_conv_layer is None
Can you share your source to confirm?

3

Please see modified finetune.py and modified prune.py which I updated few lines from the original github repo finetune.py and prune.py respectively

4

Problem is setting old_conv_layer to None in prune_squeezenet_conv_layer

Please verify that

if ( int(_i)>2  and isinstance(module, torch.nn.modules.conv.Conv2d)):
                old_conv_layer = module

Is true or not.
Can you check this condition? I would recommend to initialize old_conv_layer to some base model (for non-specialized case)
Also, Can you add an assert for old_conv_layer to be non-none (good programming practice)

1 Like
5

initialize old_conv_layer to some base model (for non-specialized case)

Could you show how to do this ?

You are right about old_conv_layer being NONE type

6

any idea why int(_i) > 2 ?

I think this is a bug, the variable _i is never assigned any value before the if statement check, right ?

7

Regarding initializing old_conv_layer to some base layer

This is design decision. If you need to fall back to some network then only you should do this. Otherwise, have assert to ensure you get the error.
How to add assert? assert(old_conv_layer), “Old Conv Layer is not initialized”

Regarding int(_i) > 2

it is iterating over module items and is number is classifier net of current model.
I would recommend to print model and model.classifier just to understand which all layers are there
also print model.classifier._modules.items()

If condition is ensuring you are replacing first Conv2d placed after two layers.
This is again design decision.
Try to understand the architecture and what is being accomplished and why second layer needs to be replaced.

I guess, if old_conv_layer is None i.e. not found then you don’t have to use old_weights. So, just use initialized values as new_weights and ensure you don’t use old_conv_layer related variables (this will avoid use of assert)

8

replacing first Conv2d placed after two layers

why AFTER two layers ?

By the way, I am setting the whole project up in colab soon, so people will be able to run these code on their own in the cloud

[phung@archlinux SqueezeNet-Pruning]$ python finetune.py --prune
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
warnings.warn("The use of the transforms.Scale transform is deprecated, " +
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:562: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
warnings.warn("The use of the transforms.RandomSizedCrop transform is deprecated, " +
Accuracy : 1.0
Number of prunning iterations to reduce 67% filters 13
Ranking filters…
Layers that will be prunned {3: 2, 6: 11, 12: 70, 9: 17, 10: 11, 7: 5, 11: 10, 0: 1, 4: 1}
Prunning filters…
model = ModifiedSqueezeNetModel(
(classifier): Sequential(
(0): Dropout(p=0.5)
(1): Conv2d(512, 2, kernel_size=(1, 1), stride=(1, 1))
(2): ReLU(inplace)
(3): AvgPool2d(kernel_size=13, stride=1, padding=0)
)
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(2, 2))
(1): ReLU(inplace)
(2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)
(3): Fire(
(squeeze): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(16, 62, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(4): Fire(
(squeeze): Conv2d(126, 16, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(16, 64, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)
(6): Fire(
(squeeze): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(32, 121, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(32, 124, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(7): Fire(
(squeeze): Conv2d(245, 32, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(32, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(8): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)
(9): Fire(
(squeeze): Conv2d(256, 48, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(48, 192, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(48, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(10): Fire(
(squeeze): Conv2d(384, 48, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(48, 192, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(48, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(11): Fire(
(squeeze): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
(12): Fire(
(squeeze): Conv2d(512, 64, kernel_size=(1, 1), stride=(1, 1))
(squeeze_activation): ReLU(inplace)
(expand1x1): Conv2d(64, 255, kernel_size=(1, 1), stride=(1, 1))
(expand1x1_activation): ReLU(inplace)
(expand3x3): Conv2d(64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(expand3x3_activation): ReLU(inplace)
)
)
)
model.classifier = Sequential(
(0): Dropout(p=0.5)
(1): Conv2d(512, 2, kernel_size=(1, 1), stride=(1, 1))
(2): ReLU(inplace)
(3): AvgPool2d(kernel_size=13, stride=1, padding=0)
)
model.classifier._modules.items() = odict_items([(‘0’, Dropout(p=0.5)), (‘1’, Conv2d(512, 2, kernel_size=(1, 1), stride=(1, 1))), (‘2’, ReLU(inplace)), (‘3’, AvgPool2d(kernel_size=13, stride=1, padding=0))])
Traceback (most recent call last):
File “finetune.py”, line 306, in
fine_tuner.prune()
File “finetune.py”, line 242, in prune
model = prune_squeezenet_conv_layer(model, layer_index, filter_index)
File “/home/phung/Documents/Grive/Personal/Coursera/Machine_Learning/pruning/Pruning-CNN/SqueezeNet-Pruning/prune.py”, line 150, in prune_squeezenet_conv_layer
assert (old_conv_layer), “Old Conv Layer is not initialized”
AssertionError: Old Conv Layer is not initialized
[phung@archlinux SqueezeNet-Pruning]$

9

That’s the issue.
i_2 is iterating over index mentioned below

model.classifier = Sequential(
(0): Dropout(p=0.5)
(1): Conv2d(512, 2, kernel_size=(1, 1), stride=(1, 1))
(2): ReLU(inplace)
(3): AvgPool2d(kernel_size=13, stride=1, padding=0)
)

Since, classifier has only one Conv2d, that too at index 1, _i > 2 condition is never satisfied.
Please remove _i > 2 from the condition and only check for

isinstance(module, torch.nn.modules.conv.Conv2d)

1 Like
10

Just a side question:

Now that the pruning code runs without any errors.

How do I know that it actually works ? Do I need to implement some bounding box detection mechanism such as YOLO or Fast-RCNN ? Could you comment about this ?

11

Yes. You can do that.
There are multiple ways to validate this -

  1. Get object detection accuracy with and without pruning and compare - this will give exact stats and compare points
  2. Manual bounding boxes will help you understand that it works but will not tell you which one works better and by what margin.
12

Object detection accuracy ? you still need bounding box to measure detection accuracy

I will open a thread on this question in darknet googlegroup mailing list.

  1. You should label each object on images from your dataset. Use this visual GUI-software for marking bounded boxes of objects and generating annotation files for Yolo v2 & v3: GitHub - AlexeyAB/Yolo_mark: GUI for marking bounded boxes of objects in images for training neural network Yolo v3 and v2

It will create .txt -file for each .jpg -image-file - in the same directory and with the same name, but with .txt -extension, and put to file: object number and object coordinates on this image, for each object in new line: <object-class> <x> <y> <width> <height>

13

What do you think about this test function which somehow has the ability of testing out the pruned model ?

[phung@archlinux SqueezeNet-Pruning]$ python finetune.py --run
/usr/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
warnings.warn("The use of the transforms.Scale transform is deprecated, " +
pred = tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], device=‘cuda:0’)
pred = tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], device=‘cuda:0’)
pred = tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], device=‘cuda:0’)
pred = tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], device=‘cuda:0’)
pred = tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], device=‘cuda:0’)
pred = tensor([0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1], device=‘cuda:0’)
pred = tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1], device=‘cuda:0’)
pred = tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1], device=‘cuda:0’)
pred = tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1], device=‘cuda:0’)
pred = tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1], device=‘cuda:0’)
pred = tensor([1, 1, 1, 1], device=‘cuda:0’)
Avg time taken per image is(over 800 Images…) 0.0007848718762397766
[phung@archlinux SqueezeNet-Pruning]$ ls -al
total 3800
drwxrwxr-x 5 phung phung 4096 Dec 2 13:59 .
drwxrwxr-x 5 phung phung 4096 Nov 21 11:07 …
-rw-rw-r-- 1 phung phung 1696 Nov 21 11:02 dataset.py
-rw-rw-r-- 1 phung phung 9545 Dec 2 23:07 finetune.py
-rw-rw-r-- 1 phung phung 2560 Nov 21 15:05 flops.py
-rw-r–r-- 1 phung phung 50 Dec 2 13:59 .gitignore
-rw-r–r-- 1 phung phung 2929897 Nov 29 22:58 model
-rw-r–r-- 1 phung phung 892984 Dec 2 21:00 model_prunned
-rw-rw-r-- 1 phung phung 8518 Dec 2 00:37 prune.py
drwxrwxr-x 2 phung phung 4096 Dec 2 09:13 __pycache__
-rw-rw-r-- 1 phung phung 661 Nov 21 11:02 README.md
drwxrwxr-x 2 phung phung 4096 Nov 29 22:57 test
drwxrwxr-x 2 phung phung 4096 Nov 29 22:56 train
[phung@archlinux SqueezeNet-Pruning]$

By the way, I have uploaded the project to colab

14

If you have ground truth then compare and print accuracy as well (Optional)

15

What does this line actually do ? Does this have to do with ground truth ?

In my previous post, the ‘pred’ prints out to be a tensor array ?

16

You are working on a classification problem and applying softmax layer to the output which will emit the probability for all classes.

Here, you are predicting a particular class (i.e. a class which has max probability predicted by your network)

Ground truth is actual label for current image and pred is your predicted class.
pred is tensor of size [ batch_size x 1 ] similar to ground truth tensor.

I would recommend to read up on classification problems here for learning: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html

1 Like
17

This really does not sum up to 1 which is based on softmax().

I do not think the code is using softmax() here, I presume ? The softmax is inside the squeezeNet model itself
I have searched for the occurrence of the keyword “softmax”, but it is not found in the code.

Besides, why pred is tensor of size [ batch_size x 1 ] ?