I have been trying to teach myself Pytorch using some Kaggle data on the seedlings competition. My code was working but taking a long time to run so decided to try to freeze all layers except the last, so I added the below code:
num_ftrs = resnet34.fc.in_features
## Freeze all but the last layers
for param in resnet34.parameters():
## Each tensor has the flag requires_grad, setting it to false allows freezes
## the parmaeter associated with it
param.requires_grad = False
# Parameters of newly constructed modules have requires_grad=True by default
## Create new modules that will become final layer
num_ftrs = resnet34.fc.in_features
print(num_ftrs)
## Give final layers a linear transform with twelve outputs one for each category
resnet34.fc = nn.Linear(num_ftrs, 12)
Any other pointers for my code would also be appreciated.
Been a bit busy (and frustrating) I am still having issues. I was looking at the below link:
That persons problem was that the last layer wasn’t outputting the number of categories, but mine is implementing the correct number of layers resnet34.fc = nn.Linear(num_ftrs, 12).
The line that it says is not working is the loss = criteria(output, target) and printing the output and target gives one 12 x batch_size for output and batch_size x 1 for target.
Thanks the target values of the first batch before it stops is
Target
11
1
1
9
10
9
7
5
2
2
6
3
6
6
7
12
Is it because it has a 12 in it and Python assumes 0-11 while the target output is 1-12?
If there is a problem with my target values why does it work before I put in the code to freeze all layers?
Yes, it’s because of the 12. If your mapping is in [1, 12], just subtract 1 and it should work.
That’s strange, since it shouldn’t work.
Could you try to remove the freezing part and check if again?
It would be interesting to know, if it was ignored somehow.
Sorry ptrblck, still having some problems understanding. The data loaded into the tensor thats returning the numbers 1-12 is image data taken from 12 folders of images of particular seedlings. So when I tried to subtract 1 from train_raw I get an error that you can’t subtract an int from an image.
So the program is actually creating the 1-12 numbers itself, so I can I get it to create 0-11 instead.
This is on Google cloud and there seems to be a folder called ’.ipynb_checkpoints’ which is 0. I did it by changing the number of outputs in the final layer to 13:
resnet34.fc = nn.Linear(num_ftrs, 13)
I’m not sure what to do with the invisible folder besides changing the number of outputs in the final layer, is there any other way I can deal with that? Will this create problems by having an unnecessary layer?
I would move the valid data into a subfolder. As far as I know .ipynb_checkpoints will be created in your current working directory where your notebook was created.