PyTorch tutorial for Neural transfert of artistic style

21

Thanks ! I am on my way to try it ! If you don’t mind, I will also borrow your download_model.sh, so I can add VGG usage to the tutorial :innocent:

22

I’ve been meaning to upload this for awhile. May not be useful anymore since @leongatys has been so generous as to give us the ‘official’ version… :slight_smile:

anyway:
https://github.com/tymokvo/pt-styletransfer

@alexis-jacq
I stole your style transfer code and made some changes. I mainly wanted to use VGG and make the st network importable in other scripts.

Sorry the code is sloppy, I made it in a hurry.

Also, if you haven’t yet, you should try saving every iteration into a GIF. Makes some cool animations.

1 Like
23

@tymokvo It looks nice ! And thanks for the citation :wink:

I think we should definitely move to Leon’s way to extract features from VGG. As soon I have time (By dint of playing with Pytorch I am getting very late with my PhD…), I will adapt the tutorial based on his idea.

1 Like
24

Thanks for this implementation, the feature extraction and loss code is pythonic and elegant, learned a lot!

1 Like
25

@alexis-jacq When you use pretrained vggnet, you should normalize the input image using mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225]. However, your code do not normalize the input. I think adding the normalization will makes the results better than now.

See here for details.

1 Like
26

Thanks for the code. I see you are not clipping the image range in each iteration, but only at the very end, is that correct?

27

I find this issue still in the pytorch tutorial.

28

will be fixed in https://github.com/pytorch/tutorials/pull/223

2 Likes
29

Just fyi, this has been fixed already! The result doesn’t look too different though :wink:

1 Like
30

Thanks for the tutorial, Alexis! I have one comment about the feature layer selection. Not an error per se, but I think the default content and style layers in the tutorial don’t match what Gatys’ et al. describe in their paper or what you can find on Gatys’ github. I am seeing him collect style features after what he labels conv{1_1, 2_1, 3_1 4_1 ,5_1}, which in your notation would be conv_1, conv_3, conv_5, conv_9, conv_13, if I’m not mistaken. Likewise his default content layer is conv4_2 which corresponds to conv_10 in your numbering. I guess the discrepancy arises from the fact that his major index counts how many pooling layers the image has gone through, not how many convolutions.

If I’m wrong I hope you’ll let me know, though I would say I’ve tended to have better luck using the deeper feature maps.

31

It is true that I did not use exactly the same combinations of layers for style and content losses than Leon. I would say it’s “my style”. If you change the choice of layers, you change the degree of deformations and the deepness of style concepts.

I found this setup makes the best artistic results with cubism inputs. But feel free to try different combinations!

32

why you use F.mse_loss?
is it possible that use another loss function?

33

Hello, my friend!
I find that I can’t download vgg_conv.pth through ‘https://bethgelab.org/media/uploads/pytorch_models/vgg_conv.pth’. Can you provide another way to download the model. Thank you very much!