PyTorch tutorial for Neural transfert of artistic style

Thanks for having reported this issue.

The code is working on my computer but anyway, I did it quickly when I discovered Pytorch, so I am not surprised if it causes bugs on another system. It is fool of hacks and the implementation is not clean (as you can see here: How to extract features of an image from a trained model). I have to re-write it, I will do it as soon as I have time for myself.

@ecolss Wouldn’t data.view be more appropriate than data.resize in this case? The output tensor has the same size, just a different shape. I think PyTorch’s view is very similar to numpy’s reshape method.

Yes, it’s better to use .view

.view is cool, I just wasn’t aware of it before.
However, .resize also returns a view, doesn’t it? I mean any particular difference between the two?

.view is way way safer than .resize and there are hardly any cases when .resize should be used in user scripts. It will raise an error if you try to get a tensor with a different number of elements, or if it’s not contiguous (.resize can give you a tensor that views on a data that wasn’t used before).

@apaszke Noted, thanks

Hi Alexis, cool work!
I think you can get better results by using LBFGS though. You can check here for an implementation: https://github.com/leongatys/PytorchNeuralStyleTransfer

Best
Leon

2 Likes

Thanks ! I am on my way to try it ! If you don’t mind, I will also borrow your download_model.sh, so I can add VGG usage to the tutorial :innocent:

I’ve been meaning to upload this for awhile. May not be useful anymore since @leongatys has been so generous as to give us the ‘official’ version… :slight_smile:

anyway:
https://github.com/tymokvo/pt-styletransfer

@alexis-jacq
I stole your style transfer code and made some changes. I mainly wanted to use VGG and make the st network importable in other scripts.

Sorry the code is sloppy, I made it in a hurry.

Also, if you haven’t yet, you should try saving every iteration into a GIF. Makes some cool animations.

1 Like

@tymokvo It looks nice ! And thanks for the citation :wink:

I think we should definitely move to Leon’s way to extract features from VGG. As soon I have time (By dint of playing with Pytorch I am getting very late with my PhD…), I will adapt the tutorial based on his idea.

1 Like

Thanks for this implementation, the feature extraction and loss code is pythonic and elegant, learned a lot!

1 Like

@alexis-jacq When you use pretrained vggnet, you should normalize the input image using mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225]. However, your code do not normalize the input. I think adding the normalization will makes the results better than now.

See here for details.

1 Like

Thanks for the code. I see you are not clipping the image range in each iteration, but only at the very end, is that correct?

I find this issue still in the pytorch tutorial.

will be fixed in https://github.com/pytorch/tutorials/pull/223

2 Likes

Just fyi, this has been fixed already! The result doesn’t look too different though :wink:

1 Like

Thanks for the tutorial, Alexis! I have one comment about the feature layer selection. Not an error per se, but I think the default content and style layers in the tutorial don’t match what Gatys’ et al. describe in their paper or what you can find on Gatys’ github. I am seeing him collect style features after what he labels conv{1_1, 2_1, 3_1 4_1 ,5_1}, which in your notation would be conv_1, conv_3, conv_5, conv_9, conv_13, if I’m not mistaken. Likewise his default content layer is conv4_2 which corresponds to conv_10 in your numbering. I guess the discrepancy arises from the fact that his major index counts how many pooling layers the image has gone through, not how many convolutions.

If I’m wrong I hope you’ll let me know, though I would say I’ve tended to have better luck using the deeper feature maps.

It is true that I did not use exactly the same combinations of layers for style and content losses than Leon. I would say it’s “my style”. If you change the choice of layers, you change the degree of deformations and the deepness of style concepts.

I found this setup makes the best artistic results with cubism inputs. But feel free to try different combinations!

why you use F.mse_loss?
is it possible that use another loss function?

Hello, my friend!
I find that I can’t download vgg_conv.pth through ‘https://bethgelab.org/media/uploads/pytorch_models/vgg_conv.pth’. Can you provide another way to download the model. Thank you very much!