Adding the metadata information into CNN

I have a CNN model in pytorch. My dataset comes with a lot of metadata information. Besides using the image I want to use this metadata information to improve my accuracy. Something like a small SVM classifier that uses last convolution layer features + metadata information to finish up the model and train on that. Any codebase links/paper would be helpful which can guide me in this regard.

Please refer to following tutorial https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#finetuning-the-convnet