Hi all,
Suppose I extracted the image features from a pretrained CNN and wants to save these features to disk for later use. Which of the following will be a good method in terms of memory and speed.
Store the feature dictionary in a pickle file
Convert the feature tensor to numpy arrays and save using np.save()
Save it in a json file ( TypeError: Object of type ‘FloatTensor’ is not JSON serializable is coming)
I would probably save them as torch.tensors with torch.save.
This should be fast and as long as you load them again in PyTorch there shouldn’t be a problem.
I assume “features” refers to an intermediate forward activation. If so, you could use forward hooks as described here to grab them and that you could store them as numpy arrays afterwards.