pinocchio
(Rene Sandoval)
April 29, 2020, 8:19pm
65
What is wrong with doing:
def save_ckpt(path_to_ckpt):
from pathlib import Path
import dill as pickle
## Make dir. Throw no exceptions if it already exists
path_to_ckpt.mkdir(parents=True, exist_ok=True)
ckpt_path_plus_path = path_to_ckpt / Path('db')
## Pickle args
db['crazy_mdl'] = crazy_mdl
with open(ckpt_path_plus_path , 'ab') as db_file:
pickle.dump(db, db_file)
I currently have a neural network module:
import torch.nn as nn
class NN(nn.Module):
def __init__(self,args,lambda_f,nn1, loss, opt):
super().__init__()
self.args = args
self.lambda_f = lambda_f
self.nn1 = nn1
self.loss = loss
self.opt = opt
# more nn.Params stuff etc...
def forward(self, x):
#some code using fields
return out
I am trying to checkpoint it but because pytorch saves using state_dicts it means I can…
I was reading Save and Load model but it wasn’t clear why I’d use torch.save over pickle.dump.
What worries me is that my Neural Net modules/object have many more things inside of them besides only parameters. Since pytorch saves things using the state_dict that worried me that something might be missing (https://pytorch.org/tutorials/beginner/saving_loading_models.html ).
Isn’t for my use case easier to simply use the pickle.dump function? Why would I use torch.save in general?
Note I am usin…
what are the advantages and disadvantages of just using the python pickle
@ptrblck what are the advantages and disadvantages of just using the python pickle module vs the load/save stuff pytorch has built?