I have an NLP model and I need to use data parallelism due to large batch data size. I wrapped my model using nn.DataParallel. I need the model attributes else where during training, validation, logging etc. Here’s how I’m doing it:
class Trainer(): def __init__(self, model, config): self.model = model if torch.cuda.device_count() > 1: print("Let's use", torch.cuda.device_count(), "GPUs!") self.model = nn.DataParallel(model) self.txt_property = self.model.txt_property # many more properties def do_something(self): param = self.model.param
AttributeError: 'DataParallel' object has no attribute 'txt_property'
What is the best practice to encapsulate model with