What is the opposite of add_module?

So there’s an add_module method that lets me add modules, but how do I remove/replace them? For the time being, del works, but feels really dirty.

class net(nn.Module):
    def __init__(self):
        self.relu = nn.ReLU()
    def change(self):
            del self.relu #DIRTY
        def fun(x):
            return x
        self.relu = fun #crashes without the del statement...
    def forward(self,x):
        return self.relu(x)

You have funny ideas of cleanliness – just like we all do.
More seriously:

  • There is nothing wrong with del. This is the method Python and PyTorch forsee for removing something.
  • Even just overwriting a module with another without explicit del is OK.
  • What does seem a lot less clean (to me, but I’ll admit that I fancy myself to have very good tastes in this and well aligned with PyTorch at that) is that you sometimes have self.relu be a module and sometimes a function. This is not good, as modules are superspecial to PyTorch. I would recommend to either use functions throughout (e.g. torch.nn.functional.relu) or stay with modules when changing.

Best regards


Thanks for the fast response

  1. As long as del is appropriate (and doesn’t introduce bugs), I can live with that.
  2. Indeed, I can overwrite modules with other modules; I shot myself in the foot because I’m trying to replace a module with a function.
  3. That is indeed dirty, and I’ll think more about it.