What is the opposite of add_module?

1

So there’s an add_module method that lets me add modules, but how do I remove/replace them? For the time being, del works, but feels really dirty.

class net(nn.Module):
    def __init__(self):
        self.relu = nn.ReLU()
    def change(self):
        try:
            del self.relu #DIRTY
        except:
            pass
        def fun(x):
            return x
        self.relu = fun #crashes without the del statement...
    def forward(self,x):
        return self.relu(x)
2

You have funny ideas of cleanliness – just like we all do.
More seriously:

  • There is nothing wrong with del. This is the method Python and PyTorch forsee for removing something.
  • Even just overwriting a module with another without explicit del is OK.
  • What does seem a lot less clean (to me, but I’ll admit that I fancy myself to have very good tastes in this and well aligned with PyTorch at that) is that you sometimes have self.relu be a module and sometimes a function. This is not good, as modules are superspecial to PyTorch. I would recommend to either use functions throughout (e.g. torch.nn.functional.relu) or stay with modules when changing.

Best regards

Thomas

3

Thanks for the fast response

  1. As long as del is appropriate (and doesn’t introduce bugs), I can live with that.
  2. Indeed, I can overwrite modules with other modules; I shot myself in the foot because I’m trying to replace a module with a function.
  3. That is indeed dirty, and I’ll think more about it.