Inserting TORCH.CLAMP or range restriction into MobileNetV3

Hello everyone,

I am currently testing the influence from soft error on the MobileNetV3 large. I am basically injecting bitflips into the model weights after the training has been completed, afterwards I am testing the impacts of the soft errors with the test dataset, whether there has been a drop in accuracy or not.
My result shows that especially the MSB can have an large effect for float32 model weights since values can change from 0.0xx to 1*e^x for example.
I want to prevent the model from basically computing the output with such large weights. And I came across the troch.clamp function.

From my understanding torch.clamp sets a min and max range for a tensor. So would it be possible to insert torch.clamp to basically mitigate the effect of large numbers after the training actually has completed since I don’t want to clamp the weights during the training. I want to prevent the model to “use” these large weights which completely fall out of the scale of the rest.

As a side note, i am using the already implemented MobileNetV3_large model from torchvision. So maybe there is a way to implement the range restriction without actually changing the models architecture.

For reference I am thinking of something similar to this GitHub - DependableSystemsLab/Ranger

Any help or suggestions would be appreciated.

Right now I am using a workaround. I am using torch.clamp after I injected the fault and before evaluating with the Test-Data, this seems to mitigate values which are higher than -1 and 1 (for example) that occur due to a bitflip.
Values which are not within these borders are set either to the min value or to the max value in this case -1 and 1.
The next step would be to implement this procedure into the architecture itself. Keeping others that might be interested up to date.