can someone tell me wath is the difference between nn.threshold and torch.clamp function if I just use the min value?
Sorry if this is a necro but this is the top result on Google right now when I search this question. I ran a little test to see which is faster, but your mileage will vary based on hardware.
Clamp seems to call a cpp script where as threshold calls a forward function, which takes longer, but sometimes you want the functionality of a nn.Module function. Calling threshold in-place without the function call speeds it up. It’s also worth noting you kind of get two thresholds out of clamp, so the fact that it takes about twice as long as in-place threshold makes the results equal in my eyes.
import torch
from torch import nn
import time
input = torch.randn(1000) # random inputs
m = nn.Threshold(0.0, 0.0) # function that replaces values below 0 with 0
n_trials = int(1e5)
start = time.process_time()
result_threshold = [ m(input) for i in range(n_trials)]
print(f'Threshold time for {n_trials} loops: {time.process_time() - start} seconds')
>>> Threshold time for 100000 loops: 1.5605827209999998 seconds
start = time.process_time()
result_inplace_threshold = [ torch.nn.functional.threshold(input, threshold=0.0, value=0.0, inplace=True) for i in range(n_trials)]
print(f'in-place threshold time for {n_trials} loops: {time.process_time() - start} seconds')
>>> in-place threshold time for 100000 loops: 0.5645996109999993 seconds
start = time.process_time()
result_clamp = [ torch.clamp(input, min=0, max=100) for i in range(n_trials)]
print(f'Clamp time for {n_trials} loops: {time.process_time() - start} seconds')
>>> Clamp time for 100000 loops: 1.0484716829999998 seconds