Hi all,
I’m using a trained LinkNet34 model. I was expecting that doing batch inference of size n
would be faster than doing n
times one single inference. Although, most of the time the batch computing shows only a slight advantage over n
times single inference.
Am I missing something?
I’m only using CPU. I have applied model.eval()
before. I also tried model.foward
method, and with torch.no_grad()
context. They didn’t affect the results.
Thanks in advance,