Model slows down significantly after first iteration?

#1

I’m testing out a pretrained network, but for some reason my code reports that it slows down significantly after the first iteration, and I don’t know why.

I’m running the following:

    for i in range(maxiter):
        with torch.no_grad():
            tt2 = time.time()
            results = model(seq_ref_batch)
            tt3 = time.time()
        print("{:}/{:} times {:2.2f}s".format(i,maxiter,tt3-tt2))

Which produces the following:

0/603 times 0.16s
1/603 times 3.42s
2/603 times 3.49s
3/603 times 3.49s
4/603 times 3.50s
5/603 times 3.48s
6/603 times 3.48s
7/603 times 3.48s
8/603 times 3.48s
9/603 times 3.49s
10/603 times 3.50s
11/603 times 3.50s
12/603 times 3.54s
13/603 times 3.51s
14/603 times 3.50s
15/603 times 3.50s
16/603 times 3.50s
17/603 times 3.49s
18/603 times 3.50s
19/603 times 3.50s
20/603 times 3.50s
21/603 times 3.51s
22/603 times 3.49s
23/603 times 3.50s
24/603 times 3.50s
25/603 times 3.51s

Does anyone know what is going on here?

#2

you should add torch.cuda.synchronize() line before tt3

1 Like