Why is cpu faster the gpu in this case

I tested this code both cpu and gpu mode
For gpu mode I set device=0, for cpu I set device=-1

Why is gpu mode slower by 1 second
Am I doing something incorrect, I thought gpu was meant to be faster

from transformers import pipeline


classifier = pipeline(

text_piece = "I went to the beach"

candidate_labels=["games", "holiday", "accident"]

result = classifier(text_piece, candidate_labels)


time python3 on_gpu.py
{'sequence': 'I went to the beach', 'labels': ['holiday', 'accident', 'games'], 'scores': [0.980344831943512, 0.013516747392714024, 0.0061384160071611404]}
real    0m6.021s
user    0m5.915s
sys     0m2.293s

time python3 on_cpu.py
{'sequence': 'I went to the beach', 'labels': ['holiday', 'accident', 'games'], 'scores': [0.9803448915481567, 0.01351669616997242, 0.006138448603451252]}
real    0m4.948s
user    0m6.966s
sys     0m1.570s

You could properly profile the actual workload and get rid of the initialization and warmup time.