I tested this code both cpu and gpu mode
For gpu mode I set device=0, for cpu I set device=-1
Why is gpu mode slower by 1 second
Am I doing something incorrect, I thought gpu was meant to be faster
from transformers import pipeline
model="facebook/bart-large-mnli"
classifier = pipeline(
task="zero-shot-classification",
model=model,
device=0
)
text_piece = "I went to the beach"
candidate_labels=["games", "holiday", "accident"]
result = classifier(text_piece, candidate_labels)
print(result)
time python3 on_gpu.py
{'sequence': 'I went to the beach', 'labels': ['holiday', 'accident', 'games'], 'scores': [0.980344831943512, 0.013516747392714024, 0.0061384160071611404]}
real 0m6.021s
user 0m5.915s
sys 0m2.293s
time python3 on_cpu.py
{'sequence': 'I went to the beach', 'labels': ['holiday', 'accident', 'games'], 'scores': [0.9803448915481567, 0.01351669616997242, 0.006138448603451252]}
real 0m4.948s
user 0m6.966s
sys 0m1.570s