Performance regression in advanced indexing

jens · June 19, 2023, 10:16pm

Hello people, thanks for the amazing library. I am working on a simulation package which uses pytorch mostly as a numpy replacement, so not an ML use case. We found performance regressions in the advanced indexing going from 1.10.1 to 2.0.1.The torch versions are pip installed. We tested x86_64 and arm.

The python benchmark is attached. Each test was run more than 10k times (after a warmup of 10k iterations) and the minimum from each run was selected. All tests are basically indexing operations into arrays for reading or writing. In comparison we also tested the same operations using numpy. Tensor size is (1000,) and a random bitmask with 500 true entries was generated, which was used for all calculations.

gist.github.com

https://gist.github.com/JensWehner/3524a121f42a3ec912fa8f843249b75d

tensor_tests.py

# type: ignore
"""Performance tests for tensor access variations."""
from typing import Callable, List, Union

import itertools
import numpy as np
import pytest
import torch

This file has been truncated. show original

pytest --randomly-seed=1 --benchmark-verbose --benchmark-only --benchmark-disable-gc --benchmark-warmup=on --benchmark-warmup-iterations=10000 -m torchperformance test_performance_tensor_access.py

Thanks!

ptrblck · June 20, 2023, 8:09am

How does the mean look after performing proper warmup iterations?

jens · June 20, 2023, 9:05am

So using the average instead of the min we got this result. The warmup was 10k. Do you have any experience what kind of warm-up is sufficient?