Hi,
For me, Torch.Profiler is not working with CUDA activity only. With CPU it is working for me.
Code:
with torch.profiler.profile(
activities=[torch.profiler.ProfilerActivity.CUDA],
schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),
on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/model'),
profile_memory=False,
record_shapes=True,
with_stack=True,
use_cuda=True
) as prof:
for mini_batch_data in dataloader:
...
Error Msg: “AssertionError: No activities specified for Kineto profiler”
Hi ptrblck,
actually I can reproduce the same issue with your code snippet. Here the full error stack:
C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\autograd\profiler.py:440: UserWarning: CUPTI tracing is not available, falling back to legacy CUDA profiling
warn("CUPTI tracing is not available, falling back to legacy CUDA profiling")
Traceback (most recent call last):
File "test_profiler.py", line 14, in <module>
prof.step()
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 282, in step
self._start_warmup()
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 413, in _start_warmup
self.profiler = prof.profile(
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\autograd\profiler.py", line 447, in __init__
assert len(self.kineto_activities) > 0, \
AssertionError: No activities specified for Kineto profiler
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_profiler.py", line 14, in <module>
prof.step()
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 253, in __exit__
self._exit_actions()
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 404, in _exit_actions
self._start_trace()
File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 425, in _start_trace
assert self.profiler is not None
AssertionError
No, unfortunately I don’t have any debugging advice and would recommend to create an issue in GitHub, so that others could try to reproduce it as well.
import torch
import torchvision.models as models
model = models.densenet121(pretrained=True)
x = torch.randn((1, 3, 224, 224), requires_grad=True)
with torch.autograd.profiler.profile(use_cuda=True) as prof:
model(x)
print(prof)