Are model forward pass asynchronous?

1

Let’s say we have a module list
model_list = torch.nn.ModuleList([torch.nn.Linear(100, 1) for i in range(100)])
model_list = model_list.cuda()
x = torch.ones(1,100).cuda()
And we do

for i in range(100):
model_list[i](x)

Will the forward pass of the 100 models be asynchronous? It seems very slow if the the models do forward pass one after another.

2

Multiprocessing could be used for asynchronous computations. But, the Linear with out_feature=100 should be used with splitting the output.