Are model forward pass asynchronous?

Let’s say we have a module list
model_list = torch.nn.ModuleList([torch.nn.Linear(100, 1) for i in range(100)])
model_list = model_list.cuda()
x = torch.ones(1,100).cuda()
And we do

for i in range(100):

Will the forward pass of the 100 models be asynchronous? It seems very slow if the the models do forward pass one after another.

Multiprocessing could be used for asynchronous computations. But, the Linear with out_feature=100 should be used with splitting the output.