Questions about optimizers

1

Hi everyone

Its about split learning

1.I have one server and 10 clients. The server has one optimizer for his part/layers. Does each client need to have an individual/its own optimizer?

2.Previosly I had for each client one optimizer

client_model_list = [ResNet18Client(config).to(device) for client_num in range
(total_client_num)]


client_optimizer_list = [optim.SGD(client_model_list[client_num].parameters(), 
lr=0.05, momentum=0.9) for client_num in range(total_client_num)]

Now I have my own parameters and when I create one optimizer, everything is good with training.

for i, l in enumerate(p.children()):
    for m,n in enumerate(l.children()):
        if(m==8):
           ft1=n
        if(m==9):
           ft2=n
   params1 = list(ft1.parameters()) + list(ft2.parameters())
   frst_opt=torch.optim.SGD(params1, lr=0.05, momentum=0.9)

But when I make for each client one optimizer, the loss is increasing 2,5…2,7…3,5… etc

first_lst=[]
for p in client_model_list:
  u=0
  for i, l in enumerate(p.children()):
    for m,n in enumerate(l.children()):
      if(m==8):
        ft1=n
      if(m==9):
        ft2=n
    
    params1 = list(ft1.parameters()) + list(ft2.parameters())

  frst_opt=torch.optim.SGD(params1, lr=0.05, momentum=0.9)
  first_lst.append( torch.optim.SGD(params1, lr=0.05, momentum=0.9))

Where I made a mistake/ any help?