Split learning/ problem with trainig with different datasets

Hi everyone
I have 5 clients, one server and one cut layer between them (vanilla split learning)

In the first case, each client becomes a whole cifar10 dataset and each client have one optimizer. Training is running perfectly.

In another case, I divided the dataset into five pieces(each has 2 classes) and each client becomes one piece.
On the training, only the first client will be trained and another four clients not (accuarcy : 0.0).
If I skip the first client at the beginning of training and train client 2,3,4,5, only client nr 2 will be trained.
In the second, third etc epoch all clients have accuracy 0.
I already tried many stuffs, like to make for each client-model optimizer on the server side, but didn’t help

Any hint,help?