How to add noise to MNIST dataset when using pytorch

Thank you for the prompt response!

Hi ptrblck,
recently i came across Federated learning with Differential privacy, which is adding noise.
def train(args, model_bob, model_alice, device, federated_train_loader, epoch):

def train(args, model_bob, model_alice, device, federated_train_loader, epoch):
optimizer_bob = optim.SGD(model_bob.parameters(), lr=args.lr)
optimizer_alice = optim.SGD(model_alice.parameters(), lr=args.lr)
model_bob.train()
model_alice.train()
for batch_idx, ((input_bob, target_bob), (input_alice, target_alice)) in enumerate(zip(data_bob, data_alice)):

    input_bob, target_bob, input_alice, target_alice = input_bob.to(device), target_bob.to(device),\
                                                       input_alice.to(device), target_alice.to(device)
    optimizer_bob.zero_grad()
    optimizer_alice.zero_grad()
    # print(input_bob.size())
    output_bob = model_bob(input_bob)
    output_alice = model_alice(input_alice)
    loss_bob = F.nll_loss(output_bob, target_bob)
    loss_alice = F.nll_loss(output_alice, target_alice)
    loss_bob.backward()
    loss_alice.backward()
    for pram_bob, pram_alice in zip(model_bob.parameters(), model_alice.parameters()):
        # print(pram_bob.grad.size())
        grad_avg = (pram_bob.grad + pram_alice.grad) / 2.0
        grad_avg = grad_avg + torch.rand_like(grad_avg) * 0.0173
        pram_bob.grad = grad_avg
        pram_alice.grad = grad_avg
  
    optimizer_bob.step()
    optimizer_alice.step()
   
    print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss_bob: {:.6f}\tLoss_alice: {:.6f}'.format(
        epoch, batch_idx * args.batch_size, len(data_bob) * args.batch_size,
        100. * batch_idx / len(data_bob), loss_bob.item(), loss_alice.item()))

This was the train function.
Do you know what they are doing to gradients here?
There are averaging and doing some calculation which i wasn’t able to understand.

I’m unfortunately not familiar enough with federated learning approaches and don’t know how the noise addition was calculated or why the gradients are averaged in the first place.
Assuming you were using this code from a source code repository, you might want to ask the authors of the implementation (and share the response here if possible :wink: ).