Sequential inputs affecting model grad values

1

I made a simple toy experiment. I want to get feedback about this… so let’s get started
Last part, I made a simple question about this situation.

  1. Hypothesis
    I made a simple training code lines like this.
    I wanted to know whether the previous input value is contaminating model grad values.
    If the previously input value contaminate model, the grad values would be changed based on the previous value.
    So I made some experiment like this.
    if the mean of single_input_history is almost the same as multi_input_history:
    it means that previous input values are considered independent on the current input.
class TestNet(nn.Module):
    def __init__(self):
        super(TestNet, self).__init__()
        self.layer = nn.Linear(1, 1)
    def forward(self, x):
        out = self.layer(x)
        return out
model = TestNet()
l1 = nn.L1Loss()

single_input_history = []
multi_input_history =[]

for _ in range(10000):
    model = TestNet()
    x = torch.tensor([1.0], requires_grad = True)
    y = torch.tensor([2.0], requires_grad = True)
#     z = torch.tensor([4.0], requires_grad = True)
    temp = model(x)
    loss = l1(temp,z)
    loss.backward()
    single_input_history.append(x.grad.data.item())


for _ in range(10000):
    model = TestNet()
    x = torch.tensor([1.0], requires_grad = True)
    y = torch.tensor([2.0], requires_grad = True)
    z = torch.tensor([4.0], requires_grad = True)
    temp = model(y)
    temp = model(x)
    loss = l1(temp,z)
    loss.backward()
    multi_input_history.append(x.grad.data.item())


print(sum(single_input_history)/len(single_input_history))
print(sum(multi_input_history)/len(multi_input_history))

is my experiment method got right? any suggestions or evaluation about this outcome would be very helpful to me.