Beginner question here: I have a small simple fully connected network (1-3 layers, 50 hidden nodes) that is training properly, but I’m getting different results with small changes in input data. The network is deterministic (I have seeded everythig properly, different runs in same data yield exact same result) so that’s not it.
How can I train my simple network to be more robust? I have tried the simple obvious things like playing around with batch size and number of hidden nodes. What else can I look into? Thanks.