Number of initial input channels

Hello everyone,
I have a question about the number of initial input channels.
Currently the number of channels in my cnn looks like this for the Conv2D layers:
data input 2 → 32 → 64 → 128 → 256

I get quite good results with this. When I increase the number of initial input channels (from 2 to 4 for example since I have a lot of different variables from a climate model) I don’t see any real improvement in the classification. Can I draw the conclusion that the additional parameters don’t contribute in any meaningful way, or might it have something to do with the number of network channels? I just want to be sure that I don’t mess things up.