I see. After googling for awhile, I thought the solution might be something like this
(How to add new nodes at the last layer (fully connected layer)? - #4 by MariosOreo),
where we pad the old model input weights to the shape of the new model input weights with random values and then load_state_dict.
or
maybe replacing the first linear layer of the new model with new input shape, and then assign its weights with pretrained weights like so (How to transfer the pretrained weights for a standard ResNet50 to a 4-channel - #2 by ptrblck)
Can you have a look? and let me know your thoughts. Thank you very much.