I’ve found another guy who does exactly what you want to do (I think). And he also have to re-initialize the optimizer every time. Remember to resize bias as well not only weights.
He is resizing weights during training: resizing weights during training
However, as seen in the comments, you cannot only resize_ because you have to copy and paste the old weights over. Its computationally heavy but should work in your case as well. Hope it helps!