How to finetune a model after pruning?

I used prune.global_unstructured() to prune my CNN model. The Cifar10 dataset testing accuracy dropped from 85.6 to 82.9. Can I finetune the model to increase the accuracy?

I freeze the zero weight in the pruned model but I found that the loss will diverge after several training batches.

Does anyone know how to fix this?