dense_to_sparse does not break the gradient. corr_layer not updated at all?
dense_to_sparse
corr_layer