Soo soz but I’m not NLP so I’d need further context 
What does the BERT Classifier model output?
Aka, this shape corresponds to? batch,probabilities,seq_length¿
[68, 450, 1024]
Sounds like you have a batch of 68 elements and you have a binary score for each (which sounds ok)
The problem you seem to have is your model is returning some features which doesn’t match a binary prediction. I would say you need to add few more layers to convert those ¿features? into a binary pred