RuntimeError: mat1 and mat2 must have the same dtype

Your code is still not executable and not properly formatted.
However, it seems you have solved the issue by transforming everything to float32.

This shape mismatch is raised if the in_features of the linear layer do not match the feature dimension of the input activation, so use in_features=8568 and it should work.