Transforms aren't the same for images and masks

This is expected if you call into random transformations separately with your image and mask.
Take a look at this post which shows how the functional API can be used to transform both objects using the same “random” parameter.