Help me to Implement BYOL

Hi, Im a master’s student in South Korea.

Im implementing BYOL in PyTorch-lightning style and testing using CIFAR10 dataset and ResNet-18.

Im using the official SimSiam and MoCo repository as baseline.

My BYOL’s performance of linear classification is low (maximum 69.3), but I cannot find any reason why the performance is low.

Can anyone help me?

I have attached my GitHub repository and my some experiment below.

Repository: GitHub - comojin1994/byol: Bootstrap Your Own Latent A New Approach to Self-Supervised Learning


Screen Shot 2022-04-13 at 4.03.22 PM

Screen Shot 2022-04-13 at 4.03.46 PM

Screen Shot 2022-04-13 at 4.03.58 PM

singular value