Project: Unsupervised learning of Object Landmarks through Conditional Image Generation

1

Hi PyTorch community,

I’d like to announce a working PyTorch implementation of the paper Unsupervised learning of Object Landmarks through Conditional Image Generation

This allows the generation of keypoints on 2D images, using unsupervised learning with no labels! Works for both still images and video. (Possibly easier/better on video).

If you are at all interested in unsupervised learning, it’s a great paper. One of the best papers on unsupervised learning IMHO, due to the simplicity of the technique.

Check it out at …

I also implemented transporter network from Unsupervised Learning of Object Keypoints for Perception and Control and can confirm that it also works very well on Atari images.

Here are a few of the results.

image
image2560×2048 785 KB

Enjoy!

Duane

2

Thanks for sharing this Duane! :slight_smile:
I haven’t touched a lot of unsupervised methods yet, so this seems to be a great starter!

3

Yeah, I think it is. Super simple architecture, no adversarial losses, works with a simple reconstruction loss.

There were really 2 tricks to make it work.

  1. The data augmentation requires some tuning. Too much distortion and you don’t learn, not enough and there is no contrast or prediction gradient to learn.
  2. For still images, After distorting the input, you have to “loss mask” the parts of the image that fall outside the boundaries of the original image. Otherwise the algorithm “cheats” by using the boundary to learn the parameters of your distortion!