Generator feed an image

vision
1

Hi,

In diffusion models generators are used to generate samples. E.g. from OpenAI https://github.com/openai/consistency_models/blob/e32b69ee436d518377db86fb2127a3972d0d8716/cm/karras_diffusion.py#L381

I am wondering if I can feed an image as initialization to this generator?
https://pytorch.org/docs/stable/generated/torch.Generator.html#torch.Generator

Where can I find more details about this generator like the source code?

2

I don’t understand how this should work and you can read more about pseudorandom number generators e.g. here.

3

As far as I know, the torch.Generator class is the equivalent of NumPy Generator, a class used for generating random numbers.

https://numpy.org/doc/stable/reference/random/generator.html

Regarding initializing with an image, this is possible, but to get differentiated results, you may want to combine that image with some random noise. Here is one function you can use to do that:

def image_scale_rand(image: torch.tensor, scale):
    # image should be converted into a torch tensor already and normalized between 0 and 1
    # scale is a number between 0 and 1, where 0 is just the image and 1 is noise
    return image * (1 - scale) + torch.rand_like(image) * scale