Pytorch for vfx

1

Hi all,

In new to this forum so forgive me if I ask any stupid questions. I come from the film visual effects industry and recently one of the main vfx softwares - Nuke from the Foundry added the ability to use pytorch files. There isn’t much further information on this at present so im just reaseaching and trying to find out any pytorch models that could be useful for visual effects and image manipulation. If you have any recommendations please post below

thank you!

2

I’m not familiar with this software stack, but some ideas which models might be useful:

  • segmentation models e.g. to blur content/remove it
  • super-resolution models
  • any kind of “filters” e.g. changing the appearance of the scene/person (e.g. different color of the clothing, hair etc.)
  • super-slowmotion
  • image/video stabilization
  • Generative models to create new scenes
  • denoising or general image enhancement (e.g. “night mode” enhancement)
3

Slow motion retimes would be very useful, depth maps, any face manipulation would be great. If you know of anything that you thibk could be of interest please paste links im still new and finding it difficult to find where to look

4

I would probably just search for (updated) GitHub repositories and check if you could use these models. E.g. for SuperSlomo I’ve found this repo, which is quite outdated by now so I would expect that some code changes are needed.