How to implement the most efficient convolution

The result of stylegan(StyleGAN - Wikipedia) is amazing, but the huge input/output channels(1024, 512, etc) make the convolution heavy and slow, how could we implement lightweight convolutions to slim the model? any advices are welcome, thank you.

Btw, there are some mobile/lightweight open sources, like MobileStyleGAN, anycost-gan, etc, but almost all these approaches are trying to decrease the channels, but unfortunately, this will also decrease the image quality, I just wonder if we could make a super lightweight convolution without decreasing the channels but still maintain similar results with StyleGAN.