1D Wasserstein DCGAN size error

1

Hi all!
I keep getting this error:

RuntimeError: Calculated padded input size per channel: (3). Kernel size: (4). Kernel size can’t be greater than actual input size

I understand the error. I did all the calculations of the convolutions and transpose convolutions. There cannot be any input size channel 3. I don’t understand.

Here is my code:

class Discriminator(nn.Module):
def init(self, channels_img, features_d):
super(Discriminator, self).init()
self.disc = nn.Sequential(
# input: 1x1x4096
nn.Conv1d(1, 32, kernel_size=4, stride=2, padding=1), # 1x32x2048
nn.LeakyReLU(0.2),
# _block(in_channels, out_channels, kernel_size, stride, padding)
self._block(32, 64, 4, 2, 1), # 1x64x1024
self._block(64, 128, 4, 2, 1), # 1x128x512
self._block(128, 256, 4, 2, 1), # 1x256x256
nn.Conv1d(256, 1, kernel_size=256, stride=1, padding=0), # 1x1x1
)

def _block(self, in_channels, out_channels, kernel_size, stride, padding):
    return nn.Sequential(
        nn.Conv1d(
            in_channels,
            out_channels,
            kernel_size,
            stride,
            padding,
            bias=False,
        ),
        nn.InstanceNorm1d(out_channels, affine=True),
        nn.LeakyReLU(0.2),  
    )

def forward(self, x):
    return self.disc(x)

class Generator(nn.Module):
def init(self, channels_noise, channels_img, features_g):
super(Generator, self).init()
self.net = nn.Sequential(
# Input: 1x1x256
self._block(256, 256, 256, 1, 0), # 1x256x256
self._block(256, 128, 4, 2, 1), # 1x128x512
self._block(128, 64, 4, 2, 1), # 1x64x1024
self._block(64, 32, 4, 2, 1), # 1x32x2048
nn.ConvTranspose1d(32, 1, kernel_size=4, stride=2, padding=1),
# Output: 1x1x4096
nn.Tanh())

def _block(self, in_channels, out_channels, kernel_size, stride, padding):
    return nn.Sequential(
        nn.ConvTranspose1d(
            in_channels,
            out_channels,
            kernel_size,
            stride,
            padding,
            bias=False,
        ),
        nn.BatchNorm1d(out_channels),
        nn.ReLU(),
    )

def forward(self, x):
    return self.net(x)

Thank you!!

2

The error points to the spatial size of an activation, which is 3 and thus smaller than the spatial size of a the conv kernel, which is 4.
Check the shapes of all intermediate activations and make sure the spatial size doesn’t become smaller than the kernels.

1 Like
3

Hi ptrblck!
I noticed the one causing the error in the training section is this line:

        critic_real = critic(data).reshape(-1)

My input from real data is tensor of [16, 1, 1,] and 16 is the batchsize. The one from generator has [1,1,4096]. So I just had to squeeze and unsqueeze the real data tensor to [1,1,16].
But, now I got a different error:

ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 128, 1])

Trying to figure that out right now.

4

The new error is most likely raised by a batchnorm layer, which cannot calculate the batch statistics from a single sample (or element in this case). Increase the batch size during training and it should work.

1 Like
5

I fixed it. I am also using instancenorm. Now I got another type of error that I’ve never come across before. I am currently working on it. Note that my data is 1D vector.

save_handler = SAVE[format.upper()]

KeyError: ‘PNG’

This function causes the error:

def save(self, fp, format=None, **params):
“”"
Saves this image under the given filename. If no format is
specified, the format to use is determined from the filename
extension, if possible.

    Keyword options can be used to provide additional instructions
    to the writer. If a writer doesn't recognise an option, it is
    silently ignored. The available options are described in the
    :doc:`image format documentation
    <../handbook/image-file-formats>` for each writer.

    You can use a file object instead of a filename. In this case,
    you must always specify the format. The file object must
    implement the ``seek``, ``tell``, and ``write``
    methods, and be opened in binary mode.

    :param fp: A filename (string), pathlib.Path object or file object.
    :param format: Optional format override.  If omitted, the
       format to use is determined from the filename extension.
       If a file object was used instead of a filename, this
       parameter should always be used.
    :param params: Extra parameters to the image writer.
    :returns: None
    :exception ValueError: If the output format could not be determined
       from the file name.  Use the format option to solve this.
    :exception OSError: If the file could not be written.  The file
       may have been created, and may contain partial data.
    """

    filename = ""
    open_fp = False
    if isPath(fp):
        filename = fp
        open_fp = True
    elif isinstance(fp, Path):
        filename = str(fp)
        open_fp = True
    if not filename and hasattr(fp, "name") and isPath(fp.name):
        # only set the name for metadata purposes
        filename = fp.name

    # may mutate self!
    self._ensure_mutable()

    save_all = params.pop("save_all", False)
    self.encoderinfo = params
    self.encoderconfig = ()

    preinit()

    ext = os.path.splitext(filename)[1].lower()

    if not format:
        if ext not in EXTENSION:
            init()
        try:
            format = EXTENSION[ext]
        except KeyError as e:
            raise ValueError(f"unknown file extension: {ext}") from e

    if format.upper() not in SAVE:
        init()
    if save_all:
        save_handler = SAVE_ALL[format.upper()]
    else:
        save_handler = SAVE[format.upper()]

    if open_fp:
        if params.get("append", False):
            # Open also for reading ("+"), because TIFF save_all
            # writer needs to go back and edit the written data.
            fp = builtins.open(filename, "r+b")
        else:
            fp = builtins.open(filename, "w+b")

    try:
        save_handler(self, fp, filename)
    finally:
        # do what we can to clean up
        if open_fp:
            fp.close()
6

Alright. Solved it. For anyone following the WGAN from Aladdin Persson on YouTube, he does the WGAN for images and obviously he adds the results on tensorboard using writer.add_image(). But my data is 1D. That’s why I was getting this weird error. I needed to change that to writer.add_scalar(). For anyone who got this kind of error, most likely this is the source of the error.
Best.