# Nan Values appears after some training on mel spectrograms but when printing to see where they turn nan, they don't appear ¿?

**URL:** <https://discuss.pytorch.org/t/nan-values-appears-after-some-training-on-mel-spectrograms-but-when-printing-to-see-where-they-turn-nan-they-dont-appear/193897>\
**Category:** audio\
**Created:** [December 18, 2023, 2:42am UTC](https://discuss.pytorch.org/t/nan-values-appears-after-some-training-on-mel-spectrograms-but-when-printing-to-see-where-they-turn-nan-they-dont-appear/193897 "2023-12-18T02:42:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex\_Chung](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex_chung/32/61813_2.png) [@Alex\_Chung](https://discuss.pytorch.org/u/Alex_Chung)\
**Post date:** [December 18, 2023, 2:42am UTC](https://discuss.pytorch.org/t/nan-values-appears-after-some-training-on-mel-spectrograms-but-when-printing-to-see-where-they-turn-nan-they-dont-appear/193897/1 "2023-12-18T02:42:14Z")

</div>

I have been working on a model for voice cloning, but I can’t make it train because I get Nan values suddenly when training. Please help!!! However, when I print the L1 Loss which is the one that turns Nan because of the input, it runs more epochs! But the progress spectrogram doesn’t look good.

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/6/a/6ad17bef5ef818dbb23b061f6adca1967e0cc97e.jpeg)  
These are the Layers that cause the Nan Values:

```auto
class DecConvBlock(nn.Module):
    def __init__ (self, 
                 input_size,
                 in_hidden_size,
                 dec_kernel_size,
                 gen_kernel_size,
                 **kwargs):
        super(DecConvBlock,self). __init__ (**kwargs)

        self.dec_conv = ConvModule(
                    input_size,
                    in_hidden_size,
                    dec_kernel_size)
        self.gen_conv = ConvModule(
                    input_size,
                    in_hidden_size,
                    gen_kernel_size)
        
    def forward(self, x):
        print("Input")
        print(x)
        y = self.dec_conv(x)
        print("After first conv")
        print(y)
        y = y + self.gen_conv(y)
        print("After second conv")
        print(y)
        return x + y

```

```auto
class ConvModule(nn.Module):
    def __init__ (self, input_size, filter_size, kernel_size, **kwargs):
        super(ConvModule,self). __init__ (**kwargs)
        
        self.seq = nn.Sequential(
                    ConvNorm(input_size,filter_size,kernel_size),
                    nn.BatchNorm1d(filter_size),
                    nn.LeakyReLU(),
                    ConvNorm(filter_size,input_size,kernel_size)
                    )
    def forward(self,x):
        return self.seq(x)

```

```auto
class ConvNorm(nn.Module):
    def __init__ (self, in_channels, out_channels, kernel_size=1, stride=1,
                 padding='same', dilation=1, groups=1, bias=True, initializer_range=0.02, padding_mode='zeros'):
        super(ConvNorm, self). __init__ ()
       
        self.conv = nn.Conv1d(in_channels, out_channels,
                              kernel_size=kernel_size, stride=stride,
                              padding=padding, dilation=dilation,
                              groups=groups,
                              bias=bias, padding_mode=padding_mode)

        nn.init.trunc_normal_(
            self.conv.weight, std=initializer_range)

    def forward(self, signal):
        conv_signal = self.conv(signal)
        return conv_signal

```

---

<div class="post-metadata">

**Author:** ![hassamwazir](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hassamwazir/32/62125_2.png) [@hassamwazir](https://discuss.pytorch.org/u/hassamwazir)\
**Post date:** [February 22, 2024, 7:22pm UTC](https://discuss.pytorch.org/t/nan-values-appears-after-some-training-on-mel-spectrograms-but-when-printing-to-see-where-they-turn-nan-they-dont-appear/193897/2 "2024-02-22T19:22:42Z")

</div>

Hello Alex,

I have a similar problem. I’m not sure if we are getting it for the same reason though. I have a model that classifies sounds. It calculates the log Melspectrogram of an audio waveform inside the model using torchAudio and the result is fed to a bunch of CNN layers. After a couple of epochs, I started getting NaNs. But previously, when I was performing the same operations outside the model, i.e., in the dataset class, and using the log Melspectrograms as input to the model, it trained well.

Currently, I’ve found a workaround. If I move everything to the CPU, my training resumes without any NaNs. I do not know the cause of this, but just want to share it here in case it helps someone.

---

<div class="post-metadata">

**Author:** ![Alex\_Chung](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex_chung/32/61813_2.png) [@Alex\_Chung](https://discuss.pytorch.org/u/Alex_Chung)\
**Post date:** [June 26, 2024, 4:55pm UTC](https://discuss.pytorch.org/t/nan-values-appears-after-some-training-on-mel-spectrograms-but-when-printing-to-see-where-they-turn-nan-they-dont-appear/193897/3 "2024-06-26T16:55:38Z")

</div>

What I found in that month to overcome this was that I needed to add a normalization layer after each layer in another module. I don’t know if this was the solution, though, because I didn’t get the results I was expecting.