Expected 4-dimensional input for 4-dimensional weight [8, 1, 10, 2], but got 5-dimensional input of size [32, 1, 1, 128, 81] instead

audio
1

Hi there
I am using below function for feature extraction. Using mfcc and melspectrograms in my model. Both outputs should be of similar dimension. There is not problem with mfcc alone but I am not getting why model is taking 5 dimensions inmelspectrogram. Can anyone figure It out plz
Regards

class FeatureExtractor(object):
    def __init__(self, rate):
        self.rate = rate

    def get_features(self, features_to_use, X):
        X_features = None
        accepted_features_to_use = ('mfcc', ''melspectrogram')
        if features_to_use not in accepted_features_to_use:
            raise NotImplementedError("{} not in {}!".format(features_to_use, accepted_features_to_use))
        
        if features_to_use in ('mfcc'):
            X_features = self.get_mfcc(X,26)
      
        if features_to_use in ('melspectrogram'):
            X_features = self.get_melspectrogram(X)
      
        return X_features

  


    def get_mfcc(self, X, n_mfcc=13):
        def _get_mfcc(x):
            mfcc_data = librosa.feature.mfcc(x, sr=self.rate, n_mfcc=n_mfcc)
            
            return mfcc_data

        X_features = np.apply_along_axis(_get_mfcc, 1, X)
        return X_features

   

    def get_melspectrogram(self, X):
        def _get_melspectrogram(x):
            mel = librosa.feature.melspectrogram(y=x, sr=self.rate, n_fft=800, hop_length=400)[np.newaxis, :]
            delta = librosa.feature.delta(mel)
            delta_delta = librosa.feature.delta(delta)
            out = np.concatenate((mel, delta, delta_delta))
            return mel

        X_features = np.apply_along_axis(_get_melspectrogram, 1, X)
        return X_features

Here the shapes of X is (5984, 32000)

2

@ptrblck sir please have a look in this code,
Regards

3

I’m unsure how this shape is created, but would guess that the transformations are creating it.
Could you add print statements before and after each transformation and check the shapes?

4

In case of mfcc

mfcc_data.shape=(26,63)
print(X_features.shape)=(8,26,63)

in melspectrogram

print(mel.shape)=(1,128,81)
print(X_features.shape)=(8,1,128,81)

The shape of both the features must be same.

5

Sir actually that [np.newaxis, :] is creating that extra axis
removing it giving both the shapes same. I am really sorry for this,
But I am confused what is the purpose of this
[np.newaxis, :] ??

6

The purpose of array[np.newaxis] is to add a new dimension in the specified dim, so you are right in removing it in case it’s not needed.