[Newbie] Unexpected Error - Sigmoid is not a module subclass

1

Hey,

I am new to PyTorch and I hope this is the right place to ask because I am sure this question would receive downvotes on StackOverflow.
However, this Code:

import torch
import torch.nn as nn
import numpy as np

#Initializing neuron_count and batchsize
n_in, n_hdn0, n_hdn1, n_hdn2, n_hdn3, n_hdn4, n_out, batchsize = 2,200,50,50,50,50,1,100;

input_arr = []
output_arr = []

print("Initializing numbers to train with...")
for i in range(10000):
    a = np.random.randint(0,1000)
    b = np.random.randint(0,1000)
    c = a*b
    input_arr.append([a,b])
    output_arr.append(c)
print("Initialized numbers to train with...")

print("Converting to tensors...")
input_arr = torch.IntTensor(input_arr)
output_arr = torch.IntTensor(output_arr)
print("Converting to tensors...")

print("Creating model...")
model = nn.Sequential(
    nn.Linear(n_in,n_hdn0), nn.Sigmoid,
    nn.Linear(n_hdn0, n_hdn1), nn.Sigmoid,
    nn.Linear(n_hdn1, n_hdn2), nn.Sigmoid,
    nn.Linear(n_hdn2, n_hdn3), nn.Sigmoid,
    nn.Linear(n_hdn3, n_hdn4), nn.Sigmoid,
    nn.Linear(n_hdn4, n_out), nn.Linear
)

print("Created model...")

print("Setting up for backpropagation...")
criterion = torch.nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
print("Set up for backpropagation...")

for epoch in range(50):
    # Forward Propagation

    y_pred = model(input_arr)
    # Compute and print loss
    loss = criterion(y_pred, output_arr)
    print('epoch: ', epoch, ' loss: ', loss.item())
    # Zero the gradients
    optimizer.zero_grad()

    # perform a backward pass (backpropagation)
    loss.backward()

    # Update the parameters
    optimizer.step()


print("Done.")

Throws an unexpected Error:

Traceback (most recent call last):
File “/Users/niclas/PycharmProjects/AI/Multiplier.py”, line 32, in
nn.Linear(n_hdn4, n_out), nn.Linear
File “/usr/local/lib/python3.7/site-packages/torch/nn/modules/container.py”, line 53, in init
self.add_module(str(idx), module)
File “/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 173, in add_module
torch.typename(module)))
TypeError: torch.nn.modules.activation.Sigmoid is not a Module subclass

Also, I am wondering if there is a proper way to automatically pass the whole input data into a function, which will divide the tensor into many tensors with correct batch_sizes automatically? (Although I cannot imagine this is possible…)

Yours sincerely,
Niclas

2

Pass nn.Sigmoid as an instance nn.Sigmoid() and it should work:

model = nn.Sequential(
    nn.Linear(n_in,n_hdn0), nn.Sigmoid(),
    nn.Linear(n_hdn0, n_hdn1), nn.Sigmoid(),
    nn.Linear(n_hdn1, n_hdn2), nn.Sigmoid(),
    nn.Linear(n_hdn2, n_hdn3), nn.Sigmoid(),
    nn.Linear(n_hdn3, n_hdn4), nn.Sigmoid(),
    nn.Linear(n_hdn4, n_out)
)

Also, it looks like the last nn.Linear shouldn’t be there.

Once this is fixed, you’ll get another error regarding the type of your tensors, since floating point data is expected.
You could fix this by using:

input_arr = torch.tensor(input_arr).float()
output_arr = torch.tensor(output_arr).float()

If you load the data into your memory, you could use torch.utils.data.TensorDataset.
Using this Dataset, you could wrap it in a DataLoader, which will create batches using multiple workers if needed (although you won’t benefit a lot from it if your data is already in memory):

dataset = TensorDataset(input_arr, output_arr)
loader = DataLoader(
    dataset,
    shuffle=True,
    batch_size=10
)

for epoch in range(50):
    for data, target in loader:    
        # Forward Propagation
        y_pred = model(data)
        # Compute and print loss
        loss = criterion(y_pred, target)
        print('epoch: ', epoch, ' loss: ', loss.item())
        # Zero the gradients
        optimizer.zero_grad()
    
        # perform a backward pass (backpropagation)
        loss.backward()
    
        # Update the parameters
        optimizer.step()

Have a look at the Data Loading tutorial for more information.

1 Like
3

Uhhh, such a mistake… Avoidable :sweat_smile:
Thanks for the other advice, I will have a look !!! :grinning:

1 Like