Hi.
I have a hard time understanding the size of tensors and how they change shape thorughout my network.
My data consist of 1000s of .csv files which each have 1000s of data samples. Some have few samples like 10 some have 10.000 samples. Due to memory a single track(sequence/.csv file) must be loaded at a time. I would prefer to have the network train on a single timestep on a single file. But atm i will settle for just getting a basic working network up and run with my data. I have 3 folders, 1 folder with .csv files from each class. My end goal is to produce a classifier which can classify ship types from movement patterns. First running the through a CNN to extract features and then a RNN to produce the predicted classes. However i start out very simply with just 2 conv-layers.
My custom data.Dataset class is the following (shown here is from Cargo ships):
class cargoShipDataset(data.Dataset):
def __init__(self, root_dir, transform=None):
self.transform = transform
self.list_of_data_files = glob.glob(root_dir) #Load list of all filenames in directory
self.len = len(self.list_of_data_files)
def __getitem__(self, index):
single_track = torch.from_numpy(genfromtxt(self.list_of_data_files[index], delimiter=',')[1:,:]) #a single track is loaded.
target = torch.ones(single_track.size(0)) # Class Cargo = 1 ? Class Passenger = 2 ?
#print(single_track)
return (single_track, target)
def __len__(self):
return self.len
My network (stolen from https://github.com/MorvanZhou/PyTorch-Tutorial/blob/master/tutorial-contents/401_CNN.py):
It inputs a timeseries with three channels (latitude, longitude and timestamp).
class CNN(nn.Module):
def __init__(self):
super(CNN,self).__init__()
self.conv1 = nn.Sequential(
nn.Conv1d(
in_channels=3, #Inputs time series, with Latitude, Longitude and Timestamp
out_channels=16, #Some random number
kernel_size=5,
stride=1,
padding=2,
),
nn.ReLU(),
nn.MaxPool1d(kernel_size=2), #size after pooling?
)
self.conv2 = nn.Sequential(
nn.Conv1d(
in_channels=16,
out_channels=8,
kernel_size=5,
stride=1,
padding=2,
),
nn.ReLU(),
nn.MaxPool1d(kernel_size=2),
)
self.out = nn.Linear(8, 3) #output 3 classes: Cargo, Passenger or Fishing
def forward(self,x):
#print(x.size())
x = self.conv1(x)
#print(x.size())
x = self.conv2(x)
#print(x.size())
output = self.out(x)
return output
My train loop:
cnn = CNN()
#print(cnn)
optimizer = torch.optim.Adam(cnn.parameters(), lr=0.0001) # optimize all cnn parameters
loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted
for epoch in range(1):
for step, data in enumerate(train_set_cargo): # gives batch data, normalize x when iterate train_loader
#print(data)
b_x = data[0] #here b_x is a tensor with size [sequence length, 3] Where the values are: lat, lon, time.
#print(b_x)
b_y = data[1] #b_y is a tensor with size [sequence length, 1] of simply ones.
#print(b_y)
output = cnn(b_x) # cnn output
loss = loss_func(output, b_y) # cross entropy loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
The problem is that b_x is of size [sequence length , 3] when printing it pre output = cnn(b_x). However, when printing it inside the Network code, pre x = self.conv1(x), the size is suddenly [1, sequence length, 3].
Since my network is set to convolve 1D over 3 channels is returns error cause it gets “sequence length” channels. See error below.
My traceback error snippet:
Traceback (most recent call last):
File "<ipython-input-14-af3912e69f8b>", line 1, in <module>
runfile('C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py', wdir='C:/Users/AWESOME-O 4000/Desktop/Speciale/Data')
File "C:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 678, in runfile
execfile(filename, namespace)
File "C:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 106, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py", line 107, in <module>
output = cnn(b_x) # cnn output
File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "C:/Users/AWESOME-O 4000/Desktop/Speciale/Data/NN.py", line 87, in forward
x = self.conv1(x)
File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "C:\Anaconda3\lib\site-packages\torch\nn\modules\container.py", line 91, in forward
input = module(input)
File "C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "C:\Anaconda3\lib\site-packages\torch\nn\modules\conv.py", line 176, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [16, 3, 5], expected input[1, 1429, 3] to have 3 channels, but got 1429 channels instead
I am very new to pytorch (still havn’t done a single network yet). And have a hard time understanding when tensors change size. Once this is fixed my next problem is that my sequences have different length, but this i will fix later 
Any help/insight is greatly appreciated. I feel like its most likely an error stemming from my custom data.Dataset or the network/training loop. Which is why i have that code pasted in.
My plan is to do 3 data.Dataset classes, one for each class. And then concanate them into one data set.
If any information is missing let me know and i will provide as best as i can.
regards
)