Hello. I am trying to prepare some data for training and testing.
I am running in a problem as most of the tutorials i’ve seen are built around image datasets but i’ve got some custom data that i want to use for the Neural Network now…
Explanations
Basically, let’s say i have 5 CSV files (data1,2,3,4,5) with data and 5 CSV files (state1,2,3,4,5) with states.
The network should use the corresponding data file to guess the value that is written in the state file.
The data files contain 2 columns, X and Y (timeseries) of 100 points each.
The state files only contain 1 column and 1 data, (kept or not kept (0 or 1))
The issue i have now
I can succesfully create a dataloader and input tensors that i create after the CSV files.
I’ll get 3 tensors, 1 for DataXvalues, 1 for DataYvalues, 1 for State
I create a dataLoader like this
myDataLoader = torch.utils.data.DataLoader((myXTensor,myYTensor,myStateTensor))
And it seems to work good, but…
But i only have the stuff i need for 1 train or test pass now, and i don’t see how im supposed to fit the 4 other sets i should add next! (Data2,3,4,5 and State2,3,4,5)
When i lookup my DataLoader the data in it is somewhat like this
[0] - Data for X
[1] - Data for Y
[2] - State
Should i just add a lot of tensors and i could itterate doing jumps of 3 with other sets of data (is that a setting in the DataLoader?) (I would do one pass with Input 0,1 Output 2 - Next pass with Input 3,4 Output 5 - The next one with 6,7,8…)?
If there’s more info required i’ll give all the details i can!