Hello, long time listener first time caller!
I’m trying to create custom a dataset from a set of time series .csv’s with accompanying ground truth .csv’s. Let’s say my folder structure looks like:
root/ - action1/ -sync/ -data1.csv -data2.csv -truth1.csv -truth2.csv - ... - action2/ -sync/ -data1.csv -data2.csv -truth1.csv -truth2.csv - ...
I would like to take windows from each .csv (e.g. data1 and truth1) as my x and y to feed into my NN. I can write a function that stores the path to each .csv and stores the accompanying length. My issue is how to write the custom dataset class - if it is possible?
All the .csv’s are of unique length so I would like to interate over each .csv at a time and take len(data[i]) - windowlength sequences from each.
When defining the len, is this the TOTAL number of windows from all the data?
Any help would be greatly GREATLY appreciated. Thankyou:)