Create a custom dataset using a sliding window from multiple sets of time series data

1

Hello, long time listener first time caller!

I’m trying to create custom a dataset from a set of time series .csv’s with accompanying ground truth .csv’s. Let’s say my folder structure looks like:

root/
    - action1/
        -sync/
            -data1.csv
            -data2.csv
            -truth1.csv
            -truth2.csv
            - ...
    - action2/
        -sync/
            -data1.csv
            -data2.csv
            -truth1.csv
            -truth2.csv
            - ...

I would like to take windows from each .csv (e.g. data1 and truth1) as my x and y to feed into my NN. I can write a function that stores the path to each .csv and stores the accompanying length. My issue is how to write the custom dataset class - if it is possible?

All the .csv’s are of unique length so I would like to interate over each .csv at a time and take len(data[i]) - windowlength sequences from each.

When defining the len, is this the TOTAL number of windows from all the data?

Any help would be greatly GREATLY appreciated. Thankyou:)