You could store all file paths to the .csv files in your Dataset
's __init__
method and load each file separately using a module operation in __getitem__
.
E.g. if file1 contains 100 sliding windows, you could check, if 100 <= index < 200
and load the window from the second file.
Have a look at this small example for a sliding window approach.