I have 3D dataset(1000 files) in .mat format. and I wish to load for training. can anyone please suggest to me how to do this?
You could use
scipy to load the
mat files as shown here.
Hi. @ptrblck I created the data loader by using the referenced function. but my data is not divided into batches while training.(mat file contain: complex128)
(one thing I don’t understand I am unable to define batch size greater than path length.)
import scipy.io as io
def init(self, mat_paths,col_name,transform):
self.paths = mat_paths
self.matX = io.loadmat(self.paths)[col_name] self.transform = transform def __getitem__(self, index): X = self.matX[index].astype(np.float32) X = torch.from_numpy(X) X = abs(X) return X def __len__(self): return len(self.paths)
mat_paths = (r’C:\Users\Abhi\Downloads\dataset\outputs\images\train1.mat’)
trainDataset = MyDataset(mat_paths,‘train4’,transform=transform)
trainLoader = DataLoader(trainDataset, batch_size=32, shuffle=True, num_workers=0, pin_memory=True,drop_last=True)
self.matX is a numpy array containing all samples in
If that’s the case, you might need to return
self.matX.shape in the
__len__ function instead of the length of
PS: you can post code snippets by wrapping them into three backticks ```, which makes your code easier to debug.
@ptrblck it works. thankyou.
okay, I will wrap my code in three ‘’’ before posting.