How to load all the .nii from the directory without augmentation using pytorch dataloader?

Actually I am making data loader for MRI images collected from ADNI. I loaded a single image from training folder now I want to load all the MRI images as it is, in a iterative way and than apply some neural network for classification purposes.
Please help me that how you load your whole MRI data from the directory
I have 900 MRI images in three different folder i.e Alzheimer have three main classes
CN, MCI, AD so I want to load all the data from each folder but how I to do?
Further more I read 1000 post and tutorial but I couldn’t get an idea to implement as I am not much expert in pytorch and 3D data handling.
I am using following IDE and libraries
IDE- Spyder
using Pytorch and tensorflow
python 3.7
Thanks in advance

Have you checked the below tutorial on data loading?
https://pytorch.org/tutorials/beginner/data_loading_tutorial.html

thanks for your reply mailcorahul
ye i checked already but couldn’t get idea how to load my MRI .nii file, but here I am talking about 3D and 4D data which is related to medical image analysis. I am stuck with dataloading step as I have 17GB of data which is in the form of nii file not any simple image file. I want to load them all in iterative way into my memory and after that give as input for my CNN.

You will need a custom Dataset.

How I use the custom for my use case (I assume you use nibabel to handle nifti files), I preload all the file handle with nibabel.
During the iteration you could get the data with the np.asarray(nii_image.dataobj). If you use get_data() you should specify the caching by default it keeps every loaded image in cache this could overload you system memory by loading to many image.

class MyCustomDataset(Dataset):
    def __init__(self, path):
        # load all nii handle in a list
        self.images_list = [nib.load(image_path) for image_path in path]

    def __len__(self):
        return len(self.images_list)

    def __getitem__(self, idx):
        nii_image = self.images_list[idx]
        data = torch.from_numpy(np.asarray(nii_image.dataobj))
        # find how to retrieve the target
        return data, target

You will need a ways to associate the target of your __getitem__ function, this part depend on how you will handle the data/directory, etc.

thanks czotti for quick reply
Yes dude i check the mentioned links and also create the same code which is given in the link and you provided here below is

class Dataloder_img(data.Dataset):
    def __init__(self,root_dir,transforms ):
        self.root_dir = root_dir
        self.seg_dir = seg_dir
        self.transforms = transforms
        self.files = os.listdir(self.root_dir)
        self.lables = os.listdir(self.seg_dir)
        print(self.files)
    
    def __len__(self):
        return len(self.files)
    
    def __getitem__(self,idx):
        img_name = self.files[idx]
        label_name = self.lables[idx]
        img = nib.load(os.path.join(self.root_dir,img_name)) 
        #change to numpy
        img = np.array(img.dataobj)
        #change to PIL 
#        img = Image.fromarray(img.astype('uint8'), 'RGB')    
        print(img.size)
        
        label = nib.load(os.path.join(self.seg_dir,label_name))
        #change to numpy
        label = np.array(label.dataobj)
        #change to PIL 
#        label = Image.fromarray(label.astype('uint8'), 'RGB')
        
        print(label.size)
        
        if self.transforms:
            img = self.transforms(img)
            label = self.transforms(label)
            return img,label
        else:
            return img, label
full_dataset = Dataloder_img('C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1', 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1/ADNI_136_S_0300_MR_MPR__GradWarp__B1_Correction__N3__Scaled_Br_20080529142830882_S50401_I107759.nii' ,tfms.Compose([tfms.RandomRotation(180).tfms.ToTensor()]))
                                   

train_size = int(0.8 * len(full_dataset))
val_size = len(full_dataset) - train_size
train_dataset, val_dataset = torch.utils.data.random_split(full_dataset, [train_size, val_size])
train_loader = data.DataLoader(train_dataset,shuffle=False,batch_size=bs)
val_loader = data.DataLoader(val_dataset,shuffle=False,batch_size=bs)

test_img, test_lb = next(iter(full_dataset))
print(test_img[0].shape)
plt.imshow(test_img[1])
plt.show()

But its doesn’t work for me as
the error is shown below

full_dataset = Dataloder_img('C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1', 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1/ADNI_136_S_0300_MR_MPR__GradWarp__B1_Correction__N3__Scaled_Br_20080529142830882_S50401_I107759.nii' ,tfms.Compose([tfms.RandomRotation(180).tfms.ToTensor()]))

AttributeError: 'RandomRotation' object has no attribute 'tfms'

Here when I remove this error than nothing is working except error.
Please help in this regard that how I can get my all the data which are in my directory.
Thanks in advance

I tried this method without using dataset class but still its have error. Actually I have not much expertise in coding side but I trying to be a good programmer. Here is the code for another method which I am using.

cwd = 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1'
#data_dir = 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/test/0/ADNI_137_S_0800_MR_MPR__GradWarp__N3__Scaled_Br_20081024125416989_S56178_I123506.nii'


#cwd = os.getcwd()#return current working directory
print(cwd)
#debugger
patient_id = os.listdir('C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1') #create a list that encompasses all the images we downloaded
len(patient_id)
print(patient_id)
#

#filename = glob.glob('C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1/ADNI_137_S_0800_MR_MPR__GradWarp__N3__Scaled_Br_20081024125416989_S56178_I123506.nii' + 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1/ADNI_137_S_0800_MR_MPR__GradWarp__N3__Scaled_Br_20081024125416989_S56178_I123506.nii')
#print(filename)    

dataimage = [patient_id]#[patient_id, image_matrix_normalized]
for patient in patient_id:
    for root, dirs, files in os.walk(cwd + "/ADNI/" + patient):
        flag = 0 
        for file in files:
            if file.endswith(".nii"):
                print(os.path.join(root, file))
                if flag < 1:
                    datapath.append(os.path.join(root, file))
                    print(os.path.join(root, file))
                    print(flag)
                    flag = flag + 1
datapath = [patient_id, 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1'] #[patient_id, path]
tar_dim = [256, 256, 160]

for path in datapath:
    mri = nib.load(cwd).get_data()
    mri = (mri - mri.min())/(mri.max() - mri.min())
    mri = mri - mri.mean()
    tar_mri = ndimage.zoom(mri,[tar_dim[0]/mri.shape[0],tar_dim[1]/mri.shape[1], tar_dim[2]/mri.shape[2]], order = 1)

    print(mri)
    print(mri.mean())
    print(mri.var())
    print(tar_mri.shape)
    plt.imshow(mri[125,:,:],'gray')
    plt.show()
    
    dataimage.append(tar_mri)
    

after running this code I got below error which belongs to unsupported files

ImageFileError: Cannot work out file type of "C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1"

Please any one who already work with Nii file format help me here?

For the first error:

AttributeError: ‘RandomRotation’ object has no attribute ‘tfms’

It’s a typo ., :

tfms.Compose([tfms.RandomRotation(180), tfms.ToTensor()])

Also your __init__ functione have 2 parameters (self is used by python and represent the object instance) root_dir and transforms.
On the line full_dataset = Dataloader_img(...) you provides 3 parameters, the directory, the nii file and transform object. At least you need to removes the nii file.

For your second error it’s normal you tried to use a directory as input for nib.load(cwd) replace this with a .nii file and you should be fine.

still its not working I am talking about the 2nd pecide of code.
as it has generated the error of not handling this type of file from its built in files.
have a look on this screenshot. I change the mentioned and replace with nii file but it show below error

As the error reports it didn’t load the .nii file but still the C:\Users\....\train\1 directory.

In your exemple you should changes these lines:

datapath = [patient_id, 'C:/Users/Ali ktk/.spyder-py3/dataloader/data/train/1'] #[patient_id, path]

into

datapath = patient_id

And to load the images properly with nibabel change

mri = nib.load(cwd).get_data()

into

mri = nib.load(path).get_data()

BTW all these questions are not related to pytorch at all maybe it’s more suited to go on the nibabel forum/gitter/whatever they use to communicate, to learn how to properly load medical imaging data

Thanks Czotti I am trying and you are right that these issue related with nibable
forum,
actually I want to discuss it here because millions of user are now turning to pytorch. If they have experience with medical imaging than they will solve my problem here more in suitable way
thanks for your valuable time

I am still stuck in the same problem.
Let me know that
is I load MRI 3D image having 256,256,166 dimension directly without patching tham or resampling tham.
My target in to load all of tham without any pre-processing steps or patching to the input.
Is it possible>>>???
furthermore If I am wrong please mentioned the steps and guide me please to get out from here.
Thanks in advance
Clement Zotti and Raghul Asokan