Make patches from images with their masks and reconstruct those images

vision
5

Dear @ptrblck,

For this question, I have written a python code that given an image, stride, and filter size, I will extract overlap patches from each image. Now in my custom Dataset, in __init__, I will load the image folder. Instead of returning the path of images, I will return a list of NumPy array that has the patches. Also, I will return the name of the image that each patch is extracted from, and the id of patches. In below, I mention code to understand a bit more.

Question: How are you determining the number of batches and how do you store it with the image name?

Answer: CustomDataset:
init(path_to_image, patch_size, stride):
for each image in path_to_image :

  1. Run the extract_patches.py
  2. Add this patches(numpy arrays) to a list
  3. Add the Name of the Image
  4. Add Patch ID

at the end, return the lists. Then in __getitem__, I just return the list[index].

The Second Question:

Question: Would you like to create batches with patches from mixed images or should a batch only contain patches from the same image? In the latter case: should these patches from the same image be shuffled or do you want a specific order?

Answer: No, there is no need to be the same image, but I want to know which patch belongs to which image and be able to reconstruct the whole image.

I have another question too. let’s assume for each image, I have two labels {Mask, Edge} which are images too. The output of the model is the mask and based on the mask I am going to extract the patches. I want to write a custom Loss Function. Based on other discussions, as long as I am working with Variables, I can write the function.
But in my case, because I have to calculate the edge based on mask (which needs some filtering and …) I have to access the .data() of Variable, so I am a bit confused about how to do that. Can you help with this part too? The loss should be in this format:

CustomLoss(Model_Output, TruthMask, TruthEdge):

  1. extract data by Model_Output.data().cpu()
  2. run edge_extraction.py on the previous data
  3. calculate the difference between truth and predicted {edge, mask}

Thanks in advance.