Instance Segmentation Using Mutex Watershed

[Assefa_Seyoum]

Occluded segments of the same instance assigned different IDs after Mutex Watershed

I am doing the instance segmentation of cables by predicting affinities then post-processing the result using mutex watershed.

However, I couldn’t figure out how to use the mutex watershed algorithm to model model occlusion.

A short explanatory example can be found at Mutex-Watershed-Instance-segmentation.

Use two types of neighborhoods / offsets:

1. Offset 1

offsets = [
    [0, -1, 0], [0, 0, -1],
    [0, -1, -1], [0, 1, 1], [0, -1, 1], [0, 1, -1],
    [0, -9, 0], [0, 0, -9],
    [0, -9, -9], [0, 9, -9], [0, -9, -4],
    [0, -4, -9], [0, 4, -9], [0, 9, -4],
    [0, -27, 0], [0, 0, -27], [0, -27, 27]
]

2. 4 neighborhoods at distance of {1, 2, 4, 8, 32, 64} (Similar to Affinity Derivation and Graph Merge for Instance Segmentation).
   i.e.,

1. On CVPPP Plant dataset

Input Image:

Output After Affinity and Mutex Watershed:

Note that the mutex watershed perfectly reconstructs the ground truth.

2. On Cable dataset

Input Image:

Output After Affinity and Mutex Watershed:

Disjoint parts of the same cable are not connected after mutex watershed even though the area of the kernel / neighborhood (64) is larger than the width of a cable (10-15px)

Zoomed-in view of the disconnected segments

Why does this happen, and how to correct it?

Thanks.

[ptrblck]

The issue doesn’t seem to be PyTorch-related, so I would recommend to ask the question in a CV-related discussion board or the authors in the repository.