I have a MDNet with 3 conv layers and a few fully-connected layers afterwards, and I want to do feature selection after the third conv layer, because there’s some redundancy (the net is doing tracking for a single object, thus it’s like a two class classification - background and object).
I have the feature vector (of length 4608. actually it’s [batch_size, 4608]) from 512 feature maps and I want to do a feature selection, and choose only 256 feature maps with low MI between themselves.
From the mutual information I should get a 512X512 matrix like described in the bottom of page 5 and do the feature selection.
The mutual information between
two feature maps is estimated using histograms of them. The
activation values are distributed in 20 bins and the we can
get probabilities for each bin. The mutual information of any
two feature maps are calculated and these values are stored
in a 512x512 matrix.
I have a few questions:
- First, how do I distribute the feature vector into 20 bins and calculate the mutual info matrix?
Not sure if there’s a way to do it with pytorch, maybe I should convert to np, calculate bins using np.histogram, find somehow the distribution and then assist mutual_info_score from sk.learn(?) and then convert back to tensor.
Maybe stay with the tensors, calculate the bins and assign the data, and calculate the MI myself?
- Assuming I get that matrix and find the 256 features with the lowest values between themselves (excluding 0), how do I that selection of the specific feature map and pass to the next fc layer?
- I pass batches to the network and get from the feature extractor features from different batches, not just sample, thus the feature vector’s shape is [batch_size, 4096] and not only 4096. Should it change anything?
In short - I want to do feature selection for the feature maps using mutual information between the feature maps and don’t know exactly how to do it and probably making it more complicated than it should be.
It’s not necessary for answering, but that’s the link to MDNet files on github (the net is in modules/model, and the feature extractor - forward _samples is in tracking/run_tracker.py) :