I am trainin a model with pytorch, where I need to calculate the degree of dependence between two tensors (lets say they are the two tensor each containing values very close to zero or one, e.g. v1 = [0.999, 0.998, 0.001, 0.98] and v2 = [0.97, 0.01, 0.997, 0.999]) as a part of my loss function. I am trying to calculate mutual information, but I can’t find any mutual information estimation implementation in pytroch. Has such a thing been provided anywhere?
1 Like
I wondered if the v1 and v2 can be considered probability values?
They are the outputs of a gumbel softmax function to approximate sampling categorical data. But I don’t think it really matters here.
Well, to calculate the MI we will be requiring joint and their individual distribution. Is there any way you can find this during your batch execution?
There isn’t any built in MI score implementation unfortunately. Take a look at this and see if you can perhaps somehow use it in your problem.
1 Like