Custom loss function for accounting outliers as non-outliers

21

@googlebot I’ve found and tried the following long tail fitting curve distribution analysis pipeline I’ve tested it with the intensity values of one of my point clouds and I’ve got the results that I am posting here as I understand from the graphs the pairwise power law transformation seems to fit better for this specific values (which is similar to what I have in my whole dataset). But I am not really sure whether I interpret it correctly or not.

Moreover I’ve got this is output with xmins=125 (tried 25 did not work that well), which I am not really sure what that also means. Log-normal for example that you were suggesting above and it is one of the fitting curve analysis seems to do a bit worse. Do you understand whether this seems any kind of helpful or not?

22

If you’re trying quantile approach, very accurate approximating distribution is not necessary - it is only used for calibration, in other words its cdf slope (=pdf) will control step sizes. So, you want some closed-form quantile formula, and perhaps right tail that is bigger than actual, for stability as p approaches 1.

If that whole thing doesn’t work, check Catastrophic interference - Wikipedia, that describes what I’ve called “forgetfulness” in more detail.