I am using autoencoders to reconstruction images and then based on a certain threshold value I calculate the detection accuracy using;
# threshold = 95th percentile
# flag_list = {0, 1} Meaning -> 0: Benign image, 1: adversarial image
def detection_accuracy(total_recon, flag_list, threshold):
recon_np, label_np = np.asarray(total_recon), np.asarray(flag_list)
recon_label_np = np.stack((recon_np, label_np), axis=1)
recon_label_np = recon_label_np[recon_label_np[:,0].argsort()]
FN, TN, TP, FP = 0, 0, 0, 0
for i in range(recon_label_np.shape[0]):
if recon_np[i] < threshold:
if label_np[i]!=flag:
FP += 1
else:
TP += 1
else:
if label_np[i]!=flag:
TN += 1
else:
FN += 1
acc = 100.*(TP + TN) / (TP + TN + FP + FN)
print('Accuracy: ', acc)
return acc
I am doubtful about the way I do, as the accuracy doesn’t come out well as per the theoretical expectations. Can anyone tell a better way to find the accuracy based on reconstruction error? I tried passing the reconstruction through a resnet50 classifier and calculate the classification accuracy but that doesn’t come out well and that does not even support the objective.