A High-Recall Membership Inference Attack Based on Confidence-Thresholding Method with Relaxed Assumption

Membership inference attack (MIA) aims to infer whether a given data sample is in the target training dataset or not, which poses a severe privacy risk in particular data-sensitive fields like the military, national security department, as well as enterprise. Observing that a model generated from adversarial training is more vulnerable against MIA, a novel attack method based on confidence-thresholding was proposed by Song et al. recently. However, it is not a straightforward work to deploy such an attack into real-world application scenarios, since shadow training and redundant assumptions are prerequisites. To address the above issues, in this paper, we propose an improved confidence-thresholding method with relaxed assumption, evaluating the prediction accuracy as the threshold. Our attack can be released without using shadow training and an additional dataset. Instead of collecting an additional dataset, attackers use their target data records, which are needed to be inferred about membership, to achieve MIA. As a result, our proposed attack against robust model has an overwhelming advantage on model recall with fewer accuracy and precision loss. Extensive experiments are conducted on real-world data, i.e., Yale Face, and the results show that our proposed MIA attack is effective and feasible.