Adversarial Robustness of Phishing Email Detection: A Comparative Study of TF-IDF + Logistic Regression and Fine-Tuned DistilBERT
A new study compares the adversarial robustness of two phishing email detection models: TF-IDF + Logistic Regression and fine-tuned DistilBERT. Both models achieved over 98% accuracy on clean, in-distribution data, but their performance dropped sharply to around 64% when tested against adversarially modified phishing emails. Analysis showed that while the models relied on different features, they exhibited similar vulnerabilities and complementary error patterns. The findings demonstrate that high accuracy on clean data does not guarantee robustness to adversarial attacks, highlighting the need for adversarial testing in phishing detection research.
Why it matters: This research reveals that widely used phishing detection models are highly vulnerable to adversarial attacks, emphasizing the importance of evaluating models beyond clean-data accuracy.
Full story at: arXiv Cryptography and Security ↗