Cyber-attacks are critical threats for both organizations and individual users. Such threats aim to gain access to confidential information, which is often used to steal financial assets. While much work is done on a technical level to improve automated cybersecurity systems, the human user is ultimately the last line of defense. In a domain that is rapidly transforming, it is of critical importance to not only prepare human users to detect fraudulent attacks, but also understand how automated safeguards, like spam filters, influence the users’ detection. As such, the present study investigated the effect of an automated email filter on users’ classifications of legitimate and phishing emails.