AI-Based Phishing Attack Using Deepfake Audio

As artificial intelligence (AI) continues to advance, cybercriminals are finding new ways to exploit this technology to carry out sophisticated attacks. A notable example of this is the use of deepfake audio to conduct AI-based phishing attacks. In this case study, we explore a real-life incident where cybercriminals used AI-generated voices to deceive the U.K.-based energy company, resulting in significant financial loss. This blog provides insights into the attack, outlines best practices for mitigation, and includes resources for reporting such incidents to authorities.

How Criminals Are Using AI and Voice Deepfakes

Cybercriminals are using generative AI and deep learning for creating convincing audio content for scams.
Voice samples are gathered from publicly available sources, such as interviews or social media posts.
Advanced AI algorithms like neural networks and speech synthesis models are used to clone these voices.
Sophisticated generative adversarial networks (GANs) produce synthesized audio.
This audio is used to impersonate trusted individuals in phishing calls.
Criminals create urgency and pressure to trick victims into transferring money or sharing sensitive information.
The realistic nature of these deepfake voices, enhanced by text-to-speech (TTS) and voice conversion (VC) techniques, makes it hard for targets to recognize the scam.

For more info: How Cybercriminals Exploit AI: Voice Cloning & Deepfakes Explained

Incident Overview

The Attack

In March 2024, a U.K.-based energy company fell victim to a deepfake audio attack, resulting in a financial loss of US$243,000. The fraudsters used AI software to mimic the voice of the chief executive of the company’s Germany-based parent company. By doing so, they convinced the U.K. company’s CEO to make an urgent wire transfer to a supposed supplier in Hungary.

Execution

The cybercriminals called the U.K. company’s CEO, impersonating the parent company’s CEO. They demanded an urgent wire transfer, assuring the CEO of a reimbursement. Once the money was transferred, it was moved to an account in Mexico and then dispersed to various other locations, complicating the identification of the fraudsters.

Follow-up Attempts

The fraudsters made subsequent calls to request additional transfers, claiming that the first payment had been reimbursed. However, the U.K. company’s CEO grew suspicious, especially when the calls were made using an Austrian phone number, and refused further transactions.

Identifying AI-Based Phishing Attacks

Behavioral Indicators

Unexpected Requests: Be cautious of urgent, unexpected requests for sensitive information or financial transactions.
Inconsistent Details: AI-generated voices may struggle with specific details and context. Listen for inaccuracies and inconsistencies.
Verification Issues: Beware of caller ID spoofing and unverified communication channels.

Technological Red Flags

Robotic Tone: Despite advancements, AI-generated voices often lack the natural flow of human conversation.
Avoidance of Complex Questions: AI bots may avoid answering direct or complex questions, indicating a potential phishing attempt.

Mitigation Strategies: Protecting Your Organization

Enhance Employee Awareness

1. Regular Training Sessions:

Conducting effective training sessions on AI-based deepfake voice cloning involves specific strategies to ensure employees understand and can respond to these threats. Here’s how to do it:

Set Clear Objectives: Define the goals of each training session. Objectives may include recognizing deepfake voice phishing attempts, understanding the technology behind deepfakes, and knowing the proper response protocols.
Explain AI Voice Cloning: Start with a basic explanation of what AI voice cloning is and how it works. Use simple, non-technical language and provide examples of both legitimate uses and malicious uses.
Demonstrate Real-Life Scenarios: Develop case studies based on real incidents, such as the deepfake audio attack on the U.K. energy company. Use audio clips of both real and deepfake voices to illustrate the differences and challenges in detection.
Interactive Role-Playing: Create role-playing exercises where employees can practice handling suspicious calls. Provide scripts and scenarios where one employee acts as the attacker using a deepfake voice, while another acts as the target. After the exercise, discuss what happened and how the target could have responded better.
Provide Handouts and Audio Samples: Create handouts summarizing key points from the training, including checklists for recognizing voice phishing attempts and steps for verifying suspicious requests. Include links to audio samples of deepfake voices for further study.

Phishing Simulation Exercises:

Simulations are a practical way to reinforce training on deepfake voice phishing. Here’s how to implement them:

Simulate Voice Phishing Calls: Use AI tools to create deepfake voice recordings that mimic the voices of company executives or other trusted individuals. Conduct these simulations periodically and without prior notice to gauge real reactions.
Track Responses: Record how employees respond to these simulated attacks. Note if they followed verification protocols, asked for additional authentication, or complied with the request.
Debrief After Simulations: After each simulation, hold a debriefing session to discuss the results. Highlight what was done correctly and what could be improved. Provide constructive feedback and reinforce best practices.

Mitigata will assist in conducting these phishing simulation exercises. Our comprehensive phishing simulation console product already includes email phishing exercise training, and we are working on an audio deepfake feature. This will provide employees with hands-on experience in detecting and responding to deepfake voice phishing attempts, further enhancing your organization’s cybersecurity defenses.

Continuous Learning and Updates:

To ensure employees remain vigilant against deepfake voice phishing, provide ongoing education:

Webinars and Online Courses: Offer access to webinars and online courses focused on deepfake technology and voice phishing. Partner with cybersecurity organizations that specialize in these topics.
Regular Updates: Subscribe to cybersecurity newsletters and share relevant articles with employees. Hold monthly or quarterly meetings to review new threats and discuss how they impact the organization. Include updates on the latest advancements in AI and deepfake technology.

Additional Resources:

Provide employees with access to various resources to reinforce their training on deepfake voice phishing:

Guides and Checklists: Develop easy-to-follow guides and checklists specifically focused on voice phishing. These should be visually appealing and include examples of suspicious calls and common red flags.
Online Training Videos: Curate a playlist of training videos on deepfake voice phishing. Here are some useful links:

Develop a Culture of Security:

Embedding security into the company culture, especially regarding deepfake voice phishing, requires continuous effort:

Encourage Open Communication: Create channels for employees to report suspicious calls anonymously if they prefer. This can be an email hotline or a dedicated section on the company intranet.
Recognition Programs: Implement a recognition program where employees receive rewards for identifying voice phishing attempts or demonstrating excellent cybersecurity practices. Rewards can include certificates, public acknowledgment, or small bonuses.
Lead by Example: Ensure that management consistently follows cybersecurity protocols. When leaders prioritize security, employees are more likely to do the same.

6. Strengthen Verification Processes

Multi-Factor Authentication (MFA): Enforce MFA for all critical systems. Ensure employees understand that OTPs and other sensitive information should never be shared, even with trusted voices.
Out-of-Band Verification: Encourage employees to verify any unusual requests through a separate communication channel, such as a direct call to a verified number

7. Implement Robust Security Policies

Least Privilege Principle: Restrict access to sensitive information to only those who need it.
Incident Response Plan: Develop and regularly update an incident response plan. Ensure employees know how to report suspicious activities and understand the steps to take in response.

8. Leverage Advanced Security Technologies

AI-Based Detection Tools: Deploy AI-based tools to detect and prevent phishing attempts by analyzing communication patterns and identifying anomalies.
Voice Recognition with Anti-Spoofing: Implement voice recognition technologies with anti-spoofing measures to identify and block AI-generated voice phishing attempts.

Best Practices for Preventing BEC and AI-Based Phishing Attacks

Verify Fund Transfer Requests: Always verify fund transfer and payment requests, especially those involving large amounts, by contacting the supplier directly and confirming the transaction through a known phone number.
Red Flags in Business Transactions: Be alert to changes in bank account information without prior notice, as this is a common sign of a BEC attempt.
Scrutinize Emails: Employees should scrutinize received emails for any suspicious elements, such as unusual domains or changes in email signatures.
Security Technology: Utilize security technologies like Writing Style DNA, which detects email impersonation tactics by analyzing the writing style of emails and comparing them to a user’s typical writing style.

Reporting and Government Resources

To enhance your security posture, report suspicious activities and phishing attempts to relevant authorities. Here are some useful links:

Government Advisory Links on Phishing Attacks

India: CERT-In Advisory
United States: CISA Phishing Guidance
United Kingdom: NCSC Phishing Advice
Australia: ACSC Phishing Resources

Conclusion

The rise of AI-based phishing attacks underscores the need for heightened vigilance and robust security measures. By learning from real-life incidents like the U.K. energy company’s deepfake audio attack, organizations can better prepare and protect themselves against these sophisticated threats. Educating employees, strengthening verification processes, implementing advanced security technologies, and staying informed about the latest attack vectors are critical steps in mitigating the risk of such cybercrimes.

For more information on protecting your organization from AI-based phishing attacks and other cybersecurity threats, contact Mitigata today. Stay vigilant, stay informed, and stay protected.