A familiar voice on a phone call used to be a useful clue that you were speaking to the person you expected.
AI-generated media weakens that assumption.
Modern generative systems can create or modify audio and video that appears to come from a real person. The security problem is not simply that fake media exists. It is that an attacker can combine a convincing synthetic voice or video with an ordinary social-engineering story, urgency and a request for money, credentials or confidential information.
The Federal Trade Commission has specifically warned that scammers can use voice cloning to impersonate relatives, business executives and others. It has also emphasised that there is no single technical solution that eliminates the problem. citeturn1search0turn1search7
What is a deepfake?
Deepfake is a broad term commonly used for synthetic or manipulated media generated or altered with machine-learning techniques.
It can involve:
- synthetic speech that resembles a real person’s voice;
- face replacement or facial reenactment;
- generated video of a person appearing to speak;
- manipulated footage that changes what an existing video appears to show;
- combinations of real and synthetic audio or video.
Not every AI-generated image, voice or video is a deepfake, and synthetic media is not inherently malicious. The security concern arises when deceptive media is used to impersonate someone or create false evidence.
Why voice cloning is particularly useful to scammers
Voice has strong psychological power.
Imagine receiving a call that sounds like a family member saying there is an emergency and money is needed immediately. Or hearing what appears to be a senior executive instructing an employee to make a payment.
The attacker does not need the voice to be perfect in every technical detail. The goal is to create enough confidence for the target to stop verifying the request.
The FTC has described family-emergency and executive-impersonation scenarios involving voice cloning. citeturn1search2turn1search5
This is why deepfake defence is partly a process problem, not just a media-detection problem.
Where does the source material come from?
Attackers may obtain useful material from publicly available content such as:
- public videos;
- interviews;
- social-media posts;
- podcasts;
- livestream recordings;
- company presentations;
- previously published audio.
The existence of a public recording does not mean an attack will happen. It simply means that an impersonator may have source material from which to construct a more convincing deception.
The practical response is not necessarily to remove every public recording. It is to avoid treating familiar voice or appearance as sufficient proof of identity for high-risk actions.
A deepfake can be only one part of the attack
A realistic fraud attempt may combine several techniques:
- Find information about the target and their relationships.
- Obtain public audio or video.
- Generate or manipulate media that resembles a trusted person.
- Contact the target through a familiar or plausible channel.
- Create urgency or secrecy.
- Request a payment, credential, code or confidential document.
- Pressure the target to act before independent verification.
The synthetic media supports the social engineering. It does not have to perform the entire attack by itself.
This is similar to phishing: the technology improves the credibility of the message, while the final failure often occurs because the victim is pushed into an unsafe action.
Why detecting deepfakes is difficult
Detection tools can be useful, but they should not be treated as universal proof.
Synthetic media changes quickly. Compression, background noise, microphones, editing and platform processing can affect the signals available to a detector.
The FTC’s work on voice-cloning countermeasures has considered several intervention points, including prevention and authentication, real-time detection and post-use evaluation. It also notes limitations and the need for continued testing as the technology evolves. citeturn1search7
A practical security programme should therefore avoid a single rule such as “the detector says real, so approve the transaction.”
The strongest control is often independent verification
Suppose an employee receives a video call that appears to be from a manager and asks for an urgent payment.
Instead of deciding whether the face looks genuine, the employee can verify the request through a separate trusted channel.
For example:
- call the manager using a number already stored in the corporate directory;
- use an established internal messaging channel;
- require a normal approval workflow;
- verify the payment details independently;
- require two authorised people for high-risk transactions.
The important property is independence.
Calling the same suspicious number back does not provide strong independent verification. Opening a link supplied by the suspicious message is not independent either.
Use transaction controls that do not depend on identity cues
Deepfakes become much less useful when a sensitive action requires controls that synthetic media cannot satisfy on its own.
For financial processes, useful controls can include:
- separation of duties;
- transaction limits;
- dual approval;
- trusted beneficiary management;
- confirmation of changed payment details;
- out-of-band verification;
- anomaly monitoring;
- strong authentication.
For access to sensitive systems, use established identity and authorization controls rather than accepting a face or voice as proof.
Protect the organisation against executive impersonation
A common mistake is allowing informal messages from senior staff to override formal controls.
A safer policy is simple:
Urgency never bypasses the approval process.
An executive may genuinely need an urgent payment, but the employee should still follow the normal procedure.
Security teams can make this easier by defining:
- which actions require dual approval;
- which requests can never be completed solely through email or chat;
- how bank-detail changes are verified;
- which channels are trusted for approvals;
- how suspected impersonation is escalated.
The goal is to make secure behaviour operationally normal.
Family-level protection matters too
The same principle applies outside work.
Families can agree on a simple verification rule for emergency requests. For example, a request for urgent money should trigger a callback using a known number or a private family verification phrase.
The point is not to invent a complicated secret system. It is to create a second step that an impersonator cannot easily obtain from a public profile.
The FTC recommends slowing down and checking before acting when a caller claims to be a loved one in an emergency. citeturn1search12
Do not trust caller ID alone
Caller ID can be spoofed, so the displayed number should not be treated as proof of identity.
The same principle applies to email display names, profile photographs and social-media accounts.
A message can look authentic while still arriving through an account or channel controlled by someone else.
Identity should be established using controls that are harder for the attacker to reproduce.
What organisations should monitor
Security teams should watch for process signals as well as media signals.
Useful indicators include:
- unexpected requests to change payment instructions;
- unusual urgency or secrecy;
- requests that bypass established approval paths;
- new recipients or beneficiaries;
- unusual login or transaction context;
- sudden requests for credentials or MFA codes;
- senior-person impersonation reports;
- suspicious use of public information about staff.
These signals can be more actionable than trying to classify every video or voice recording as real or fake.
What if a detector says a video is authentic?
Treat a detection result as one signal, not as authorisation.
A media detector answers a question about the media. It may not answer:
- Is this person authorised to make the request?
- Is the request consistent with policy?
- Is the destination account correct?
- Has the instruction been independently confirmed?
- Is this transaction expected?
Those are identity, authorization and business-process questions.
This distinction is important because cybersecurity is about controlling actions, not merely classifying files.
What individuals can do
A practical personal checklist is:
- Be cautious with urgent requests for money or confidential information.
- Do not rely on voice, video, caller ID or profile images alone.
- Verify through a known, independent channel.
- Do not share passwords, PINs or one-time codes because a caller sounds familiar.
- Slow down when a request demands secrecy or immediate action.
- Keep important accounts protected with strong authentication.
- Review public information that makes impersonation easier.
- Report suspicious activity quickly to the relevant service provider or authority.
For Sri Lankan users, the Central Bank of Sri Lanka advises the public to verify suspicious financial communications through official bank channels and not to share credentials or OTPs. citeturn2search0turn2search5
What organisations can do
A useful organisational baseline is:
- define high-risk actions;
- require independent verification for those actions;
- separate initiation and approval;
- train staff using realistic social-engineering scenarios;
- protect executive communication channels;
- monitor unusual transaction requests;
- use strong authentication for privileged access;
- log approvals and changes;
- provide a clear incident-reporting route;
- test the process regularly.
The important point is that deepfake defence should fit into existing identity, fraud and authorization controls.
A better question than “Can I spot a fake?”
People often ask whether there is a visual or audio clue that reliably exposes a deepfake.
That can be useful for awareness, but it is not the right primary security question.
Ask instead:
What happens if this media is fake and we fail to notice?
If the answer is “the attacker can immediately transfer money,” the process needs stronger controls.
If the answer is “the request still requires independent approval,” the deepfake has much less power.
That is the difference between relying on detection and designing for failure.
The practical conclusion
AI-generated voice and video can make impersonation scams more convincing, but the core security problem is still trust.
Attackers want the target to believe that a person is authentic, that a request is legitimate and that action must happen now.
The strongest defence is to break that chain.
Do not make a payment because a voice sounds familiar. Do not approve a sensitive request because a video looks convincing. Use independent verification, strong authentication, authorization controls and established approval processes.
Deepfake technology will continue to change. A good security process does not need to perfectly predict the next generation of synthetic media. It needs to remain safe when the media cannot be trusted.