ORCID
- Asif Khan: 0000-0002-5863-8467
Abstract
Lip-sync deepfakes preserve a victim’s visual identity while replacing their spoken content, combining high credibility with low generation cost. Two developments have destabilised their detection: audio–visual feature-inconsistency detectors reach high accuracy on curated benchmarks but degrade under realistic transmission and language shifts, and lip-sync generators now inherit a physiologically valid pulse from their driving video, defeating unimodal remote-photoplethysmography (rPPG) detectors that inspect only the marginal cardiac signal. We argue that physiological deepfake detection is not obsolete after this inherited-heartbeat result but must be reformulated. Real speech is produced by a coupled respiratory–cardiac–vocal system; a re-synced deepfake pairs a pulse drawn from one physiological history with audio drawn from another, so its joint pulse–audio statistics cannot match those of a real speaker even when each signal is individually valid. We formalise this asymmetry as physio-acoustic decoupling and introduce PhysAcou, which learns the natural coupling manifold from large-scale unlabelled real speech video through a self-supervised cross-modal coupling encoder, a biomechanically structured state-space prior, and a likelihood-based forensic decision rule. On AVLips, FaceForensics++, and DFDC, PhysAcou attains 96.41%, 95.88%, and 95.36% accuracy. Under the inherited-heartbeat attack—where rPPG detectors collapse to 52.18–57.06% and the strongest audio–visual baseline reaches 78.41%—PhysAcou retains 91.63% accuracy and 92.47% AUC, with consistent gains across four generators, four languages, and degraded real-world conditions, and with fewer parameters than the strongest baseline. These results reframe physiological deepfake detection from pulse presence to pulse–audio coupling consistency.
Keywords
audio–visual forensics, biomechanical state-space modelling, cross-modal consistency, inherited-heartbeat attack, lip-sync deepfake detection, media authentication, multimodal self-supervised learning, mutual information, physio-acoustic coupling, remote photoplethysmography
DOI Link
Publication Date
2026-01-01
Publication Title
IEEE Access
Acceptance Date
2026-01-01
Deposit Date
2026-11-05
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Additional Links
Recommended Citation
Mamarasulov, S., Khan, A., Khodjamov, N., Kholov, A., Ruziev, O., & Nuralieva, N. (2026) 'Audio Has No Pulse: Detecting Lip-Sync Deepfakes via Broken Physio-Acoustic Coupling', IEEE Access, . Available at: 10.1109/ACCESS.2026.3742533