Loughborough University
Browse
Computer Graphics Forum - 2023 - Liu - Semantics‐guided generative diffusion model with a 3DMM model condition for face.pdf (6.47 MB)

Semantics-guided generative diffusion model with a 3DMM model condition for face swapping

Download (6.47 MB)
journal contribution
posted on 2023-12-14, 17:26 authored by Xiyao Liu, Yang Liu, Yuhao Zheng, Ting Yang, Jian Zhang, Victoria Wang, Hui FangHui Fang

Face swapping is a technique that replaces a face in a target media with another face of a different identity from a source face image. Currently, research on the effective utilisation of prior knowledge and semantic guidance for photo-realistic face swapping remains limited, despite the impressive synthesis quality achieved by recent generative models. In this paper, we propose a novel conditional Denoising Diffusion Probabilistic Model (DDPM) enforced by a two-level face prior guidance. Specifically, it includes (i) an image-level condition generated by a 3D Morphable Model (3DMM), and (ii) a high-semantic level guidance driven by information extracted from several pre-trained attribute classifiers, for high-quality face image synthesis. Although swapped face image from 3DMM does not achieve photo-realistic quality on its own, it provides a strong image-level prior, in parallel with high-level face semantics, to guide the DDPM for high fidelity image generation. The experimental results demonstrate that our method outperforms state-of-the-art face swapping methods on benchmark datasets in terms of its synthesis quality, and capability to preserve the target face attributes and swap the source face identity.

Funding

Science and Technology Innovation Program of Hunan Province. Grant Number: 2022GK5002

Special Foundation for Distinguished Young Scientists of Changsha. Grant Number: kq2209003

111 Project. Grant Number: D23006

Foreign Expert Project of China. Grant Number: G2023041039L

History

School

  • Science

Department

  • Computer Science

Published in

Computer Graphics Forum

Volume

42

Issue

7

Publisher

Wiley

Version

  • VoR (Version of Record)

Rights holder

© The Authors

Publisher statement

This is an Open Access article published by Wiley under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. See https://creativecommons.org/licenses/by/4.0/

Acceptance date

2023-09-11

Publication date

2023-10-30

Copyright date

2023

ISSN

0167-7055

eISSN

1467-8659

Language

  • en

Depositor

Dr Hui Fang. Deposit date: 11 September 2023

Article number

e14949

Usage metrics

    Loughborough Publications

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC