In this project, the main problem we aim to investigate how to synthesize and to transfer human motion and appearance from video to video preserving motion features, body shape, and visual quality, which increase the creative possibilities of visual content. In our approaches, we present  a formulation that leverages the user body shape into the retargeting while considering physical constraints of the motion in 3D and the 2D image domain. We also present a new video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people’s videos, which can be used as a common base to improve tracking the progress in the field.

Publications


[IJCV 2021] Thiago L. Gomes and Renato Martins, João Ferreira, Rafael Azevedo, Guilherme Torres, Erickson R. Nascimento . A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos, International Journal of Computer Vision (IJCV), 2021.
Visit the page for more information and paper access.

[C&G 2021] João P. Ferreira,Thiago M. Coutinho,Thiago L. Gomes, José F. Neto, Rafael Azevedo and Renato Martins and Erickson R. Nascimento. Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio, Computers & Graphics, 2021.
Visit the page for more information and paper access.

[WACV 2020] Thiago L. Gomes, Renato Martins, João Ferreira, Erickson R. Nascimento. Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraints, IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
Visit the page for more information and paper access.

Datasets


[IJCV 2021] Retargeting dataset. Video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people’s videos.
Visit the dataset page for video info and download.

[C&G 2021] Audio motion dataset. New dataset composed of paired videos of people dancing different music styles: Ballet, Michael Jackson, and Salsa.
Visit the dataset page for video info and download.


Acknowledgment


The authors would like to thank CAPES, CNPq, FAPEMIG, and ATMOSPHERE PROJECT for funding different parts of this work. We also thank NVIDIA Corporation for the donation of a Titan XP GPU used in this research.

Team



Thiago Luange Gomes

PhD Candidate

João Pedro Moreira Ferreira

MSc Student

Rafael Augusto Vieira de Azevedo

MSc Student

Thiago Malta Coutinho

MSc Student

Renato José Martins

Post-doctoral Researcher


Former members



José Francisco da Silva Neto

PhD Student

Guilherme Alvarenga Torres

Undergraduate Student

Thiago Martin Poppe

Undergraduate Student