In this project, the main problem we aim to investigate how to synthesize and to transfer human motion and appearance from video to video preserving motion features, body shape, and visual quality, which increase the creative possibilities of visual content. In our approaches, we present a formulation that leverages the user body shape into the retargeting while considering physical constraints of the motion in 3D and the 2D image domain. We also present a new video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people’s videos, which can be used as a common base to improve tracking the progress in the field.
Publications
[IJCV 2021] Thiago L. Gomes and Renato Martins, João Ferreira, Rafael Azevedo, Guilherme Torres, Erickson R. Nascimento . A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos, International Journal of Computer Vision (IJCV), 2021. Visit the page for more information and paper access.
@Article{gomes2021ijcv,
author={Gomes, Thiago L.and Martins, Renato and Ferreira, Jo{\~a}o and Azevedo, Rafael and Torres, Guilherme and Nascimento, Erickson R.},
title={A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos},
journal={International Journal of Computer Vision},
year={2021},
month={Apr},
day={29},
issn={1573-1405},
doi={10.1007/s11263-021-01471-x},
url={https://doi.org/10.1007/s11263-021-01471-x}
}
[C&G 2021] João P. Ferreira,Thiago M. Coutinho,Thiago L. Gomes, José F. Neto, Rafael Azevedo and Renato Martins and Erickson R. Nascimento. Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio, Computers & Graphics, 2021. Visit the page for more information and paper access.
@ARTICLE{ferreira2021cag,
author = {João P. {Ferreira} and Thiago M. {Coutinho} and Thiago L. {Gomes} and José F. {Neto} and Rafael {Azevedo} and Renato {Martins} and Erickson R. {Nascimento}},
title = {Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio},
journal = {Computers & Graphics},
volume = {94},
pages = {11 – 21},
year = {2021},
issn = {0097-8493},
doi = {https://doi.org/10.1016/j.cag.2020.09.009}
}
[WACV 2020] Thiago L. Gomes, Renato Martins, João Ferreira, Erickson R. Nascimento. Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraints, IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. Visit the page for more information and paper access.
@INPROCEEDINGS {gomes2020wacv,
author = {T. L. Gomes and R. Martins and J. Ferreira and E. R. Nascimento},
booktitle = {2020 IEEE Winter Conference on Applications of Computer Vision (WACV)},
title = {Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraints},
year = {2020},
pages = {3355-3364},
doi = {10.1109/WACV45572.2020.9093395},
url = {https://doi.ieeecomputersociety.org/10.1109/WACV45572.2020.9093395},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
month = {mar}
}
Datasets
[IJCV 2021] Retargeting dataset. Video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people’s videos. Visit the dataset page for video info and download.
The authors would like to thank CAPES, CNPq, FAPEMIG, and ATMOSPHERE PROJECT for funding different parts of this work. We also thank NVIDIA Corporation for the donation of a Titan XP GPU used in this research.
