In this project, we deal with a central challenge that is to make egocentric videos watchable. First person videos are generally long-running streams with unedited content, which make them boring and visually unpalatable. Efforts have been applied to try accelerate them, because just fast-forward them is not enough, because it amplifies the natural motion of the recorder’s body turning the video nauseate. In this project we propose a novel methodology to compose the new fast-forward video by selecting frames in a smart manner and based in the semantic information extracted from images.

Publications:


Fast-Forward Video Based on Semantic Extraction (ICIP 2016)

2016 IEEE International Conference on Image Processing (ICIP)

Abstract

Thanks to the low operational cost and large storage capacity of smartphones and wearable devices, people are recording many hours of daily activities, sport actions and home videos. These videos, also known as egocentric videos, are generally long-running streams with unedited content, which make them boring and visually unpalatable, bringing up the challenge to make egocentric videos more appealing. In this work we propose a novel methodology to compose the new fast-forward video by selecting frames based on semantic information extracted from images. The experiments show that our approach outperforms the state-of-the-art as far as semantic information is concerned and that it is also able to produce videos that are more pleasant to be watched.
Keywords: Semantic Information, First-person Video, Fast-Forward, Video Sampling, Video Segmentation

SemanticHyperlapse_Methodology

Links

            Source code – Available here! (NEW)

            GitXiv

            ArXiv

            Paper Draft

            Conference Poster

Citation

@INPROCEEDINGS{Ramos2016,
  author    = {W. L. S. Ramos and M. M. Silva and M. F. M. Campos and E. R. Nascimento},
  booktitle = {2016 IEEE International Conference on Image Processing (ICIP)},
  title     = {Fast-forward video based on semantic extraction},
  year      = {2016},
  pages     = {3334-3338},
  keywords  = {Biomedical monitoring;Cameras;Data mining;Face;Legged locomotion;
              Semantics;Streaming media;Fast-Forward;First-person Video;Hyperlapse;
              Semantic Information;Video Sampling},
  doi       = {10.1109/ICIP.2016.7532977},
  month     = {Sept}
}

Presentation Video

Extra Visual Results

Team:


Towards Semantic Fast-Forward and Stabilized Egocentric Videos (ECCVW 2016)

First International Workshop on Egocentric Perception, Interaction and Computing at European Conference on Computer Vision (EPIC@ECCV) 2016

Abstract:

The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fast-forward state-of-the-art methods are facing challenges of balancing the smoothness of the video and the emphasis in the relevant frames given a speed-up rate. In this work, we present a methodology capable of summarizing and stabilizing egocentric videos by extracting the semantic information from the frames. This paper also describes a dataset collection with several semantically labeled videos and introduces a new smoothness evaluation metric for egocentric videos that is used to test our method.
Keywords: Semantic Information, First-person Video, Fast-Forward, Egocentric Stabilization

methodology figure_stabilization

Links

            Source code – Available here! (NEW)

                    Accelerated Video Stabilizer Documentation

            Dataset Download Page

            GitXiv

            ArXiv

            Paper Draft

            Conference Poster

dataset_new

Citation

@InBook{Silva2016,
  Title     = {Towards Semantic Fast-Forward and Stabilized Egocentric Videos},
  Author    = {Silva, Michel Melo and Ramos, Washington Luis Souza and Ferreira, 
              Joao Pedro Klock and Campos, Mario Fernando Montenegro and Nascimento, Erickson Rangel},
  Editor    = {Hua, Gang and J{\'e}gou, Herv{\'e}},
  Pages     = {557--571},
  Publisher = {Springer International Publishing},
  Year      = {2016},
  Address   = {Cham},
  Booktitle = {Computer Vision -- ECCV 2016 Workshops: Amsterdam, The Netherlands, 
              October 8-10 and 15-16, 2016, Proceedings, Part I},
  Doi       = {10.1007/978-3-319-46604-0_40},
  ISBN      = {978-3-319-46604-0},
  Url       = {http://dx.doi.org/10.1007/978-3-319-46604-0_40}
}

Presentation Video

Extra Visual Results

Team: