Skip to Content

Sketch-based Video Browser (or Video Hunter)

Type: 
Multimedia
One line description: 
A video retrieval tool for known-item search tasks.
Annotation: 

Sketch-based Video Browser (or Video Hunter) is a new tool focusing on known-item search tasks, where users have seen some video scene, know it is contained in a collection, but do not know where it is located. Therefore, users have to search/browse the collection with advanced techniques enabling query idea initialization, result visualization and browsing. Given such support, known-item search is not restricted just to ideal query formulation. The Video Hunter tool has participated in the Video Browser Showdown, winning the competition in 2014 and 2015.

The Video Hunter tool focuses on intuitive interactive interface, enabling users to materialize a query idea, supporting various forms of visualizations and browsing.

Query formulation

In order to materialize a known-item, users can upload an example image, provide keywords and/or draw a multi-sketch with color/edge features. The tool emplys state-of-the-art deep convolutional neural networks to process queries based on example query objects (using DeCAF features) and keywords (annotations based on network classifications). The multi-sketch retrieval relies on position-color feature signatures and edge histograms.

Visualization

The visualization component for results supports two modes. The first mode displays the list of keyframes ordered with respect to the relevance to a query. The keyframes are accompanied with the temporal context, i.e., keyframes preceding/following the matched keyframe in the particular video. Basically, the results are displayed in the mode with one matched keyframe per one line with the temporal context. However, more results per line are enabled. The second mode displays hundreds of keyframes as a color-sorted map, without a temporal context. This second option is suitable for first phases of the retrieval where users first try to find a candidate query object. Beside this two main modes, the tool supports also interactive navigation summaries and comact visualizations of groups of similar keyframes.

Browsing

Vertical scrolling of the pages of ranked results is a standard way to improve recall. Once users find a promising scene or video, inspection of the temporal neighborhood is required. The Video Hunter tool enables horizontal browsing for each result line (extension of the temporal context) and also browsing using the interactive navigation summary.

 

References

  1. C. Cobarzan, K. Schoeff mann, W. Bailer, W. Hurst, A. Blažek, J. Lokoč, S. Vrochidis, K. U. Barthel, L. Rossetto. Interactive Video Search Tools: A Detailed Analysis of the Video Browser Showdown 2015, Multimedia tools and applications, Springer
  2. Adam Blazek, David Kubon, Jakub Lokoc: Known-Item Search in Video Databases with Textual Queries. SISAP 2016: 117-124
  3. David Kubon, Adam Blazek, Jakub Lokoc, Tomás Skopal: Multi-sketch Semantic Video Browser. MMM (2) 2016: 406-411
  4. Adam Blazek, Jakub Lokoc, Filip Matzner, Tomás Skopal: Enhanced Signature-Based Video Browser. MMM (2) 2015: 243-248
  5. Jakub Lokoc, Adam Blazek, Tomás Skopal: Signature-Based Video Browser. MMM (2) 2014: 415-418
  6. Adam Blazek, Jakub Lokoc, Tomás Skopal: Video Retrieval with Feature Signature Sketches. SISAP 2014: 25-36