Multimedia Annotation, Querying, and Analysis in Anvil

While the availability of multimedia data, including human movement recording by motion capture, is steadily growing, the integrated viewing, annotation, and analysis of such complex data is still a challenge. The ANVIL tool, a widely used multi-track video and audio annotation tool, has now been extended to allow the synchronized handling of multiple media, especially the 3D viewing of motion capture data, to perform SQL queries and to conduct automated statistical analysis. The underlying database in conjunction with association detection allow analysis across tracks and modalities. This can be exploited in many contexts, from qualitative behavior analysis to the collection of training data for information extraction. Apart from describing the underlying methods and their realization in ANVIL, we discuss open issues like tool interoperabilty and scheme standardization.