A semantic parliamentary multimedia approach for retrieval of video clips with content understanding

Digital videos of parliamentary activity play an important role in enhancing transparency and accountability for open e-government. The rapid growth in these videos and the lack of semantic annotations and relationships between video and knowledge resources make it increasingly difficult to find accurate video clips with contextual information for content understanding. To overcome this problem, we highlight the need for building multimedia systems based on a semantic vision. With this aim, we focus on (1) how to address the knowledge representation for automatic extraction of contextual information for video content understanding; (2) how to link the parliamentary knowledge structures within video resources to provide accurate video clips retrieval; and (3) how to perform semantic annotation on video resources. The methodology applied is focused on a systematic approach that uses techniques from ontology engineering. This approach is based on the definition of two models: the semantic model and the reference architecture. The semantic model is composed of a reference ontology and a semantic video annotation framework. The ontology provides the support for video content understanding and the semantic vocabulary for annotating video resources. The video annotation framework is based on an RDF-powered semantic video annotation to effectively relate low- and mid-level visual features, corresponding to speakers’ interventions, to high-level parliamentary concepts. To evaluate the proposed system, a prototype for the Canary Islands Parliament (Spain) has been carried out. The results show how semantic enhancement is a key enabler for improved video retrieval on parliamentary multimedia content.