Survey of Semantic Media Annotation Tools for the Web: Towards New Media Applications with Linked Media
---
paper_title: That obscure object of desire: multimedia metadata on the Web, part 2

paper_content:

This article discusses the state of the art in metadata for audio-visual media in large semantic networks, such as the Semantic Web. Our discussion is predominantly motivated by the two most widely known approaches towards machine-processable and semantic-based content description, namely the Semantic Web activity of the W3C and ISO's efforts in the direction of complex media content modeling, in particular the Multimedia Content Description Interface (MPEG-7). We explain that the conceptual ideas and technologies discussed in both approaches are essential for the next step in multimedia development. Unfortunately, there are still many practical obstacles that block their widespread use for providing multimedia metadata on the Web. Based on a scenario to explain our vision of a media-aware Semantic Web, we derive in Part I a number of problems regarding the semantic content description of media units. We then discuss the multimedia production chain, in particular emphasizing the role of progressive metadata production. As a result we distill a set of media-based metadata production requirements and show how current media production environments fail to address these. We then introduce those parts of the W3C and ISO standardization works that are relevant to our discussion. In Part II of this article, we analyze their abilities to define structures for describing media semantics, discuss syntactic and semantic problems, ontological problems for media semantics, and the problems of applying the theoretical concepts to real world problems. Part II concludes with implications of the findings for future action with respect to the actions the community should take.
---
paper_title: Multimedian e-culture demonstrator

paper_content:

The main objective of the MultimediaN E-Culture project is to demonstrate how novel semantic-web and presentation technologies can be deployed to provide better indexing and search support within large virtual collections of cultural-heritage resources. The architecture is fully based on open web standards, in particular XML, SVG, RDF/OWL and SPARQL. One basic hypothesis underlying this work is that the use of explicit background knowledge in the form of ontologies/vocabularies/thesauri is in particular useful in information retrieval in knowledge-rich domains.
---
paper_title: Adding Multimedia to the Semantic Web-Building an MPEG-7 Ontology

paper_content:

For the past two years the Moving Pictures Expert Group (MPEG), a working group of ISO/IEC, have been developing MPEG-7 [1], the "Multimedia Content Description Interface", a standard for describing multimedia content. The goal of this standard is to develop a rich set of standardized tools to enable both humans and machines to generate and understand audiovisual descriptions which can be used to enable fast efficient retrieval from digital archives (pull applications) as well as filtering of streamed audiovisual broadcasts on the Internet (push applications). MPEG-7 is intended to describe audiovisual information regardless of storage, coding, display, transmission, medium, or technology. It will address a wide variety of media types including: still pictures, graphics, 3D models, audio, speech, video, and combinations of these (e.g., multimedia presentations). MPEG-7 is due for completion in October 2001. At this stage MPEG-7 definitions (description schemes and descriptors) are expressed solely in XML Schema [2-4]. XML Schema has been ideal for expressing the syntax, structural, cardinality and datatyping constraints required by MPEG-7. However it has become increasingly clear that in order to make MPEG-7 accessible, re-usable and interoperable with other domains then the semantics of the MPEG-7 metadata terms also need to be expressed in an ontology using a machine-understandable language. This paper describes the trials and tribulations of building such an ontology represented in RDF Schema [5] and demonstrates how this ontology can be exploited and reused by other communities on the semantic web (such as TV-Anytime [6], MPEG-21 [7], NewsML [8], museum, educational and geospatial domains) to enable the inclusion and exchange of multimedia content through a common understanding of the associated MPEG-7 multimedia content descriptions.
---
paper_title: Creating Enriched YouTube Media Fragments With NERD Using Timed-Text

paper_content:

This demo enables the automatic creation of semantically annotated YouTube media fragments. A video is first ingested in the Synote system and a new method enables to retrieve its associated subtitles or closed captions. Next, NERD is used to extract named entities from the transcripts which are then temporally aligned with the video. The entities are disambiguated in the LOD cloud and a user interface enables to browse through the entities detected in a video or get more information. We evaluated our application with 60 videos from 3 YouTube channels.
---
paper_title: Statement-based semantic annotation of media resources

paper_content:

Currently the media production domain lacks efficient ways to organize and search for media assets. Ontology-based applications have been identified as a viable solution to this problem, however, sometimes being too complex for non-experienced users. We present a fast and easy to use approach to create semantic annotations and relationships of media resources. The approach is implemented in the SALERO Intelligent Media Annotation & Search system. It combines the simplicity of free text tagging and the power of semantic technologies and by that makes a compromise in the complexity of full semantic annotations. We present the implementation of the approach in the system and an evaluation of different user interface techniques for creating annotations.
---
paper_title: Enriching media fragments with named entities for video classification

paper_content:

With the steady increase of videos published on media sharing platforms such as Dailymotion and YouTube, more and more efforts are spent to automatically annotate and organize these videos. In this paper, we propose a framework for classifying video items using both textual features such as named entities extracted from subtitles, and temporal features such as the duration of the media fragments where particular entities are spotted. We implement four automatic machine learning algorithms for multiclass classification problems, namely Logistic Regression (LG), K-Nearest Neighbour (KNN), Naive Bayes (NB) and Support Vector Machine (SVM). We study the temporal distribution patterns of named entities extracted from 805 Dailymotion videos. The results show that the best performance using the entity distribution is obtained with KNN (overall accuracy of 46.58%) while the best performance using the temporal distribution of named entities for each type is obtained with SVM (overall accuracy of 43.60%). We conclude that this approach is promising for automatically classifying online videos.
---
paper_title: NERD: A Framework for Unifying Named Entity Recognition and Disambiguation Extraction Tools

paper_content:

Named Entity Extraction is a mature task in the NLP field that has yielded numerous services gaining popularity in the Semantic Web community for extracting knowledge from web documents. These services are generally organized as pipelines, using dedicated APIs and different taxonomy for extracting, classifying and disambiguating named entities. Integrating one of these services in a particular application requires to implement an appropriate driver. Furthermore, the results of these services are not comparable due to different formats. This prevents the comparison of the performance of these services as well as their possible combination. We address this problem by proposing NERD, a framework which unifies 10 popular named entity extractors available on the web, and the NERD ontology which provides a rich set of axioms aligning the taxonomies of these tools.
---

Title: Survey of Semantic Media Annotation Tools for the Web: Towards New Media Applications with Linked Media

Section 1: Introduction
Description 1: Introduce the limitations of current Web technologies in media retrieval and reuse, and the necessity for semantic media annotations for richer media descriptions and innovative media-centric applications.

Section 2: Principles for Web-friendly Semantic Annotation of Media
Description 2: Propose a set of principles for semantic media annotations called "Linked Media," and show how emerging Web specifications can address them.

Section 3: Currently Available Semantic Multimedia Annotation Tools for the Web
Description 3: Examine existing tools for semantic media annotations on the Web, compare them with Linked Media principles, and highlight tools that align with this vision.

Section 4: Examples of New Media Applications Enabled by Linked Media
Description 4: Outline examples of new media applications that could be enabled by Linked Media, demonstrating their potential impact on media retrieval, reuse, and re-mixing.

Section 5: Conclusion and Future Work
Description 5: Conclude with a vision of widespread semantic annotations of online media, call for future research to align with Linked Media principles, and discuss the potential for new applications utilizing semantically annotated media.