SemanticMedia

Preeliminar Presentation: semanticMedia.ppt

Group Members

• GreciaLapizco - glapizco at umd.edu
• AlejandroRodriguez - alejandr at cs.umd.edu

Goal

Online system for SEMANTIC querying of multimedia content.

Example: “Find an image of John Smith (the one at www.cs.umd.edu/~jsmith) in which he is camping”

Project Summary

As multimedia content become more popular, people collect greater amounts of multimedia files, ranging from personal pictures to work-related presentations. As these collections grow they tend to do it in a unorganized fashion which makes them harder to use, (e.g. try looking for that particular picture of the ocean that you took during the summer vacation of 1998). On the other hand, we have already experimented the difficulties of manually marking up personal documents with semantic meaning for the computer. These brings the oportunity of linking these two tasks in a painless manner for the user. One single tool could be used to organize, mantain and publish multimedia documents annotated with semantic markup.

SemanticMedia is intended to be a basic application targeted to non-computer scientist mor knowledge engineers that allows users to do exactly what we describe above, in a basic but user-friendly fashion. Among the advantages of this kind of tool, besides the direct utility for personal users, is that they serve as an incentive for the creation of semantic content. Also future extensions of this tool would facilitate automatic inferencing and categorization of multimedia content.

Related work

Ontology-based photo annotation

Features:

• Semantic retrieval (RDFS)

Limitations:

• Domain dependent
• Only images
• No web-based
• Ontologies expressed on RDFS instead of DAML+OIL or OWL

Querying System for Online Images Based on the PNG Format and Embedded Metadata

Features

• Java and open source
• Based on Dublin Core
• Crawling
• Metadata within the image files

Limitations:

• Only PNG files
• Only conjunctive queries
• No standard ontology language

Architecture

General architecture of the system:

Client

The client is a tool that allows users to markup their documents in daml using our ontology in association with user-specific terms and elements. For this part we decided to use an off the shelf tool: SMORE.

Ontology

The ontology is based on the ABC and MPEG7 Ontology.

ABC was motivated by the recognition that many existing metadata approaches are based on a resource-centric traditional cataloguing approach which assumes that the objects being described, and therefore their attributes, are more or less stable. Such an approach is inadequate for modeling the creation, evolution, transition, usage and rights of objects over time or for supporting advanced queries such as who was responsible for what, when and where?

MPEG-7 ontology to describe multimedia learning objects.

Some extensions were made to these ontologies. The final version of our ontology can be found here.

SemanticMedia Ontology.

Server

The server is implemented in java. Is a web server, it receives client requests for:

• Adding a new instance to the Knowledge Base
• Queries about the KB.

Then, it gives these requests to the reasoner, and presents the results back in a web page.

Reasoner

Architecture of Reasoner:



The Reasoner uses is DAMLJessKB, Jess and Jena and a Jess-DAML+OIL Translator. The reasoner receives a daml file; it translates these daml assertions to a jess format using DAMLJessKB (DAMLJessKB executes Jena as a parser). If the request is an addition to the KB, the facts are loaded to Jess and the persistent KB is updated (currently, the KB is stored in a txt file that stores the facts in jess format). If the request is a query, the query is presented to jess and a results file is produced in jess format. This jess-file is sent to the Jess-DAML+OIL Translator.

Translator

This is a Jess+DAML+OIL Translator, implemented in Java. Source code available here.

Original input File

File in jess format

Output of the translator

The daml files (original and output) seem different. But if we compare the Triple graph according to the RDF Validator, we can see that they are equivalent.

Example

Example of Mark-up image

Source Code

The full implementation including DAMLJessKb, Jena, Jess, can be found here.

Source Code

Tools and Links

Tools:

• SMORE Semantic Markup, Ontology and RDF Editor.
• DamlJessKB
• Jena
• Jess

Ontologies and Standards:

• MPEG-7 Standard
• ABC Ontology – Ontology for resources and their lifecycles.
• MPEG7 Ontology – Ontology to describe multimedia content.
• MPEG21 MPEG21 Ontology – Ontology for Rights Metadata Semantics.

Papers:

• The ABC Ontology and Model.
• Enhancing the Semantic Interoperability of Multimedia through a Core Ontology.

Questions/Suggestions Area

We welcome your suggestions.

Have you ever heard of OntoLog? It uses RDF to markup audio and video. -- DavidMihalcik

Thanks, David. We'll look at it.