SemanticMedia
This project was for the Semantic Web class at UMD. Was developed with Alejandro Rodriguez.
Goal
Online system for Semantic querying of multimedia content.
Example: “Find an image of John Smith (the one at www.cs.umd.edu/~jsmith) in which he is camping”
Project Summary
As multimedia content become more popular, people collect greater amounts of multimedia files, ranging from personal pictures to work-related presentations. As these collections grow they tend to do it in a unorganized fashion which makes them harder to use, (e.g. try looking for that particular picture of the ocean that you took during the summer vacation of 1998). On the other hand, we have already experimented the difficulties of manually marking up personal documents with semantic meaning for the computer. These brings the opportunity of linking these two tasks in a painless manner for the user. One single tool could be used to organize, maintain and publish multimedia documents annotated with semantic markup.
SemanticMedia is intended to be a basic application targeted to non-computer scientist or knowledge engineers that allows users to do exactly what we describe above, in a basic but user-friendly fashion. Among the advantages of this kind of tool, besides the direct utility for personal users, is that they serve as an incentive for the creation of semantic content. Also future extensions of this tool would facilitate automatic inferencing and categorization of multimedia content.
Related work
Ontology-based photo annotation
Features:
- Semantic retrieval (RDFS)
Limitations:
- Domain dependent
- Only images
- No web-based
- Ontologies expressed on RDFS instead of DAML+OIL or OWL
Querying System for Online Images Based on the PNG Format and Embedded Metadata
Features
- Java and open source
- Based on Dublin Core
- Crawling
- Metadata within the image files
Limitations:
- Only PNG files
- Only conjunctive queries
- No standard ontology language
Architecture
General architecture of the system:
- Adding a new instance to the Knowledge Base
- Queries about the KB.
Client
The client is a tool that allows users to markup their documents in daml using our ontology in association with user-specific terms and elements. For this part we decided to use an off the shelf tool: SMORE.
Ontology
The ontology is based on the ABC and MPEG7 Ontology.
ABC was motivated by the recognition that many existing metadata approaches are based on a resource-centric traditional cataloguing approach which assumes that the objects being described, and therefore their attributes, are more or less stable. Such an approach is inadequate for modeling the creation, evolution, transition, usage and rights of objects over time or for supporting advanced queries such as who was responsible for what, when and where?
MPEG-7 ontology to describe multimedia learning objects.
Some extensions were made to these ontologies. The final version of our ontology can be found here.
Server
The server is implemented in java. Is a web server, it receives client requests for:
Then, it gives these requests to the reasoner, and presents the results back in a web page.
Reasoner
Architecture of Reasoner:
The Reasoner uses is DAMLJessKB, Jess and Jena and a Jess-DAML+OIL Translator. The reasoner receives a daml file; it translates these daml assertions to a jess format using DAMLJessKB (DAMLJessKB executes Jena as a parser). If the request is an addition to the KB, the facts are loaded to Jess and the persistent KB is updated (currently, the KB is stored in a txt file that stores the facts in jess format). If the request is a query, the query is presented to jess and a results file is produced in jess format. This jess-file is sent to the Jess-DAML+OIL Translator.
Translator
This is a Jess+DAML+OIL Translator, implemented in Java. Source code available here.
The daml files (original and output) seem different. But if we compare the Triple graph according to the RDF Validator, we can see that they are equivalent.
Example
Source Code
The full implementation including DAMLJessKb, Jena, Jess, can be found here.
Tools and Links
- MPEG-7 Standard
- ABC Ontology – Ontology for resources and their lifecycles.
- MPEG7 Ontology – Ontology to describe multimedia content.
- MPEG21 MPEG21 Ontology – Ontology for Rights Metadata Semantics.
Tools:
Ontologies and Standards:
Papers: