MultimediaN meets Art

N9C news flash, Amsterdam - Tuesday 20 December 2005

The Centre of Mathematics and Computer Science (CWI) in Amsterdam is crowded today with computer-science researchers posing as pseudo art experts. The project team of E-Culture (MultimediaN subproject N9C) is gathering today to test the web demonstrator of their semantic image annotation and search system. The atmosphere is lively. The leader of the experiment, Laura Hollink, gives instructions to the pack of 13 MultimediaN co-workers from CWI, the two Amsterdam universities (UvA and VU) and Foundation Digital Heritage Netherlands (DEN). The demonstrator contains an initial set of 4,000 images of paintings on the web, which have been automatically annotated.

The project team of E-Culture at work

Today, the manual annotation facilities are being tested. The pseudo experts have to select appropriate categories from the available ontologies (4 million statements in total) to describe the content of paintings. Heated discussions arise ranging from missing ears to nudes on a beach. The exercise takes most part of the afternoon, after which the results are discussed at a meeting. In the evening the birth of the first demonstrator is celebrated with a project dinner in a popular restaurant in the Amsterdam Nes.

Snapshot of the painting demonstrator: search for Derain (French Fauve painter) retrieves also information about artists with similar styles. Biographical information is used to generate the time line at the bottom.

The demonstrator is built as open-source software and is based on web standards, in particular XML, SVG, RDF and OWL. The system can be accessed with a standard web server, that allows users to search the painting database using novel semantic-search techniques. Semantics are provided through four large vocabularies that play the role of background knowledge: the three thesauri of the Getty Foundation, namely the Art and Architecture Thesaurus (120,000 art-related terms) the Union List of Artist Names (300,000 artists with biographical info), and the Thesaurus of Geographical Names (900,000 places), as well as the lexical resource WordNet (120,000 English terms).

Annotating the content of painting images

The project is making its first steps in making this rich source useful for advanced search options, for new presentation paradigms. In 2006 the project will start including Dutch collections of the "Cultuurwijzer" (a Dutch cultural-heritage portal, see Make sure you grab an opportunity to look at this demo, which gives insights into the future of domain-specific search.

Valid XHTML 1.0!