Music Information Retrieval

The Music Information Retrieval (MIR) / Music Digital Library (MDL) Evaluation Project covers the period from November 2003 to December 2007. The project is being run out of the International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL). IMIRSEL is located at the Graduate School of Library and Information Science (GSLIS) of the University of Illinois at Urbana-Champaign (UIUC). The project comprises two complementary research goals:

1. the establishment of internationally accessible mechanisms and evaluation standards for the comprehensive evaluation of MIR and MDL systems; and,
2. the formal investigation of the human factors involved in the creation, use and evaluation of MIR and MDL systems.

To achieve these research goals, IMIRSEL has divided its work into three mutually supporting research subprojects. In order of presentation in the report, these projects are:

1. The HUMIRS (Human Use of Music Information Retrieval Systems) Project: The HUMIRS subproject is designed to provide answers to the Who, What, Where, When, Why and How questions as they pertain to the use of MIR and MDL systems. By focusing on real-world examples of music information seeking the HUMIRS subproject will allow IMIRSEL to develop a set of experimental MIR/MDL evaluation tasks grounded in reality. This real-world grounding will thus make the set of evaluation tasks much more meaningful as developers prepare their MIR/MDL systems for real-world deployment..

2. The Virtual Research Labs (VRL) using Music-to-Knowledge (M2K) Project: The VRL subproject is being undertaken to provide a uniform mechanism for the international MIR/MDL community to access the standardized resources of IMIRSEL in a robust, yet secure, manner. The VRLs are being constructed using IMIRSEL's M2K rapid prototyping and evaluation environment. M2K is an open-sourced extension of the D2K (Data-to-Knowledge)/Text-to-Knowledge (T2K) Java-based data mining framework, developed by the Automated Learning Group (ALG) at the National Center for Supercomputing Applications (NCSA).
   

3. The Music Information Retrieval Evaluation eXchange (MIREX) Project: MIREX is the evaluation framework under which the MIR/MDL research community comes together to scientifically compare and contrast their technological solutions to a variety of MIR/MDL problems. Based upon community input, MIREX establishes a series of standardized tasks, standardized collections, and standardized evaluation metrics against which researchers submit their algorithms for formal evaluation. MIREX is based, in part, on the TREC approach to the scientific evaluation of information retrieval systems.
   

Useful URLS

Project Website:

http://music-ir.org/evaluation

M2K Demonstration Videos:

http://www.music-ir.org/m2kvid/

Music Information Retrieval Evaluation eXchange Wiki:

http://music-ir.org/mirexwiki

MIREX “Do-It-Yourself” Webservice Prototype:

http://cluster3.lis.uiuc.edu:8080/mirexdiydemo

Recent MIREX Overview Paper:

Downie, J. Stephen. 2006. The Music Information Retrieval Evaluation eXchange (MIREX). In D-Lib Magazine 12 (Issue 12). Available: http://dlib.org/dlib/december06/downie/12downie.html .