Toward intelligent music information retrieval

Efficient and intelligent music information retrieval is a very important topic of the 21st century. With the ultimate goal of building personal music information retrieval systems, this paper studies the problem of intelligent music information retrieval. Huron points out that since the preeminent functions of music are social and psychological, the most useful characterization would be based on four types of information: genre, emotion, style,and similarity. This paper introduces Daubechies Wavelet Coefficient Histograms (DWCH)for music feature extraction for music information retrieval. The histograms are computed from the coefficients of the db/sub 8/ Daubechies wavelet filter applied to 3 s of music. A comparative study of sound features and classification algorithms on a dataset compiled by Tzanetakis shows that combining DWCH with timbral features (MFCC and FFT), with the use of multiclass extensions of support vector machine,achieves approximately 80% of accuracy, which is a significant improvement over the previously known result on this dataset. On another dataset the combination achieves 75% of accuracy. The paper also studies the issue of detecting emotion in music. Rating of two subjects in the three bipolar adjective pairs are used. The accuracy of around 70% was achieved in predicting emotional labeling in these adjective pairs. The paper also studies the problem of identifying groups of artists based on their lyrics and sound using a semi-supervised classification algorithm. Identification of artist groups based on the Similar Artist lists at All Music Guide is attempted. The semi-supervised learning algorithm resulted in nontrivial increases in the accuracy to more than 70%. Finally, the paper conducts a proof-of-concept experiment on similarity search using the feature set.     

Note segmentation and quantization for music information retrieval

Much research in music information retrieval has focused on query-by-humming systems, which search melodic databases using sung queries. The database retrieval aspect of such systems has received considerable attention, but query processing and the melodic representation have not been examined as carefully. Common methods for query processing are based on musical intuition and historical momentum rather than specific performance criteria; existing systems often employ rudimentary note segmentation or coarse quantization of note estimates. In this work, we examine several alternative query processing methods as well as quantized melodic representations. One common difficulty with designing query-by-humming systems is the coupling between system components. We address this issue by measuring the performance of the query processing system both in isolation and coupled with a retrieval system. We first measure the segmentation performance of several note estimators. We then compute the retrieval accuracy of an experimental query-by-humming system that uses the various note estimators along with varying degrees of pitch and duration quantization. The results show that more advanced query processing can improve both segmentation performance and retrieval performance, although the best segmentation performance does not necessarily yield the best retrieval performance. Further, coarsely quantizing the melodic representation generally degrades retrieval accuracy.     

Capturing the workflows of music information retrieval for repeatability and reuse

Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation—including inputs, parameters, and outputs—is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome? Beyond workflows how much research context can, and should, be captured? We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.     

The neglected user in music information retrieval research

Personalization and context-awareness are highly important topics in research on Intelligent Information Systems. In the fields of Music Information Retrieval (MIR) and Music Recommendation in particular, user-centric algorithms should ideally provide music that perfectly fits each individual listener in each imaginable situation and for each of her information or entertainment needs. Even though preliminary steps towards such systems have recently been presented at the “International Society for Music Information Retrieval Conference” (ISMIR) and at similar venues, this vision is still far away from becoming a reality. In this article, we investigate and discuss literature on the topic of user-centric music retrieval and reflect on why the breakthrough in this field has not been achieved yet. Given the different expertises of the authors, we shed light on why this topic is a particularly challenging one, taking computer science and psychology points of view. Whereas the computer science aspect centers on the problems of user modeling, machine learning, and evaluation, the psychological discussion is mainly concerned with proper experimental design and interpretation of the results of an experiment. We further present our ideas on aspects crucial to consider when elaborating user-aware music retrieval systems.     

Live digital,remember digital: State of the art and research challenges

The so called trend “live digital, remember digital” is acquiring higher relevance within the international research community, due to its several appealing challenges in a multitude of different fields within the Information and Communication Technologies. Today, many people live daily connected to the Internet through their mobile phones, laptops, tablets, etc. and the need to audit or log every single digital interaction emerges in many environments. By seamlessly recording those digital interactions and storing them in a privacy-preserving fashion, a number of benefits are brought to end users, like the provision of user-tailored services, amongst many others. In this paper we will particularly focus on the study of the security and privacy challenges within this field, as well as on the analysis of the currently existing solutions addressing these issues and we will propose an architecture for the so called live digital systems.     

An effective music information retrieval method using three-dimensional continuous DP

This paper describes a music information retrieval system that uses humming as the key for retrieval. Humming is an easy way for a user to input a melody. However, there are several problems with humming that degrade the retrieval of information. One problem is the human factor. Sometimes, people do not sing accurately, especially if they are inexperienced or unaccompanied. Another problem arises from signal processing. Therefore, a music information retrieval method should be sufficiently robust to surmount various humming errors and signal processing problems. A retrieval system has to extract the pitch from the user's humming. However, pitch extraction is not perfect. It often captures half or double pitches, which are harmonic frequencies of the true pitch, even if the extraction algorithms take the continuity of the pitch into account. Considering these problems, we propose a system that takes multiple pitch candidates into account. In addition to the frequencies of the pitch candidates, the confidence measures obtained from their powers are taken into consideration as well. We also propose the use of an algorithm with three dimensions that is an extension of the conventional Dynamic Programming (DP)algorithm, so that multiple pitch candidates can be treated. Moreover, in the proposed algorithm, DP paths are changed dynamically to take deltaPitches and IOIratios (inter-onset-interval) of input and reference notes into account in order to treat notes being split or unified. We carried out an evaluation experiment to compare the proposed system with a conventional system . When using three-pitch candidates with conference measure and IOI features, the top-ten retrieval accuracy was 94.1%. Thus, the proposed method gave a better retrieval performance than the conventional system.     

Learning to embed music and metadata for context-aware music recommendation

Contextual factors greatly influence users’ musical preferences, so they are beneficial remarkably to music recommendation and retrieval tasks. However, it still needs to be studied how to obtain and utilize the contextual information. In this paper, we propose a context-aware music recommendation approach, which can recommend music pieces appropriate for users’ contextual preferences for music. In analogy to matrix factorization methods for collaborative filtering, the proposed approach does not require music pieces to be represented by features ahead, but it can learn the representations from users’ historical listening records. Specifically, the proposed approach first learns music pieces’ embeddings (feature vectors in low-dimension continuous space) from music listening records and corresponding metadata. Then it infers and models users’ global and contextual preferences for music from their listening records with the learned embeddings. Finally, it recommends appropriate music pieces according to the target user’s preferences to satisfy her/his real-time requirements. Experimental evaluations on a real-world dataset show that the proposed approach outperforms baseline methods in terms of precision, recall, F1 score, and hitrate. Especially, our approach has better performance on sparse datasets.     

State of the art content based image retrieval techniques using deep learning: a survey

In the recent years the rapid growth of multimedia content makes the image retrieval a challenging research task. Content Based Image Retrieval (CBIR) is a technique which uses features of image to search user required image from large image dataset according to the user’s request in the form of query image. Effective feature representation and similarity measures are very crucial to the retrieval performance of CBIR. The key challenge has been attributed to the well known semantic gap issue. The machine learning has been actively investigated as possible solution to bridge the semantic gap. The recent success of deep learning inspires as a hope for bridging the semantic gap in CBIR. In this paper, we investigate deep learning approach used for CBIR tasks under varied settings from our empirical studies; we find some encouraging conclusions and insights for future research.

     

Toward an understanding of the history and impact of user studies in music information retrieval

Most Music Information Retrieval (MIR) researchers will agree that understanding users’ needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s, reflecting this growing appreciation of the need for empirical studies of users. However, despite the growing number of user studies and the wide recognition of their importance, it is unclear how great their impact has been in the field: on how systems are developed, how evaluation tasks are created, and how MIR system developers in particular understand critical concepts such as music similarity or music mood. In this paper, we present our analysis on the growth, publication and citation patterns, topics, and design of 198 user studies. This is followed by a discussion of a number of issues/challenges in conducting MIR user studies and distributing the research results. We conclude by making recommendations to increase the visibility and impact of user studies in the field.     

Model-based approaches for interoperability of next generation enterprise information systems: state of the art and future challenges

Enterprise businesses are more than ever challenged by competitors that frequently refine and tailor their offers to clients. In this context, enterprise information systems (EIS) are especially important because: (1) they remain one of the last levers to increase the performance and competitiveness of the enterprise, (2) we operate in a business world where the product itself has reached a limit of performance and quality due to uniform capacity of industrial tools in a globalized economy and (3) the EIS can increase the product value thanks to additional digital services (built on data associated to the product) in order to meet and fit better client’s needs. However, the use of EISs reaches a limit in collaborative environments because enterprises management methods diverge and EISs are mainly inflexible resource packages that are not built with an interoperability objective. Consequently, we need to make EISs interoperable in order to achieve the needed gains competitiveness and performance. This paper contribution can be summarized as follows: (1) it relates existing work and it examines barriers that, at the moment, are preventing further improvements due to current methodological and technological limits, and (2) it proposes a conceptual framework and five challenges that model based approaches must overcome to achieve interoperability between EIS in the near and long term.     

Information retrieval in medicine: state of the art

Conventional information retrieval systems usually involve searching by terms from controlled vocabularies or by individual words in the text. These systems have been commercially successful but are limited by several problems, including cumbersome interfaces and inconsistency with human indexing. Research on methods that automate indexing and retrieval has been performed to address these problems. The three major types of automated systems are vector-based, probabilistic, and linguistic. This article describes these systems and provides an overview of the field of information retrieval in medicine.     

Multimedia data mining: state of the art and challenges

Advances in multimedia data acquisition and storage technology have led to the growth of very large multimedia databases. Analyzing this huge amount of multimedia data to discover useful knowledge is a challenging problem. This challenge has opened the opportunity for research in Multimedia Data Mining (MDM). Multimedia data mining can be defined as the process of finding interesting patterns from media data such as audio, video, image and text that are not ordinarily accessible by basic queries and associated results. The motivation for doing MDM is to use the discovered patterns to improve decision making. MDM has therefore attracted significant research efforts in developing methods and tools to organize, manage, search and perform domain specific tasks for data from domains such as surveillance, meetings, broadcast news, sports, archives, movies, medical data, as well as personal and online media collections. This paper presents a survey on the problems and solutions in Multimedia Data Mining, approached from the following angles: feature extraction, transformation and representation techniques, data mining techniques, and current multimedia data mining systems in various application domains. We discuss main aspects of feature extraction, transformation and representation techniques. These aspects are: level of feature extraction, feature fusion, features synchronization, feature correlation discovery and accurate representation of multimedia data. Comparison of MDM techniques with state of the art video processing, audio processing and image processing techniques is also provided. Similarly, we compare MDM techniques with the state of the art data mining techniques involving clustering, classification, sequence pattern mining, association rule mining and visualization. We review current multimedia data mining systems in detail, grouping them according to problem formulations and approaches. The review includes supervised and unsupervised discovery of events and actions from one or more continuous sequences. We also do a detailed analysis to understand what has been achieved and what are the remaining gaps where future research efforts could be focussed. We then conclude this survey with a look at open research directions.     

Multimedia indexing and retrieval: ever great challenges

In this introduction, we present a brief state of the art of multimedia indexing and retrieval as well as highlight some notions explored in the special issue. We hope that the contributions of this special issue will present ingredients for further investigations on this ever challenging domain. The special issue is actually situated between old problems and new challenges, and contribute to understand the next multimedia indexing and retrieval generation. The contributions explore wide range of fields such as signal processing, data mining and information retrieval.