面向推荐系统的音乐特征抽取

音乐推荐系统是指根据用户的历史浏览数据,从候选库中推荐给用户可能喜欢的音乐的一种新型网络服务。该系统的关键在于需要对整个数据库按照音乐风格进行分类,基于此提出一种新的音乐特征处理方法来完成音乐库分类,以有效实现音乐推荐。该方法首先为候选音乐库构建常规的音乐特征数据集,然后基于分形理论对数据集进行属性约简,获取每一首音乐的推荐特征向量,并且依据特征向量的特点,定义了一种新的距离度量方法。在包含六种风格的音乐数据库的实验中,仿真结果证明了提出的音乐推荐特征和距离度量的有效性,与现有的基于内容的音乐检索研究相比,音乐推荐特征的使用极大地降低了对数据库存储量的需求,对音乐推荐系统的网络开发具有很好的应用价值。     

Music classification via the bag-of-features approach

A central problem in music information retrieval is audio-based music classification. Current music classification systems follow a frame-based analysis model. A whole song is split into frames, where a feature vector is extracted from each local frame. Each song can then be represented by a set of feature vectors. How to utilize the feature set for global song-level classification is an important problem in music classification. Previous studies have used summary features and probability models which are either overly restrictive in modeling power or numerically too difficult to solve. In this paper, we investigate the bag-of-features approach for music classification which can effectively aggregate the local features for song-level feature representation. Moreover, we have extended the standard bag-of-features approach by proposing a multiple codebook model to exploit the randomness in the generation of codebooks. Experimental results for genre classification and artist identification on benchmark data sets show that the proposed classification system is highly competitive against the standard methods.     

Regional classification of Chinese folk songs based on CRF model

Music regional classification, which is an important branch of music automatic classification, aims at classifying folk songs according to different regional style. Chinese folk songs have developed various regional musical styles in the process of its evolution. Regional classification of Chinese folk songs can promote the development of music recommendation systems which recommending proper style of music to users and improve the efficiency of the music retrieval system. However, the accuracy of existing music regional classification systems is not high enough, because most methods do not consider temporal characteristics of music for both features extraction and classification. In this paper, we proposed an approach based on conditional random field (CRF) which can fully take advantage of the temporal characteristics of musical audio features for music regional classification. Considering the continuity, high dimensionality and large size of the audio feature data, we employed two ways to calculate the label sequence of musical audio features in CRF, which are Gaussian Mixture Model (GMM) and Restricted Boltzmann Machine (RBM). The experimental results demonstrated that the proposed method based on CRF-RBM outperforms other existing music regional classifiers with the best accuracy of 84.71% on Chinese folk songs datasets. Besides, when the proposed methods were applied to the Greek folk songs dataset, the CRF-RBM model also performs the best.

     

音频检索技术研究

结合音频检索发展现状,描述了当前相关研究的进展,介绍了现在最常用到的音频检索方法,讨论了与音频检索相关的关键技术：音频特征提取、音频分割和分类。基于内容的音乐检索研究是一种涉及音乐理论、信号处理、模式识别等相关领域的综合学科研究,其在音乐数据库管理、Internet音乐检索以及生活娱乐等方面都具有非常重要的意义。分析并总结出音乐内容及其检索的概念,给出音乐检索的系统结构,综述了基于内容的音乐检索方法,最后指出了音频检索发展的前景。     

Continuous query processing over music streams based on approximate matching mechanisms

It is foreseen that more and more music objects in symbolic format and multimedia objects, such as audio, video, or lyrics, integrated with symbolic music representation (SMR) will be published and broadcasted via the Internet. The SMRs of the flowing songs or multimedia objects will form a music stream. Many interesting applications based on music streams, such as interactive music tutorials, distance music education, and similar theme searching, make the research of content-based retrieval over music streams much important. We consider multiple queries with error tolerances over music streams and address the issue of approximate matching in this environment. We propose a novel approach to continuously process multiple queries over the music streams for finding all the music segments that are similar to the queries. Our approach is based on the concept of n-grams, and two mechanisms are designed to reduce the heavy computation of approximate matching. One mechanism uses the clustering of query n-grams to prune the query n-grams that are irrelevant to the incoming data n-gram. The other mechanism records the data n-gram that matches a query n-gram as a partial answer and incrementally merges the partial answers of the same query. We implement a prototype system for experiments in which songs in the MIDI format are continuously broadcasted, and the user can specify musical segments as queries to monitor the music streams. Experiment results show the effectiveness and efficiency of the proposed approach.     

A Multi-Resolution Content-Based Retrieval Approach for Geographic Images

Current retrieval methods in geographic image databases use only pixel-by-pixel spectral information. Texture is an important property of geographical images that can improve retrieval effectiveness and efficiency. In this paper, we present a content-based retrieval approach that utilizes the texture features of geographical images. Various texture features are extracted using wavelet transforms. Based on the texture features, we design a hierarchical approach to cluster geographical images for effective and efficient retrieval, measuring distances between feature vectors in the feature space. Using wavelet-based multi-resolution decomposition, two different sets of texture features are formulated for clustering. For each feature set, different distance measurement techniques are designed and experimented for clustering images in a database. The experimental results demonstrate that the retrieval efficiency and effectiveness improve when our clustering approach is used.     

Content-based image retrieval using growing hierarchical self-organizing quadtree map

In this paper, a growing hierarchical self-organizing quadtree map (GHSOQM) is proposed and used for a content-based image retrieval (CBIR) system. The incorporation of GHSOQM in a CBIR system organizes images in a hierarchical structure. The retrieval time by GHSOQM is less than that by using direct image comparison using a flat structure. Furthermore, the ability of incremental learning enables GHSOQM to be a prospective neural-network-based approach for CBIR systems. We also propose feature matrices, image distance and relevance feedback for region-based images in the GHSOQM-based CBIR system. Experimental results strongly demonstrate the effectiveness of the proposed system.     

基于相对特征的音乐哼唱多句检索算法*

面对海量音乐数据,如何在基于内容检索时对其建立索引、提高检索速度是一个非常重要的研究内容。以句为单位对音乐内容提取特征并建立索引时,库中保存的是音乐旋律的相对特征,但用户在哼唱检索时,有时会哼唱一段包含多句的音乐来检索,这时就需要生成多句特征。针对此本文提出单句特征和多句转换及匹配问题的解决方案并应用于检索系统中,获得了较好的检索效果,相关成果也可应用于具有相似结构的时间序列数据的检索系统中。     

Automatic music genre classification based on musical instrument track separation

The aim of this article is to investigate whether separating music tracks at the pre-processing phase and extending feature vector by parameters related to the specific musical instruments that are characteristic for the given musical genre allow for efficient automatic musical genre classification in case of database containing thousands of music excerpts and a dozen of genres. Results of extensive experiments show that the approach proposed for music genre classification is promising. Overall, conglomerating parameters derived from both an original audio and a mixture of separated tracks improve classification effectiveness measures, demonstrating that the proposed feature vector and the Support Vector Machine (SVM) with Co-training mechanism are applicable to a large dataset.     

The design and implementation of a digital music library

The design and implementation of Harbin Institute of Technology—Digital Music Library (HIT-DML) is presented in this paper. Firstly, a novel framework, a music data model, and a query language are proposed as the theoretical foundation of the library. Secondly, music computing algorithms used in the library for feature extracting and matching are described. In addition, indices are introduced for both mining themes of music objects and accelerating content-based information retrieval. Finally, experimental results on the indices and the current development of the library are provided. HIT-DML is distinguished by the following points. First, it is inherently based on database systems, and combines database technologies with multimedia technologies seamlessly. Musical data are structurally stored. Second, it has a solid theoretical foundation, from framework and data model to query language. Last, it can retrieve musical information based on content against different kinds of musical instruments. The indices used, also power the library.     

基于内容的音频检索:概念和方法

Ｆ过去对视觉媒体的检索,如图象和视频,进行了大量的研究。但是我们注意到音频也是多媒体中的一种典型媒体,是信息的一种常用载体。常规的自理是把数字音频当成非结构化流媒体。然而音频是语音的载体、包含丰富的听觉特征,并且具有结构信息。因此需要并且可以基于这些内容对音频进行存取。本文根据当前相关研究的进展,综述基于内容的音频检索方法,包括面向语音、音乐和音频分析的检索、音频分割等;分析并总结出音频内容及其检     

Precise pitch profile feature extraction from musical audio for key detection

The majority of pieces of music, including classical and popular music,are composed using music scales, such as keys. The key or the scale information of a piece provides important clues on its high level musical content, like harmonic and melodic context. Automatic key detection from music data can be useful for music classification, retrieval or further content analysis. Many researchers have addressed key finding from symbolically encoded music(MIDI); however, works for key detection in musical audio is still limited. Techniques for key detection from musical audio mainly consist of two steps:pitch extraction and key detection. The pitch feature typically characterizes the weights of presence of particular pitch classes in the music audio. In the existing approaches to pitch extraction, little consideration has been taken on pitch mistuning and interference of noisy percussion sounds in the audio signals, which inevitably affects the accuracy of key detection. In this paper, we present a novel technique of precise pitch profile feature extraction, which deals with pitch mistuning and noisy percussive sounds. The extracted pitch profile feature can characterize the pitch content in the signal more accurately than the previous techniques, thus lead to a higher key detection accuracy. Experiments based on classical and popular music data were conducted. The results showed that the proposed method has higher key detection accuracy than previous methods, especially for popular music with a lot of noisy drum sounds.     

Adaptive content-based music retrieval system

This paper presents a tunable content-based music retrieval (CBMR) system suitable the for retrieval of music audio clips. The audio clips are represented as extracted feature vectors. The CBMR system is expert-tunable by altering the feature space. The feature space is tuned according to the expert-specified similarity criteria expressed in terms of clusters of similar audio clips. The main goal of tuning the feature space is to improve retrieval performance, since some features may have more impact on perceived similarity than others. The tuning process utilizes our genetic algorithm. The R-tree index for efficient retrieval of audio clips is based on the clustering of feature vectors. For each cluster a minimal bounding rectangle (MBR) is formed, thus providing objects for indexing. Inserting new nodes into the R-tree is efficiently performed because of the chosen Quadratic Split algorithm. Our CBMR system implements the point query and the n-nearest neighbors query with the O(logn) time complexity. Different objective functions based on cluster similarity and dissimilarity measures are used for the genetic algorithm. We have found that all of them have similar impact on the retrieval performance in terms of precision and recall. The paper includes experimental results in measuring retrieval performance, reporting significant improvement over the untuned feature space.     

Tree-based versus distance-based key recognition in musical audio

A tree-based method for the recognition of the tonal center or key in a musical audio signal is presented. Time-varying key feature vectors of 264 synthesized sounds are extracted from an auditory-based pitch model and converted into character strings using PCA-analysis and classification trees. The results are compared with distance-based methods. The characteristics of the new tonality analysis tool are illustrated on various examples. The potential of this method as a building stone in a music retrieval system is discussed.     

TWO GRAMMATICAL INFERENCE APPLICATIONS IN MUSIC PROCESSING

This article reviews our work in the field of music processing (MP) using grammatical inference (GI), where regular grammars are used for modeling musical style. These models can be used to generate automatic composition (AC) and classify music by style (musical style identification) with their resulting applications. The latter, for instance, would improve content-based retrieval in multimedia databases, joining indexing by musical style to other suitable indexes. In this work, several GI techniques are used to learn from examples of melodies, stochastic grammars for different musical styles. Then, each of the learned grammars is used to generate new melodies (composition) or to classify test melodies (style identification). Our studies in this field show the need of proper music coding schemes, so different coding schemes are presented and compared. Results from our previous studies have been improved, achieving in style identification a classification error rate that ranges from 0.5 to 1.7%, depending on the corpus used.     

一种启发式的用哼唱检索音乐的层次化方法

“用哼唱检索音乐”是一种友好的基于内容的音乐检索方法，它已经引起了广泛的研究兴趣；在对音乐库做了统计分析的基础上，总结了一些启发式规则，帮助对哼唱输入进行基音检测、音符分割，哼唱输入表达为音高轮廓图和节奏，音乐库中的音乐按音乐的节奏类型分为不同的节奏区域，并从每首音乐中抽取旋律轮廓图和节奏信息，用递归神经网络记忆旋律轮廓，音乐库的索引是神经网络的权值矩阵，将哼唱输入与音乐库中的音乐匹配的过程就是计算神经网络的输出过程。实验结果显示了所提方法的有效性。     

图像检索中基于粗集理论的特征加权方法

为了提高图像检索的效率，近年来相关反馈机制被引入到基于内容的图像检索领域，而在基于内容的图像检索系统中，多特征融合检索中的特征加权又是一个重要的问题。该文提出了一种新的基于特征加权的相关反馈方法，在粗集理论的基础上，结合用户标记的反馈图像建立决策表，通过决策规则的精度来对多个特征加权，使图像检索和人的感知更加接近。实验表明该方法是有效的，并较Rui的相关反馈方法在性能上有很大提高。