Music Data Analysis: A State-of-the-art Survey
---
paper_title: A Machine Learning Approach to Musical Style Recognition

paper_content:

Much of the work on perception and understanding of music by computers has focused on low-level ::: perceptual features such as pitch and tempo. Our work demonstrates that machine learning can be ::: used to build effective style classifiers for interactive performance systems. We also present an analysis explaining why these techniques work so well when hand-coded approaches have consistently ::: failed. We also describe a reliable real-time performance style classifier.
---
paper_title: Genre classification of symbolic music with SMBGT

paper_content:

Automatic music genre classification is a task that has attracted the interest of the music community for more than two decades. Music can be of high importance within the area of assistive technologies as it can be seen as an assistive technology with high therapeutic and educational functionality for children and adults with disabilities. Several similarity methods and machine learning techniques have been applied in the literature to deal with music genre classification, and as a result data mining and Music Information Retrieval (MIR) are strongly interconnected. In this paper, we deal with music genre classification for symbolic music, and specifically MIDI, by combining the recently proposed novel similarity measure for sequences, SMBGT, with the k-Nearest Neighbor (k-NN) classifier. For all MIDI songs we first extract all of their channels and then transform each channel into a sequence of 2D points, providing information for pitch and duration of their music notes. The similarity between two songs is found by computing the SMBGT for all pairs of the songs' channels and getting the maximum pairwise channel score as their similarity. Each song is treated as a query to which k-NN is applied, and the returned genre of the classifier is the one with the majority of votes in the k neighbors. Classification accuracy results indicate that there is room for improvement, especially due to the ambiguous definitions of music genres that make it hard to clearly discriminate them. Using this framework can also help us analyze and understand potential disadvantages of SMBGT, and thus identify how it can be improved when used for classification of real-time sequences.
---
paper_title: Capturing the Temporal Domain in Echonest Features for Improved Classification Effectiveness

paper_content:

This paper proposes Temporal Echonest Features to harness the information available from the beat-aligned vector sequences of the features provided by The Echo Nest. Rather than aggregating them via simple averaging approaches, the statistics of temporal variations are analyzed and used to represent the audio content. We evaluate the performance on four traditional music genre classification test collections and compare them to state of the art audio descriptors. Experiments reveal, that the exploitation of temporal variability from beat-aligned vector sequences and combinations of different descriptors leads to an improvement of classification accuracy. Comparing the results of Temporal Echonest Features to those of approved conventional audio descriptors used as benchmarks, these approaches perform well, often significantly outperforming their predecessors, and can be effectively used for large scale music genre classification.
---
paper_title: Enriching music mood annotation by semantic association reasoning

paper_content:

Mood annotation of music is challenging as it concerns not only audio content but also extra-musical information. It is a representative research topic about how to traverse the well-known semantic gap. In this paper, we propose a new music-mood-specific ontology. Novel ontology-based semantic reasoning methods are applied to effectively bridge content-based information with web-based resources. Also, the system can automatically discover closely relevant semantics for music mood and thus a novel weighting method is proposed for mood propagation. Experiments show that the proposed method outperforms purely content-based methods and significantly enhances the mood prediction accuracy. Furthermore, evaluations show the system's accuracy could be promisingly increased with the enrichment of metadata.
---
paper_title: Machine Learning Approaches for Mood Classification of Songs toward Music Search Engine

paper_content:

Human often wants to listen to music that fits best his current emotion. A grasp of emotions in songs might be a great help for us to effectively discover music. In this paper, we aimed at automatically classifying moods of songs based on lyrics and metadata, and proposed several methods for supervised learning of classifiers. In future, we plan to use automatically identified moods of songs as metadata in our music search engine. Mood categories in a famous contest about Audio Music Mood Classification (MIREX 2007) are applied for our system. The training data is collected from a LiveJournal blog site in which each blog entry is tagged with a mood and a song. Then three kinds of machine learning algorithms are applied for training classifiers: SVM, Naive Bayes and Graph-based methods. The experiments showed that artist, sentiment words, putting more weight for words in chorus and title parts are effective for mood classification. Graph-based method promises a good improvement if we have rich relationship information among songs.
---
paper_title: A Neural Probabilistic Model for Predicting Melodic Sequences

paper_content:

We present an approach for modelling melodic sequences us- ing Restricted Boltzmann Machines, with an application to folk melody classification. 1Results show that this model's predictive performance is slightly better in our experiment than that of previously evaluated n- gram models (7). The model has a simple structure and in our evaluation it scaled linearly in the number of free parameters with length of the modelled context. A set of these models is used to classify 7 different styles of folk melodies with an accuracy of 61.74%.
---
paper_title: Enhanced peak picking for onset detection with recurrent neural networks

paper_content:

We present a new neural network based peak-picking algo- rithm for common onset detection functions. Compared to existing hand- crafted methods it yields a better performance and leads to a much lower number of false negative detections. The performance is evaluated on basis of a huge dataset with over 25k annotated onsets and shows a signicant improvement over existing methods in cases of signals with previously unknown levels.
---
paper_title: Dance Hit Song Science

paper_content:

With annual investments of several billions of dollars world- wide, record companies can benet tremendously by gaining insight into what actually makes a hit song. This question is tackled in this re- search by focussing on the dance hit song problem prediction problem. A database of dance hit songs from 1985 until 2013 is built, including basic musical features, as well as more advanced features that capture a temporal aspect. Dierent classiers are used to build and test dance hit prediction models. The resulting model has a good performance when predicting whether a song is a \top 10" dance hit versus a lower listed position.
---
paper_title: Automatic Music Classification with jMIR

paper_content:

Automatic music classification is a wide-ranging and multidisciplinary area of inquiry that offers significant benefits from both academic and commercial perspectives. This dissertation focuses on the development of jMIR, a suite of powerful, flexible, accessible and original software tools that can be used to design, share and apply a wide range of automatic music classification technologies. ::: jMIR permits users to extract meaningful information from audio recordings, symbolic musical representations and cultural information available on the Internet; to use machine learning technologies to automatically build classification models; to automatically collect profiling statistics and detect metadata errors in musical collections; to perform experiments on large, stylistically diverse and well-labelled collections of music in both audio and symbolic formats; and to store and distribute information that is essential to automatic music classification in expressive and flexible standardised file formats. ::: In order to have as diverse a range of applications as possible, care was taken to avoid tying jMIR to any particular types of music classification. Rather, it is designed to be a general-purpose toolkit that can be applied to arbitrary types of music classification. Each of the jMIR components is also designed to be accessible not only by users with a high degree of expertise in computer-based research technologies, but also by researchers with valuable musical expertise, but perhaps less of a background in computational research. Moreover, although the jMIR software can certainly be used as a set of ready-to-use tools for solving music classification problems directly, it is also designed to serve as an open-source platform for developing and testing original algorithms. ::: This dissertation also describes several experiments that were performed with jMIR. These experiments were intended not only to verify the effectiveness of the software, but also to investigate the utility of combining information from different types of musical data, an approach with the potential to significantly advance the performance of automatic music classification in general.
---
paper_title: Exploring the music similarity space on the web

paper_content:

This article comprehensively addresses the problem of similarity measurement between music artists via text-based features extracted from Web pages. To this end, we present a thorough evaluation of different term-weighting strategies, normalization methods, aggregation functions, and similarity measurement techniques. In large-scale genre classification experiments carried out on real-world artist collections, we analyze several thousand combinations of settings/parameters that influence the similarity calculation process, and investigate in which way they impact the quality of the similarity estimates. Accurate similarity measures for music are vital for many applications, such as automated playlist generation, music recommender systems, music information systems, or intelligent user interfaces to access music collections by means beyond text-based browsing. Therefore, by exhaustively analyzing the potential of text-based features derived from artist-related Web pages, this article constitutes an important contribution to context-based music information research.
---
paper_title: Combining sources of description for approximating music similarity ratings

paper_content:

In this paper, we compare the effectiveness of basic acoustic features and genre annotations when adapting a music similarity model to user ratings. We use the Metric Learning to Rank algorithm to learn a Mahalanobis metric from comparative similarity ratings in in the MagnaTagATune database. Using common formats for feature data, our approach can easily be transferred to other existing databases. Our results show that genre data allow more effective learning of a metric than simple audio features, but a combination of both feature sets clearly outperforms either individual set.
---
paper_title: A Filter-and-Refine Indexing Method for Fast Similarity Search in Millions of Music Tracks

paper_content:

We present a filter-and-refine method to speed up acoustic audio similarity queries which use the Kullback-Leibler divergence as similarity measure. The proposed method rescales the divergence and uses a modified FastMap [1] implementation to accelerate nearest-neighbor queries. The search for similar music pieces is accelerated by a factor of 10 30 compared to a linear scan but still offers high recall values (relative to a linear scan) of 95 99%. We show how the proposed method can be used to query several million songs for their acoustic neighbors very fast while producing almost the same results that a linear scan over the whole database would return. We present a working prototype implementation which is able to process similarity queries on a 2:5 million songs collection in about half a second on a standard CPU.
---
paper_title: An Approach to Automatic Music Band Member Detection Based on Supervised Learning

paper_content:

Automatically extracting factual information about musical entities, such as detecting the members of a band, helps building advanced browsing interfaces and recommendation systems. In this paper, a supervised approach to learning to identify and to extract the members of a music band from related Web documents is proposed. While existing methods utilize manually optimized rules for this purpose, the presented technique learns from automatically labelled examples, making therefore also manual annotation obsolete. The presented approach is compared against existing rule-based methods for band-member extraction by performing systematic evaluation on two different test sets.
---
paper_title: Multidisciplinary Perspectives on Music Emotion Recognition: Implications for Content and Context-Based Models

paper_content:

The prominent status of music in human culture and every day life is due in large part to its striking ability to elicit emotions, which may manifest from slight variation in mood to changes in our physical condition and actions. In this paper, we first review state of the art stud- ies on music and emotions from dierent disciplines including psychology, musicology and music information retrieval. Based on these studies, we then propose new insights to enhance automated music emotion recog- nition models.
---
paper_title: Automated Music Emotion Recognition: A Systematic Evaluation

paper_content:

Abstract Automated music emotion recognition (MER) is a challenging task in Music Information Retrieval with wide-ranging applications. Some recent studies pose MER as a continuous regression problem in the Arousal-Valence (AV) plane. These consist of variations on a common architecture having a universal model of emotional response, a common repertoire of low-level audio features, a bag-of-frames approach to audio analysis, and relatively small data sets. These approaches achieve some success at MER and suggest that further improvements are possible with current technology. Our contribution to the state of the art is to examine just how far one can go within this framework, and to investigate what the limitations of this framework are. We present the results of a systematic study conducted in an attempt to maximize the prediction performance of an automated MER system using the architecture described. We begin with a carefully constructed data set, emphasizing quality over quantity. We address affect ind...
---
paper_title: Music Emotion Recognition: The Importance of Melodic Features

paper_content:

We study the importance of a melodic audio (MA) feature set in music emotion recognition (MER) and compare its performance to an approach using only standard audio (SA) features. We also analyse the fusion of both types of features. Employing only SA features, the best attained performance was 46.3%, while using only MA features the best outcome was 59.1% (F- measure). A combination of SA and MA features improved results to 64%. These results might have an important impact to help break the so-called glass ceiling in MER, as most current approaches are based on SA features.
---
paper_title: Essentia: An Audio Analysis Library for Music Information Retrieval

paper_content:

Comunicacio presentada a la 14th International Society for Music Information Retrieval Conference, celebrada a Curitiba (Brasil) els dies 4 a 8 de novembre de 2013.
---
paper_title: Contextual Music Information Retrieval and Recommendation: State of the Art and Challenges

paper_content:

Abstract Increasing amount of online music content has opened new opportunities for implementing new effective information access services–commonly known as music recommender systems–that support music navigation, discovery, sharing, and formation of user communities. In the recent years a new research area of contextual (or situational) music recommendation and retrieval has emerged. The basic idea is to retrieve and suggest music depending on the user’s actual situation, for instance emotional state, or any other contextual conditions that might influence the user’s perception of music. Despite the high potential of such idea, the development of real-world applications that retrieve or recommend music depending on the user’s context is still in its early stages. This survey illustrates various tools and techniques that can be used for addressing the research challenges posed by context-aware music retrieval and recommendation. This survey covers a broad range of topics, starting from classical music information retrieval (MIR) and recommender system (RS) techniques, and then focusing on context-aware music applications as well as the newer trends of affective and social computing applied to the music domain.
---
paper_title: Music Search and Recommendation

paper_content:

In the last ten years, our ways to listen to music have drastically changed: In earlier times, we went to record stores or had to use low bit-rate audio coding to get some music and to store it on PCs. Nowadays, millions of songs are within reach via on-line distributors. Some music lovers already got terabytes of music on their hard disc. Users are now no longer desparate to get music, but to select, to find the music they love. A number of technologies has been developed to adress these new requirements. There are techniques to identify music and ways to search for music. Recommendation today is a hot topic as well as organizing music into playlists.
---
paper_title: SoCo: a social network aided context-aware recommender system

paper_content:

Contexts and social network information have been proven to be valuable information for building accurate recommender system. However, to the best of our knowledge, no existing works systematically combine diverse types of such information to further improve recommendation quality. In this paper, we propose SoCo, a novel context-aware recommender system incorporating elaborately processed social network information. We handle contextual information by applying random decision trees to partition the original user-item-rating matrix such that the ratings with similar contexts are grouped. Matrix factorization is then employed to predict missing preference of a user for an item using the partitioned matrix. In order to incorporate social network information, we introduce an additional social regularization term to the matrix factorization objective function to infer a user's preference for an item by learning opinions from his/her friends who are expected to share similar tastes. A context-aware version of Pearson Correlation Coefficient is proposed to measure user similarity. Real datasets based experiments show that SoCo improves the performance (in terms of root mean square error) of the state-of-the-art context-aware recommender system and social recommendation model by 15.7% and 12.2% respectively.
---
paper_title: Implementing situation-aware and user-adaptive music recommendation service in semantic web and real-time multimedia computing environment

paper_content:

With the advent of the ubiquitous era, many studies have been devoted to various situation-aware services in the semantic web environment. One of the most challenging studies involves implementing a situation-aware personalized music recommendation service which considers the user's situation and preferences. Situation-aware music recommendation requires multidisciplinary efforts including low-level feature extraction and analysis, music mood classification and human emotion prediction. In this paper, we propose a new scheme for a situation-aware/user-adaptive music recommendation service in the semantic web environment. To do this, we first discuss utilizing knowledge for analyzing and retrieving music contents semantically, and a user adaptive music recommendation scheme based on semantic web technologies that facilitates the development of domain knowledge and a rule set. Based on this discussion, we describe our Context-based Music Recommendation (COMUS) ontology for modeling the user's musical preferences and contexts, and supporting reasoning about the user's desired emotions and preferences. Basically, COMUS defines an upper music ontology that captures concepts on the general properties of music such as titles, artists and genres. In addition, it provides functionality for adding domain-specific ontologies, such as music features, moods and situations, in a hierarchical manner, for extensibility. Using this context ontology, we believe that logical reasoning rules can be inferred based on high-level (implicit) knowledge such as situations from low-level (explicit) knowledge. As an innovation, our ontology can express detailed and complicated relations among music clips, moods and situations, which enables users to find appropriate music. We present some of the experiments we performed as a case-study for music recommendation.
---
paper_title: dbrec — Music Recommendations Using DBpedia

paper_content:

This paper describes the theoretical background and the implementation of dbrec, a music recommendation system built on top of DBpedia, offering recommendations for more than 39,000 bands and solo artists. We discuss the various challenges and lessons learnt while building it, providing relevant insights for people developing applications consuming Linked Data. Furthermore, we provide a user-centric evaluation of the system, notably by comparing it to last.fm.
---
paper_title: Leveraging Social Media Sources to Generate Personalized Music Playlists

paper_content:

This paper presents MyMusic, a system that exploits social media sources for generating personalized music playlists. This work is based on the idea that information extracted from social networks, such as Facebook and Last.fm, might be effectively exploited for personalization tasks. Indeed, information related to music preferences of users can be easily gathered from social platforms and used to define a model of user interests. The use of social media is a very cheap and effective way to overcome the classical cold start problem of recommender systems. In this work we enriched social media-based playlists with new artists related to those the user already likes. Specifically, we compare two different enrichment techniques: the first leverages the knowledge stored on DBpedia, the structured version of Wikipedia, while the second is based on the content-based similarity between descriptions of artists. The final playlist is ranked and finally presented to the user that can listen to the songs and express her feedbacks. A prototype version of MyMusic was made available online in order to carry out a preliminary user study to evaluate the best enrichment strategy. The preliminary results encouraged keeping on this research.
---
paper_title: Contextualize Your Listening: The Playlist as Recommendation Engine

paper_content:

It is not hyperbole to note that a revolution has occurred in the way that we as a society distribute data and information. This revolution has come about through the confluence of Web-related technologies and the approaching universal adoption of internet connectivity. Add to this mix the normalised use of lossy compression in digital music and the increase in digital music download and streaming services; the result is an environment where nearly anyone can listen to nearly any piece of music nearly anywhere. This is in many respects the pinnacle in music access and availability. Yet, a listener is now faced with a ::: dilemma of choice. Without being familiar with the ever-expanding millions of songs available, how does a listener know what to listen to? If a near-complete collection of recorded music is available what does one listen to next? While the world of music distribution underwent a revolution, the ubiquitous access and availability it created brought new problems in recommendation and discovery. ::: ::: In this thesis, a solution to these problems of recommendation and discovery is presented. We begin with an introduction to the core concepts around the playlist (i.e. sequential ordering of musical works). Next, we examine the history of the playlist as a recommendation technique, starting from before the invention of audio recording and moving through to modern automatic methods. This leads to an awareness that the creation of suitable playlists requires a high degree of knowledge of the relation between songs in a collection (e.g. song similarity). To better inform our base of knowledge of the relationships between songs we explore the use of social network analysis in combination with content-based music information retrieval. In an effort to show the promise of this more complex relational space, a fully automatic interactive radio system is proposed, using audio-content and social network data as a backbone. The implementation of the system is detailed. The creation of this system presents another problem in the area of evaluation. To that end, a novel distance metric between playlists is specified and tested. We then conclude with a discussion of what has been shown and what future work remains.
---

Title: Music Data Analysis: A State-of-the-art Survey

Section 1: Introduction
Description 1: Introduce the importance of music in online activities and the potential for data science research in music data analysis.

Section 2: Prediction and Recognition of Musical Aspects
Description 2: Discuss the use of music data analysis for predicting and recognizing various musical aspects such as style, genre, mood, emotion, onset detection, and song hit prediction.

Section 3: Style
Description 3: Explore techniques and datasets used for recognizing and classifying musical styles using supervised and semi-supervised learning algorithms.

Section 4: Genre
Description 4: Review state-of-the-art techniques for genre detection and classification in music data analysis, including commonly used datasets and classification methods.

Section 5: Mood
Description 5: Cover methodologies and techniques used for mood classification in music, including semantic web technologies and supervised learning algorithms.

Section 6: Melodic Sequence
Description 6: Discuss machine learning approaches for modeling melodic sequences in music, primarily focusing on monophonic melodies.

Section 7: Onset Detection
Description 7: Detail the use of algorithms for detecting the onset of musical events in audio streams.

Section 8: Song Hit Prediction
Description 8: Investigate methods and machine learning techniques used to predict the potential success of songs before their release.

Section 9: Music Classification
Description 9: Examine various approaches and tools used in music classification, including the jMIR software for standardized MIR research.

Section 10: Similarity
Description 10: Analyze techniques used for measuring music similarity, including acoustic features and text-based features extracted from web pages.

Section 11: Emotion
Description 11: Discuss methods for music emotion recognition using melodic and standard audio features, and the potential of semantic web ontology in this field.

Section 12: Audio Analysis
Description 12: Explore the use of open-source libraries like Essentia for audio analysis and their contributions to music information retrieval.

Section 13: Music Recommendation
Description 13: Review various music recommendation systems, including context-aware and social network-based recommendation approaches.

Section 14: Playlist Recommendation
Description 14: Investigate systems and methodologies for generating personalized music playlists using social media data and content-based similarity measures.

Section 15: Discussion
Description 15: Summarize the identified aspects of music data analysis, common datasets used, and additional data types that present opportunities for future research.

Section 16: Conclusions
Description 16: Provide concluding remarks on the state-of-the-art in music data analysis, highlighting technological approaches and opportunities for further exploration.