Twitter analysis algorithms for Intelligence Circulation System

The convergence of broadcasting and broadband communications network technologies has attracted increasing attention as a means to enrich the television viewing experience of viewers. Toward this end, this study proposes the ‘Intelligence Circulation System (ICS)’, which provides several services, by using newly developed algorithms for analysing Twitter messages. Twitter users often post messages about on-air TV programmes. ICS obtains viewer responses from tweets without requiring any new infrastructure or changes in users’ habits or behaviours, and it generates and provides several outputs to heterogeneous devices based on the analysis results. The algorithms—designed by considering the characteristics of Twitter messages about TV programmes—use auxiliary programme information, similarity between messages, and time series of messages. An evaluation of our algorithms using Twitter messages about all programme genres for a month showed that the accuracy of topic extraction was 85 % for an emphasis on quality (with 56 % of messages processed) and 65 % for an emphasis on quantity (with 95 % of messages processed). The accuracy of message sentimental classification was 66 %. We also describe social recommendation services using the analysis result. We have created a Social TV site for a large-scale field trial, and we have analysed users’ behaviours by comparing four types of social recommendation services on it. The experimental result shows that active and passive communication users had different needs with regard to the recommendations. ICS can generate recommendations for satisfying the needs of both user types by using the analysis result of Twitter messages.     

Consumer preference analysis based on text comments and ratings: A multi-attribute decision-making perspective

With the popularity of social media, extracting consumer preferences from online consumer-generated content is of vital importance for product/service providers to develop tailored marketing strategies. However, existing approaches face difficulties analyzing consumer preferences over different attributes of alternatives (restaurants, hotels, etc.), which hinders product/service providers from comprehensively understanding consumer choice decisions. To address this issue, we solve for the consumer preferences over the attributes represented by attribute weights based on consumers’ historical data, including text comments and overall ratings. Specifically, for each comment and a corresponding rating, we first employ sentiment analysis to calculate values of the attributes, and then develop a quadratic programming model to solve for the weights. Based on a stream of a consumer’s text comments and overall ratings, we can correspondingly obtain a stream of weights indexed by the comment time. We then model this stream of weights as hesitant judgments and employ a hesitant multiplicative programming method to solve for the final weights that fit the consumer’s preferences over attributes at the highest satisficing level. In the application of recommendation, our approach not only provides insights into the consumer’s preferences but also has higher prediction power compared with some state-of-the-art methods.     

A hybrid content-based and item-based collaborative filtering approach to recommend TV programs enhanced with singular value decomposition

With the advent of new cable and satellite services, and the next generation of digital TV systems, people are faced with an unprecedented level of program choice. This often means that viewers receive much more information than they can actually manage, which may lead them to believe that they are missing programs that could likely interest them. In this context, TV program recommendation systems allow us to cope with this problem by automatically matching user’s likes to TV programs and recommending the ones with higher user preference.This paper describes the design, development, and startup of queveo.tv: a Web 2.0 TV program recommendation system. The proposed hybrid approach (which combines content-filtering techniques with those based on collaborative filtering) also provides all typical advantages of any social network, such as supporting communication among users as well as allowing users to add and tag contents, rate and comment the items, etc. To eliminate the most serious limitations of collaborative filtering, we have resorted to a well-known matrix factorization technique in the implementation of the item-based collaborative filtering algorithm, which has shown a good behavior in the TV domain. Every step in the development of this application was taken keeping always in mind the main goal: to simplify as much as possible the user task of selecting what program to watch on TV.     

Opinion-enhanced collaborative filtering for recommender systems through sentiment analysis

The motivation of collaborative filtering (CF) comes from the idea that people often get the best recommendations from someone with similar tastes. With the growing popularity of opinion-rich resources such as online reviews, new opportunities arise as we can identify the preferences from user opinions. The main idea of our approach is to elicit user opinions from online reviews, and map such opinions into preferences that can be understood by CF-based recommender systems. We divide recommender systems into two types depending on the number of product category recommended: the multiple-category recommendation and the single-category recommendation. For the former, sentiment polarity in coarse-grained manner is identified while for the latter fine-grained sentiment analysis is conducted for each product aspect. If the evaluation frequency for an aspect by a user is greater than the average frequency by all users, it indicates that the user is more concerned with that aspect. If a user's rating for an aspect is lower than the average rating by all users, he or she is much pickier than others on that aspect. Through sentiment analysis, we then build an opinion-enhanced user preference model, where the higher the similarity between user opinions the more consistent preferences between users are. Experiment results show that the proposed CF algorithm outperforms baseline methods for product recommendation in terms of accuracy and recall.     

An empirical study on the effect of data sparsity and data overlap on cross domain collaborative filtering performance

In the present day, the oversaturation of data has complicated the process of finding information from a data source. Recommender systems aim to alleviate this problem in various domains by actively suggesting selective information to potential users based on their personal preferences. Amongst these approaches, collaborative filtering based recommenders (CF recommenders), which make use of users’ implicit and explicit ratings for items, are widely regarded as the most successful type of recommender system. However, CF recommenders are sensitive to issues caused by data sparsity, where users rate very few items, or items receive very few ratings from users, meaning there is not enough data to give a recommendation. The majority of studies have attempted to solve these issues by focusing on developing new algorithms within a single domain. Recently, cross-domain recommenders that use multiple domain datasets have attracted increasing attention amongst the research community. Cross-domain recommenders assume that users who express their preferences in one domain (called the target domain) will also express their preferences in another domain (called the source domain), and that these additional preferences will improve precision and recall of recommendations to the user. The purpose of this study is to investigate the effects of various data sparsity and data overlap issues on the performance of cross-domain CF recommenders, using various aggregation functions. In this study, several different cross-domain recommenders were created by collecting three datasets from three separate domains of a large Korean fashion company and combining them with different algorithms and different aggregation approaches. The cross-recommenders that used high performance, high overlap domains showed significant improvement of precision and recall of recommendation when the recommendation scores of individual domains were combined using the summation aggregation function. However, the cross-recommenders that used low performance, low overlap domains showed little or no performance improvement in all areas. This result implies that the use of cross-domain recommenders do not guarantee performance improvement, rather that it is necessary to consider relevant factors carefully to achieve performance improvement when using cross-domain recommenders.     

基于码本聚类和因子分解机的多指标推荐算法

传统的协同过滤推荐算法普遍存在数据稀疏问题,且仅利用单一综合评分来计算用户相似度,无法找到在多个指标上偏好相似的用户,因而影响推荐的准确度。多指标评分推荐算法力图寻找在多个指标上偏好相似的用户,但是其评价成本高,导致数据稀疏性问题更加严重。为了找到与目标用户在多个指标上偏好相似的用户,提出基于码本聚类的思想来获取用户在各指标上的评分风格信息,然后基于评分风格信息将用户和项目在各指标上进行双向聚类,最后利用因子分解机模型(Factorization Machines,FMs)基于同一簇内的用户、项目、多指标评分信息、评分风格信息进行推荐。实验结果表明,与传统的协同过滤算法和其他多指标推荐方法相比,基于多指标评分信息的因子分解机推荐算法能够在一定程度上缓解数据稀疏问题,提高推荐的准确度。     

融合循环知识图谱和协同过滤电影推荐算法

推荐系统对筛选有效信息和提高信息获取效率具有重大的意义。传统的推荐系统会面临数据稀松和冷启动等问题。利用外部评分和物品内涵知识相结合,提出一种基于循环知识图谱和协同过滤的电影推荐模型--RKGE-CF。在充分考虑物品、用户、评分之间的相关性后,利用基于物品和用户的协同过滤进行Top-[K]推荐;将物品的外部附加数据和用户偏好数据加入知识图谱,提取实体相互之间的依赖关系,构建用户和物品之间的交互信息,以便揭示实体与关系之间的语义,帮助理解用户兴趣;将多种推荐结果按不同方法融合进行对比;模型训练时使用多组不同的负样本作为对比,以优化模型;最后利用真实电影数Movielens和IMDB映射连接成新数据集进行测试。实验结果证明该模型对于推荐效果的准确率有显著的提升,同时能更好地解释推荐背后的原因。     

面向网络新闻领域的评论情感极性分析

网络新闻评论情感分析对于互联网时代分析舆情、掌握民调具有重要意义。目前研究聚焦在评论自身的分析而忽略评论间的结构关系,因此利用该关系生成评论关系树,并基于评论关系树建立情感极性判别规则。将评论经过预处理后,同时采用基于扩展情感词典和支持向量机两种方法来进行情感极性分析,动态扩展了情感词典,设计了情感极性分类器。实验结果表明,在利用了评论结构关系之后,两种方法的分析准确率均较没利用该关系之前有了明显的提升。     

Entertainment recommender systems for group of users

Recommender systems are used to recommend potentially interesting items to users in different domains. Nowadays, there is a wide range of domains in which there is a need to offer recommendations to group of users instead of individual users. As a consequence, there is also a need to address the preferences of individual members of a group of users so as to provide suggestions for groups as a whole. Group recommender systems present a whole set of new challenges within the field of recommender systems. In this article, we present two expert recommender systems that suggest entertainment to groups of users. These systems, jMusicGroupRecommender and jMoviesGroupRecommender, suggest music and movies and utilize different methods for the generation of group recommendations: merging recommendations made for individuals, aggregation of individuals’ ratings, and construction of group preference models. We also describe the results obtained when comparing different group recommendation techniques in both domains.     

基于情感分析和LDA主题模型的协同过滤推荐算法

协同过滤推荐算法通常基于物品或用户的相似度来实现个性化推荐,但是数据的稀疏性往往导致推荐精度不理想。大多数传统推荐算法仅考虑用户对物品的总体评分,而忽略了评论文本中用户对物品各个属性面的偏好。该文提出一种基于情感分析的推荐算法SACF(reviews sentiment analysis for collaborative filtering),该算法在经典的协同过滤推荐算法的基础上,考虑评论文本对相似度计算的影响。SACF算法利用LDA主题模型挖掘物品潜在的K个属性面,通过用户在各个属性面上的情感偏好计算用户相似度,从而构建推荐模型。基于京东网上评论数据集的实验结果表明,SACF算法不但可以有效地改善传统协同过滤推荐算法中数据稀疏性的问题,而且提高了推荐系统的精度。     

基于近邻关系的个性化推荐算法研究

协同过滤是目前电子商务推荐系统中广泛应用的最成功的推荐技术,但面临严峻的用户评分数据稀疏性和推荐实时性挑战。针对协同过滤中的数据稀疏问题,提出了一种基于最近邻的个性化推荐算法。通过维数简化技术对评分矩阵进行优化,降低数据稀疏性;采用一种新颖的相似性度量方法计算目标用户的最近邻居,产生推荐预测。实验结果表明,该算法有效地解决了数据稀疏,提高了推荐系统的推荐质量。     

基于弹幕情感分析和聚类算法的视频用户群体分类

随着数字媒体等技术的发展,出现了弹幕系统这种新型的评论模式并逐渐流行。它能够使视频观众即时发布关于视频情节内容的评论,也可以帮助观众理解视频内容。弹幕文本数据的产生,为短文本处理和实时数据处理提供了新的素材。研究弹幕数据的特点和其表达的情感,可以帮助我们更好地理解视频情节;研究弹幕内容之间的相似度进而分析用户之间的关联关系,不仅能够深入了解弹幕用户的特点、发掘不同视频之间的潜在联系,而且可以为视频制作时受众群体的选择提供更为准确的解决方案。首先将弹幕文本数据进行收集和预处理,然后计算这些文本的情感值。针对弹幕文本口语化的特点,建立了网络弹幕常用词词典。通过改进传统的k-means聚类算法,对所有发表弹幕的用户进行基于情感值的分类。这样的分类可以帮助我们了解观看特定类型视频的观众在情感上的异同点。     

基于聚类和用户偏好的协同过滤推荐算法

协同过滤推荐算法使用评分数据作为学习的数据源,针对协同过滤推荐算法中存在的评分数据稀疏以及算法的可拓展性问题,提出了一种基于聚类和用户偏好的协同过滤推荐算法。为了挖掘用户的偏好,该算法引入了用户对项目类型的平均评分到评分矩阵中,并加入了基于用户自身属性的相似度;同时,为了降低数据稀疏性,该算法使用Weighted Slope One算法填充评分数据中的未评分项,并通过融入密度和距离优化初始聚类中心的K-means算法聚类填充后的评分数据中的用户,缩小了相似用户的搜索空间;最后在聚类后的数据集中使用传统的协同过滤推荐算法生成目标用户的推荐结果。通过使用MovieLens100K数据集实验证明,提出的算法对推荐效果有所改善。     

Collaborative user modeling with user-generated tags for social recommender systems

With the popularity of social media services, the sheer amount of content is increasing exponentially on the Social Web that leads to attract considerable attention to recommender systems. Recommender systems provide users with recommendations of items suited to their needs. To provide proper recommendations to users, recommender systems require an accurate user model that can reflect a user’s characteristics, preferences and needs. In this study, by leveraging user-generated tags as preference indicators, we propose a new collaborative approach to user modeling that can be exploited to recommender systems. Our approach first discovers relevant and irrelevant topics for users, and then enriches an individual user model with collaboration from other similar users. In order to evaluate the performance of our model, we compare experimental results with a user model based on collaborative filtering approaches and a vector space model. The experimental results have shown the proposed model provides a better representation in user interests and achieves better recommendation results in terms of accuracy and ranking.     

TV Program Recommendation for Multiple Viewers Based on user Profile Merging

Since today’s television can receive more and more programs, and televisions are often viewed by groups of people, such as a family or a student dormitory, this paper proposes a TV program recommendation strategy for multiple viewers based on user profile merging. This paper first introduces three alternative strategies to achieve program recommendation for multiple television viewers, discusses, and analyzes their advantages and disadvantages respectively, and then chooses the strategy based on user profile merging as our solution. The selected strategy first merges all user profiles to construct a common user profile, and then uses a recommendation approach to generate a common program recommendation list for the group according to the merged user profile. This paper then describes in detail the user profile merging scheme, the key technology of the strategy, which is based on total distance minimization. The evaluation results proved that the merging result can appropriately reflect the preferences of the majority of members within the group, and the proposed recommendation strategy is effective for multiple viewers watching TV together.     

基于非负矩阵分解的用户话题兴趣度算法

Web媒体被公认为继报纸、广播、电视之后的“第四媒体”.而Web2.0的迅速普及,又使当今的Web媒体呈现了一种“自媒体”形式,即每个用户既是信息的接受者,也是信息发布者和信息转发者,在信息传递过程中,用户与用户互动,影响信息传播的进程.用户本身的特性对于传播有很大影响,信息传播依赖于用户个体的行为模式.因此,需要对用户和传播话题之间的关系进行建模,来度量用户对某个话题的感兴趣程度.论文提出了有效的算法来对用户进行感兴趣的话题推荐,该算法基于非负矩阵分解理论,分析用户发表过的内容,将用户感兴趣的话题推荐给该用户.该文针对研究小组下载的真实数据集-科学网数据集进行实验分析,实验结果表明算法能够有效地将用户感兴趣的话题推荐给用户.     

基于Spark并行化改进混合地点推荐

推荐算法是数据挖掘中最重要的算法之一.地点推荐是推荐系统的重要研究内容.针对目前地点推荐面临的数据稀疏、冷启动、个性化程度低等问题,设计并实现了基于Spark并行化处理的改进混合地点推荐模型.该算法融合了基于内容的推荐和基于协同过滤的推荐,结合了用户当前的偏好和其他用户的意见.使用基于用户-地点属性偏好的矩阵填充方式,以此改善数据稀疏性问题;同时,对于海量数据,系统采用Spark分布式集群实现并行计算,缩短了模型训练时间.实验结果表明,与其他推荐算法相比,该算法能有效改善数据稀疏性、提升推荐效果.     

Hybrid recommendation approaches for multi-criteria collaborative filtering

Recommender systems are software tools and techniques for suggesting items in an automated fashion to users tailored their preferences. Collaborative Filtering (CF) techniques, which attempt to predict what information will meet a user’s needs from the neighborhoods of like-minded people, are becoming increasingly popular as ways to overcome the information overload. The multi-criteria based CF presents a possibility to provide accurate recommendations by considering the user preferences in multiple aspects and several methods have been proposed for improving the accuracy of these systems. However, the problem of multi-criteria recommendations with a single and overall rating is still considered an optimization problem. In addition, increasing the accuracy in predicting the appropriate items tailored to the users’ preferences is on of the main challenges in these systems. Hence, in this research new recommendation methods using Adaptive Neuro-Fuzzy Inference Systems (ANFIS) and Self-Organizing Map (SOM) clustering are proposed to improve predictive accuracy of criteria CF. In this research, SOM enables us to generate high quality clusters of dataset and ANFIS is used for discovering knowledge (fuzzy rules) from users’ ratings in multi-criteria dataset, generating appropriate membership functions (MFs), overall rating prediction and input selection. Using exhaustive search method for input selection, the effective inputs are determined to build the ANFIS models in all generated clusters. Furthermore, new fuzzy-based algorithms, Weighted Fuzzy MC-CF (WFuMC-CF), Fuzzy Euclidean MC-CF (FuEucMC-CF) and Fuzzy Average MC-CF (FuAvgMC-CF), are presented for prediction task in multi-criteria CF. FuEucMC-CF and FuAvgMC-CF algorithms uses the fuzzy-based Euclidian distance and fuzzy-based average similarity, respectively, the WFuMC-CF algorithm uses fuzzy-based user- and item-based prediction in a weighted approach. Experimental results on real-world dataset demonstrate that the proposed hybrid methods remarkably improve the accuracy of multi-criteria CF in relation to the previous methods based on multi-criteria ratings.     

Pessimists and optimists: Improving collaborative filtering through sentiment analysis

This work presents a novel application of Sentiment Analysis in Recommender Systems by categorizing users according to the average polarity of their comments. These categories are used as attributes in Collaborative Filtering algorithms. To test this solution a new corpus of opinions on movies obtained from the Internet Movie Database (IMDb) has been generated, so both ratings and comments are available. The experiments stress the informative value of comments. By applying Sentiment Analysis approaches some Collaborative Filtering algorithms can be improved in rating prediction tasks. The results indicate that we obtain a more reliable prediction considering only the opinion text (RMSE of 1.868), than when apply similarities over the entire user community (RMSE of 2.134) and sentiment analysis can be advantageous to recommender systems.