Probabilistic Language Modelling for Context-Sensitive Opinion Mining.pdf

Scientific Journal of Ination Engineering October 2015, Volume 5, Issue 5, PP.150-154 Probabilistic Language Modelling for Context- Sensitive Opinion Mining Otto K.M. Cheng, Raymond Y.K. Lau Department of Ination Systems, City University of Hong, Hong Kong SAR Email raylaucityu.edu.hk Abstract Existing opinion mining s often utilize a static lexicon-based approach or a supervised machine learning approach to identify sentiment indicators from texts. Nevertheless, the er often fails to identify context-sensitive semantics of the opinion indicators, and the latter approach requires a large number of human labeled training examples. The main contribution of this paper is the development of a novel opinion mining underpinned by context-sensitive text mining and probabilistic language modeling to improve the effectiveness of opinion mining. Our initial experiments show that the proposed the opinion mining outpers the purely lexicon-based in terms of several benchmark measures. The practical implication of our research work is that business managers can apply our ology to more effectively extract business analytics from user-contributed contents in the Social Web. Keywords Opinion Mining, Context-Sensitive Text Mining, Probabilistic Language Modeling, Artificial Intelligence 1 INTRODUCTION With the rapid growth of the Internet and the Web, increasingly more users have ted or retrieved viewpoints about products, people, or political issues via a variety of Web-based channels such as Blogs, forums, chat-rooms, and social networks. The huge volume of textual contents open the door for automated extraction and analysis of opinions related to the underlying entities such as consumer products. Opinion mining is also referred to as opinion analysis, sentiment analysis, or subjectivity analysis [1,11]. Opinion mining aims at identifying the viewpoints about some subjects rather than simply identifying the subjects [13]. Analyzing the sentiment of messages posted to Blogs or forums may generate huge business values for many organizations who want to develop a timely picture of how their products or services are perceived by the customers [2]. Another potential application of opinion mining is related to analyzing the propaganda and activities of terrorist groups which pose serious threats to national and international security [1]. Opinion mining can be applied to a phrase, a sentence, or an entire document [12]. Most of the existing opinion mining s can be classified into two main categories. The first widely used approach utilizes sentiment lexicon or heuristic rules as the background knowledge to identify opinionated text and to determine the polarity of some opinions [10,11,15]. The another common approach for opinion mining is based on statistical learning s [13,14]. However, a common sentiment lexicon may not be able to capture the context-sensitive nature of an opinion expression. For instance, while the token “unpredictable” may have a negative orientation in a Blog post related to automotive, the same token could have positive orientation such as “unpredictable plot” in a movie review. In fact, the token “unpredictable” is only classified as a negative opinion word in the OpinionFinder lexicon. On the other hand, statistical learning techniques such as supervised machine learning usually requires a large number of labeled training examples in order to build an accurate classifier. Nevertheless, manually annotating the huge number of online messages or Blog posts on the Web is extremely labor intensive and time consuming. The main contributions of this paper are 1 we illustrate a hybrid opinion mining which utilizes a general sentiment lexicon to provide the background knowledge and applies a context- - 150 - http//www.sjie.org sensitive text mining [3] to dynamically extract correlated opinion evidences e.g., “quitting” or “refund” of a particular context e.g., “dissatisfied movie” from the Web. In addition, a novel inferential language model [7] is developed to estimate the polarity of opinionated text segments. The initial experiments which are conducted based on a benchmark movie review collection show that the effectiveness of the proposed context-sensitive opinion mining is promising. 2 CONTEXT-SENSITIVE OPINION MINING A mutual ination based context-sensitive text mining [3,4] is applied to discover some potential opinion evidences which are statistically correlated to some opinion indicators pre-defined in a sentiment lexicon in a chosen Web context. The Web context is constructed by using the Google API to retrieve the first 1,000 documents related to a particular topic i.e., a query. It is believed that the collocations of opinion indicators are also important evidence for sentiment analysis [15]. The Web pages i.e., documents are processed according to standard text pre- processing techniques such as stop word removal, POS tagging, and word stemming. A windowing process is conducted over the collection of Web pages. The windowing process can help reduce the number of noisy collocations extracted from the Web context since terms which are far away from the opinion indicators are less likely to be opinion related evidences. If the association weight between an opinion indicator of the lexicon and a collocation extracted from the Web context is above a pre-defined threshold value, it will be added to the set of dynamically generated context-sensitive opinion evidences. 3 PROBABILISTIC LANGUAGE MODELS FOR OPINION MINING The term “language model” is widely used by the speech recognition community to refer to a probability distribution which represents the statistical regularities for the generation of the language [6]. In other words, a language model is a probabilistic function that assigns a probability to a string drawn from some vocabulary. In the context of Ination Retri IR, a language model is used to estimate the probability that a document generates a query [9]. In particular, such an estimation is used to approximate the measure of the “relevance” of respect to . The basic unigram language model is defined as follows [9,16] 1 2 3 where is the language model of the document . With Jelinek-Mercer smoothing [16], the probability of the document generating a query term i.e., is estimated according to the maximum likelihood model , and the maximum likelihood model of the entire collection . is the Jelinek-Mercer smoothing parameter which may take values in the range of [0.1, 0.7] [16]. The smoothing process is used to alleviate the problem of over-estimating the probabilities for query terms found in a document and the problem of under-estimating the probabilities for terms not found in the document. The term is the term frequency of term in the document , and is the document length measured by the number of tokens contained in the document. It has been shown that applying the probabilities of query related terms of a relevant context instead of the probabilities of the query terms estimated based on the entire collection i.e., a general context to a document language model will lead to a more effective smoothing process, and hence better IR perance [7]. Following the similar line of thinking, we propose an inferential language model to estimate the probability that a document e.g., a Blog post will generate a term found in a Sentiment Lexicon SL. To develop a more robust and effective smoothing process, the inferential language model will take into account terms opinion evidences associated with the opinion indicators in a relevant Web context. In particular, the associated opinion evidences are discovered based on the context-sensitive text mining process over a Web context as illustrated in Section 2. The language model for - 151 - http//www.sjie.org opinion scoring is thus defined by 4 where is the document language model for estimating the probabilities that the document will generates the opinion indicators defined in a sentiment lexicon SL. However, to cope with the challenge that sentiment lexicons are incomplete e.g., context-sensitive opinion evidences are missing, the proposed language model also considers other opinion evidences contained in the document by means of the inferential language model . The set of context-sensitive opinion evidences is dynamically generated according to context-sensitive text mining. 4 EXPERIMENTS AND RESULTS Opinion finding has become one of the main track task in the annual TExt Retri Conference TERC since 2006 [5]. For the TREC opinion finding task also called the Blog Track, opinion finding systems have to first extract the topically relevant documents e.g., Blog posts from a benchmark collection which contains spam and irrelevant documents. Then, an opinion scoring mechanism is applied to promote the rank of the opinion bearing documents and determine the polarities e.g., positive or negative of these documents. The relevance and polarity of each document is pre-defined by some human assessors and classified into various classes such as “0” for non- relevant, “1” for relevant, “2” for negatively opinionated, “3” for mixed opinion, and “4” for positively opinionated. The opinion finding run is used to examine if the opinion finding features of a system does lead to improvement on perance which is measured in terms of Mean Average Precision MAP, R-Precision i.e., the precision after R documents are retrieved; R is the number of relevant and opinionated documents of a chosen topic, and P10 i.e., the precision after ten documents are retrieved. We adopted similar uation procedure of the Blog Track in TREC [5] to uate our system. However, as the actual Blog06 collection was not available when we conducted our initial experiments, we employed a publicly available movie review data set [8] for our uation work instead. The movie review collection contains 1,000 positively opinionated and 1,000 negatively opinionated reviews documents. The Blog06 collection contains 78 non-relevant documents. To construct a collection similar to the Blog06 collection, we used the Google API to retrieve 7,000 Web documents from seven different topics such as “movie history”, “automotive”, “education”, “stock”, etc. These documents were combined with the 2,000 relevant documents from the movie review data set to build our test collection also containing 78 of non-relevant and opinion free documents. Our topic-relevant baseline run Baseline utilized the standard language modeling approach for IR i.e., Eq.1 to Eq.3 to rank our test documents according to the query “Movie Review”; no sentiment lexicon was used for this run. As in the Blog Track of TREC, the first 1,000 retrieved ranked documents were used to compute the MAP for our baseline system. The second experimental run made use of the sentiment lexicon of OpinionFinder to retrieve on-topic and opinionated documents. The OpinionFinder lexicon consists of 2,718 positive opinion indicators and 4,901 negative opinion indicators. In the second run No Context, the inferential language model and the contextually associated terms of the opinion indicators were not used. The original query terms and all the opinion indicators of OpinionFinder were used to build an expanded query which was fed into the basic language model i.e., Eq.1 to Eq.3. The third experimental run With Context was used to uate the effectiveness of our proposed inferential language model and the context-sensitive text mining for the discovery of contextually associated opinion evidences. For the construction of the appropriate Web context, each unique term e.g., “dissatisfied” from the OpinionFinder lexicon was combined with the query terms “movie review” to a Google query. The first 1,000 Web pages returned via the Google API were then used by our text mining program to discover the contextually associated opinion evidences given an particular opinion indicator from OpinionFinder. The top two associated terms i.e., for each opinion indicator were used to build the set . For this experimental run, Eq.1 to Eq.7 were used to compute the document scores. - 152 - http//www.sjie.org FIG.1 PRECISION-RECALL CURVES BEFORE AND AFTER CONTEXT-SENSITIVE OPINION MINING The precision-recall curves corresponding to the Baseline, NoContext, and WithContext runs is shown in Figure 1. As shown in Figure 1, the average precision of the WithContext run outpers the NoContext and Baseline runs at most of the recall points. Our proposed context-aware sentiment analysis is quite effective. The improvement in terms of MAP brought by context-aware opinion mining reaches 28.7 when compared with the baseline ; the context-aware sentiment analysis is also better than the purely lexicon-based by 10.2. 5 CONCLUSIONS The explosive growth of the number of opinion expressions regarding products, people, or political issues ted to the Web opens up great opportunities for organizations to extract valuable business intelligence from the corresponding messages. Existing sentiment analysis s either employ a static lexicon-based approach or a supervised machine learning approach. The er cannot handle the context-aware semantics of opinion indicators, and the latter relies on a large number of human labeled training examples to build an accurate classifier. This paper illustrates a novel sentiment analysis which combines lexicon-based approach with unsupervised learning technique underpinned by context-aware text mining to support more effective context-aware sentiment analysis without requiring large human effort to label training examples. In particular, the proposed opinion scoring mechanism is based on the theoretically rigor probabilistic language modeling approach. Our initial experiments which are based on a benchmark movie review collection and the TREC Blog track procedures show that the proposed context-aware sentiment analysis outpers the purely lexicon-based opinion finding . Future work involves applying other text mining s to bootstrap our opinion mining framework. The comparison between our and other supervised classification s such as SVM will also be conducted. ACKNOWLEDGMENT The work described in this paper was partly supported by a grant from the Science Technology and Innovation Committee of Shenzhen Municipality-the Basic Research Program Projects JCYJ20130401145617281 and JCYJ20140419115614350, and the Shenzhen Research Institute of City University of Hong Kong. REFERENCES [1] Ahmed Abbasi, Hsinchun