An important part of our information-gathering behavior has always been to find out what other people think. With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people now can, and do, actively use information technologies to seek out and understand the opinions of others. The sudden eruption of activity in the area of opinion mining and sentiment analysis, which deals with the computational treatment of opinion, sentiment, and subjectivity in text, has thus occurred at least in part as a direct response to the surge of interest in new systems that deal directly with opinions as a first-class object.

This survey covers techniques and approaches that promise to directly enable opinion-oriented information-seeking systems. Our focus is on methods that seek to address the new challenges raised by sentiment-aware applications, as compared to those that are already present in more traditional fact-based analysis. We include material on summarization of evaluative text and on broader issues regarding privacy, manipulation, and economic impact that the development of opinion-oriented information-access services gives rise to. To facilitate future work, a discussion of available resources, benchmark datasets, and evaluation campaigns is also provided.

/pdf/opinion-mining-and-sentiment-analysis-3v5ya3qqiw.pdf

Opinion Mining and Sentiment Analysis

Trust and reputation systems represent a significant trend in decision support for Internet mediated service provision. The basic idea is to let parties rate each other, for example after the completion of a transaction, and use the aggregated ratings about a given party to derive a trust or reputation score, which can assist other parties in deciding whether or not to transact with that party in the future. A natural side effect is that it also provides an incentive for good behaviour, and therefore tends to have a positive effect on market quality. Reputation systems can be called collaborative sanctioning systems to reflect their collaborative nature, and are related to collaborative filtering systems. Reputation systems are already being used in successful commercial online applications. There is also a rapidly growing literature around trust and reputation systems, but unfortunately this activity is not very coherent. The purpose of this article is to give an overview of existing and proposed systems that can be used to derive measures of trust and reputation for Internet transactions, to analyse the current trends and developments in this area, and to propose a research agenda for trust and reputation systems.

/pdf/a-survey-of-trust-and-reputation-systems-for-online-service-2vd0g0048y.pdf

A survey of trust and reputation systems for online service provision

Online feedback mechanisms harness the bidirectional communication capabilities of the Internet to engineer large-scale, word-of-mouth networks. Best known so far as a technology for building trust and fostering cooperation in online marketplaces, such as eBay, these mechanisms are poised to have a much wider impact on organizations. Their growing popularity has potentially important implications for a wide range of management activities such as brand building, customer acquisition and retention, product development, and quality assurance. This paper surveys our progress in understanding the new possibilities and challenges that these mechanisms represent. It discusses some important dimensions in which Internet-based feedback mechanisms differ from traditional word-of-mouth networks and surveys the most important issues related to their design, evaluation, and use. It provides an overview of relevant work in game theory and economics on the topic of reputation. It discusses how this body of work is being extended and combined with insights from computer science, management science, sociology, and psychology to take into consideration the special properties of online environments. Finally, it identifies opportunities that this new area presents for operations research/management science (OR/MS) research.

/pdf/the-digitization-of-word-of-mouth-promise-and-challenges-of-1nibi8no0b.pdf

The Digitization of Word of Mouth: Promise and Challenges of Online Feedback Mechanisms

Online labor markets have great potential as platforms for conducting experiments. They provide immediate access to a large and diverse subject pool, and allow researchers to control the experimental context. Online experiments, we show, can be just as valid—both internally and externally—as laboratory and field experiments, while often requiring far less money and time to design and conduct. To demonstrate their value, we use an online labor market to replicate three classic experiments. The first finds quantitative agreement between levels of cooperation in a prisoner’s dilemma played online and in the physical laboratory. The second shows—consistent with behavior in the traditional laboratory—that online subjects respond to priming by altering their choices. The third demonstrates that when an identical decision is framed differently, individuals reverse their choice, thus replicating a famed Tversky-Kahneman result. Then we conduct a field experiment showing that workers have upward-sloping labor supply curves. Finally, we analyze the challenges to online experiments, proposing methods to cope with the unique threats to validity in an online setting, and examining the conceptual issues surrounding the external validity of online results. We conclude by presenting our views on the potential role that online experiments can play within the social sciences, and then recommend software development priorities and best practices.

/pdf/the-online-laboratory-conducting-experiments-in-a-real-labor-1a5uft5msd.pdf

The online laboratory: conducting experiments in a real labor market

ReviewFeature ReviewHuman cooperation

We conducted the first randomized controlled field experiment of an Internet reputation mechanism. A high-reputation, established eBay dealer sold matched pairs of lots—batches of vintage postcards—under his regular identity and under new seller identities (also operated by him). As predicted, the established identity fared better. The difference in buyers’ willingness-to-pay was 8.1% of the selling price. A subsidiary experiment followed the same format, but compared sales by relatively new sellers with and without negative feedback. Surprisingly, one or two negative feedbacks for our new sellers did not affect buyers’ willingness-to-pay.

/pdf/the-value-of-reputation-on-ebay-a-controlled-experiment-3zq88fp3dr.pdf

The value of reputation on eBay: A controlled experiment

Many empirical studies assess the effectiveness of reputation mechanisms, such as eBay's Feedback Forum. These investigations involve products ranging from pennies to collector guitars; they vary widely in their conclusions on how well reputation systems perform. Part of the explanation for the disparity among prior studies is that they merely collect samples from the eBay population. Such observational studies significantly increase the number of other variables that are left uncontrolled. This makes it difficult to isolate the effects of reputation on auction outcome. In our main experiment, we worked with an established eBay auctioneer to sell matched pairs of items - batches of vintage postcards - under his extremely high reputation identity, and under newcomer identities with little reputation. Our second experiment followed the same format, but compared sales under newcomer identities with and without negative feedback. Having controlled the content of the auctions, and the presentation of item information, we were able to minimize the effects of variables other than reputation. As expected, the established identity fared better. The price difference was 7.6% of the selling price. Back-of-the-envelope calculations indicate that this amount is reasonable, given the level of risk that buyers incur. Surprisingly, one or two negative feedbacks for our new IDs had no price effects, even though these sellers had few positives.

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID385206_code030306530.pdf?abstractid=385206&mirid=1&type=2

The Value of Reputation on eBay: A Controlled Experiment

Many empirical studies assess the effectiveness of reputation mechanisms, such as eBay's Feedback Forum. These investigations involve products ranging from pennies to collector guitars; they vary widely in their conclusions on how well reputation systems perform. Part of the explanation for the disparity among prior studies is that they merely collect samples from the eBay population. Such observational studies significantly increase the number of other variables that are left uncontrolled. This makes it difficult to isolate the effects of reputation on auction outcome. In our main experiment, we worked with an established eBay auctioneer to sell matched pairs of items -- batches of vintage postcards -- under his extremely high reputation identity, and under newcomer identities with little reputation. Our second experiment followed the same format, but compared sales under newcomer identities with and without negative feedback. Having controlled the content of the auctions, and the presentation of item information, we were able to minimize the effects of variables other than reputation. As expected, the established identity fared better. The price difference was 7.6% of the selling price. Back-of-the-envelope calculations indicate that this amount is reasonable, given the level of risk that buyers incur. Surprisingly, one or two negative feedbacks for our new IDs had no price effects, even though these sellers had few positives.

John Swanson

Papers

The value of reputation on eBay: A controlled experiment

The Value of Reputation on eBay: A Controlled Experiment

The Value of Reputation on eBay: A Controlled Experiment