An important part of our information-gathering behavior has always been to find out what other people think. With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people now can, and do, actively use information technologies to seek out and understand the opinions of others. The sudden eruption of activity in the area of opinion mining and sentiment analysis, which deals with the computational treatment of opinion, sentiment, and subjectivity in text, has thus occurred at least in part as a direct response to the surge of interest in new systems that deal directly with opinions as a first-class object.

This survey covers techniques and approaches that promise to directly enable opinion-oriented information-seeking systems. Our focus is on methods that seek to address the new challenges raised by sentiment-aware applications, as compared to those that are already present in more traditional fact-based analysis. We include material on summarization of evaluative text and on broader issues regarding privacy, manipulation, and economic impact that the development of opinion-oriented information-access services gives rise to. To facilitate future work, a discussion of available resources, benchmark datasets, and evaluation campaigns is also provided.

/pdf/opinion-mining-and-sentiment-analysis-3v5ya3qqiw.pdf

Opinion Mining and Sentiment Analysis

Trust and reputation systems represent a significant trend in decision support for Internet mediated service provision. The basic idea is to let parties rate each other, for example after the completion of a transaction, and use the aggregated ratings about a given party to derive a trust or reputation score, which can assist other parties in deciding whether or not to transact with that party in the future. A natural side effect is that it also provides an incentive for good behaviour, and therefore tends to have a positive effect on market quality. Reputation systems can be called collaborative sanctioning systems to reflect their collaborative nature, and are related to collaborative filtering systems. Reputation systems are already being used in successful commercial online applications. There is also a rapidly growing literature around trust and reputation systems, but unfortunately this activity is not very coherent. The purpose of this article is to give an overview of existing and proposed systems that can be used to derive measures of trust and reputation for Internet transactions, to analyse the current trends and developments in this area, and to propose a research agenda for trust and reputation systems.

/pdf/a-survey-of-trust-and-reputation-systems-for-online-service-2vd0g0048y.pdf

A survey of trust and reputation systems for online service provision

A wide variety of media can be used in learning, including distance learning, such as print, lectures, conference sections, tutors, pictures, video, sound, and computers. Any one instance of distance learning will make choices among these media, perhaps using several.

Multimedia in Learning

Online feedback mechanisms harness the bidirectional communication capabilities of the Internet to engineer large-scale, word-of-mouth networks. Best known so far as a technology for building trust and fostering cooperation in online marketplaces, such as eBay, these mechanisms are poised to have a much wider impact on organizations. Their growing popularity has potentially important implications for a wide range of management activities such as brand building, customer acquisition and retention, product development, and quality assurance. This paper surveys our progress in understanding the new possibilities and challenges that these mechanisms represent. It discusses some important dimensions in which Internet-based feedback mechanisms differ from traditional word-of-mouth networks and surveys the most important issues related to their design, evaluation, and use. It provides an overview of relevant work in game theory and economics on the topic of reputation. It discusses how this body of work is being extended and combined with insights from computer science, management science, sociology, and psychology to take into consideration the special properties of online environments. Finally, it identifies opportunities that this new area presents for operations research/management science (OR/MS) research.

/pdf/the-digitization-of-word-of-mouth-promise-and-challenges-of-1nibi8no0b.pdf

The Digitization of Word of Mouth: Promise and Challenges of Online Feedback Mechanisms

Online labor markets have great potential as platforms for conducting experiments. They provide immediate access to a large and diverse subject pool, and allow researchers to control the experimental context. Online experiments, we show, can be just as valid—both internally and externally—as laboratory and field experiments, while often requiring far less money and time to design and conduct. To demonstrate their value, we use an online labor market to replicate three classic experiments. The first finds quantitative agreement between levels of cooperation in a prisoner’s dilemma played online and in the physical laboratory. The second shows—consistent with behavior in the traditional laboratory—that online subjects respond to priming by altering their choices. The third demonstrates that when an identical decision is framed differently, individuals reverse their choice, thus replicating a famed Tversky-Kahneman result. Then we conduct a field experiment showing that workers have upward-sloping labor supply curves. Finally, we analyze the challenges to online experiments, proposing methods to cope with the unique threats to validity in an online setting, and examining the conceptual issues surrounding the external validity of online results. We conclude by presenting our views on the potential role that online experiments can play within the social sciences, and then recommend software development priorities and best practices.

/pdf/the-online-laboratory-conducting-experiments-in-a-real-labor-1a5uft5msd.pdf

The online laboratory: conducting experiments in a real labor market

We conducted the first randomized controlled field experiment of an Internet reputation mechanism. A high-reputation, established eBay dealer sold matched pairs of lots—batches of vintage postcards—under his regular identity and under new seller identities (also operated by him). As predicted, the established identity fared better. The difference in buyers’ willingness-to-pay was 8.1% of the selling price. A subsidiary experiment followed the same format, but compared sales by relatively new sellers with and without negative feedback. Surprisingly, one or two negative feedbacks for our new sellers did not affect buyers’ willingness-to-pay.

/pdf/the-value-of-reputation-on-ebay-a-controlled-experiment-3zq88fp3dr.pdf

The value of reputation on eBay: A controlled experiment

Sketching is a powerful means of working out and communicating ideas. Sketch understanding involves a combination of visual, spatial, and conceptual knowledge and reasoning, which makes it both challenging to model and potentially illuminating for cognitive science. This paper describes CogSketch, an ongoing effort of the NSF-funded Spatial Intelligence and Learning Center, which is being developed both as a research instrument for cognitive science and as a platform for sketch-based educational software. We describe the idea of open-domain sketch understanding, the scientific hypotheses underlying CogSketch, and provide an overview of the models it employs, illustrated by simulation studies and ongoing experiments in creating sketch-based educational software.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1756-8765.2011.01149.x

CogSketch: sketch understanding for cognitive science research and for education.

Many empirical studies assess the effectiveness of reputation mechanisms, such as eBay's Feedback Forum. These investigations involve products ranging from pennies to collector guitars; they vary widely in their conclusions on how well reputation systems perform. Part of the explanation for the disparity among prior studies is that they merely collect samples from the eBay population. Such observational studies significantly increase the number of other variables that are left uncontrolled. This makes it difficult to isolate the effects of reputation on auction outcome. In our main experiment, we worked with an established eBay auctioneer to sell matched pairs of items - batches of vintage postcards - under his extremely high reputation identity, and under newcomer identities with little reputation. Our second experiment followed the same format, but compared sales under newcomer identities with and without negative feedback. Having controlled the content of the auctions, and the presentation of item information, we were able to minimize the effects of variables other than reputation. As expected, the established identity fared better. The price difference was 7.6% of the selling price. Back-of-the-envelope calculations indicate that this amount is reasonable, given the level of risk that buyers incur. Surprisingly, one or two negative feedbacks for our new IDs had no price effects, even though these sellers had few positives.

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID385206_code030306530.pdf?abstractid=385206&mirid=1&type=2

The Value of Reputation on eBay: A Controlled Experiment

In this paper, we describe CogSketch, an open-domain sketch understanding system built on the nuSketch architecture. CogSketch captures the multi-modal, unconstrained nature of sketching by focusing on reasoning over recognition. We describe this approach, as well as two application domains for CogSketch: cognitive modeling, and education.

/pdf/cogsketch-open-domain-sketch-understanding-for-cognitive-336vg80qzi.pdf

CogSketch: open-domain sketch understanding for cognitive science research and for education

Sketching is often used when working out ideas. This combination of drawing and conceptual labeling is a very natural and effective form of communication and problemsolving. Creating software that can participate in sketching provides many challenges. We outline the nuSketch approach to sketch understanding, which focuses on visual and conceptual understanding instead of recognition. We summarize three experiments in progress with the sketching Knowledge Entry Associate (sKEA), the first open-domain sketch understanding system. sKEA exploits a variety of visual and qualitative spatial reasoning capabilities and human-like analogical matching to tackle a variety of tasks. We present experimental results, and outline future plans. The nuSketch approach Sketching is a form of multimodal interaction where participants use a combination of interactive drawing and language to provide high-bandwidth communication. This communication relies on shared understanding, both visual and conceptual. Multimodal interface research has mainly focused on providing more natural interfaces to legacy software, using recognition technologies to provide the desired interaction (Alvarado and Davis 2001, Cohen et al. 1997). While such systems have been shown to be quite useful in practice, we take a radically different approach. The nuSketch approach is based on a key observation about human-to-human sketching: Recognition is not essential. People are not artists in real time; they rely on language for conceptual labeling much of the time. While some specialized domains have visual symbol languages that practitioners use fluently, sketching is used far more broadly than that. In other words, for people, recognition is an accelerant, not a necessity. This suggests a very different approach to sketch understanding that complements recognition-oriented research. In the nuSketch approach, we sidestep recognition issues by providing other interface mechanisms for people to conceptually label their ink. This provides two crucial advantages. First, it enables us to focus on deeper visual and conceptual understanding of sketches. Second, it enables us to build sketching systems that can operate outside the tight domain constraints that bind today’s recognition-based systems. In particular, the sketching Knowledge Entry Associate (sKEA) (Forbus and Usher 2002) is, we believe, the first open-domain sketch understanding system. sKEA’s only coverage limitation comes from the underlying knowledge base it uses – currently a subset of Cyc, consisting of over 35,000 concepts, constrained by 1.2M facts. This does not mean that we can effectively reason with all of these concepts in sketches yet – that is the nature of the challenge we have undertaken! The nuSketch Architecture and sKEA In the nuSketch architecture, the basic unit in a sketch is a glyph. Every glyph has ink and content. The ink consists of one or more poly-lines, representing what the user drew. The content is a conceptual entity, the kind of thing that the glyph is representing. There are two key problems that any sketching system must solve: 1. Segmentation. What pieces of ink should be considered together as glyphs? 2. Conceptual Labeling. What conceptual entity does a glyph denote? We solve the segmentation problem by having the user click a button when they begin drawing a glyph and click it again when they are finished. Other segmentation techniques, such as time-outs and connectivity, are in our experience very frustrating for users and error-prone. We solve the conceptual labeling problem by enabling the content of a glyph to be declared, via a simple dialog, as one or more types drawn from the underlying knowledge base. This requires users to be familiar with the subset of the knowledge base that they need in their task, which is a strong limitation with the current version of the system. (We return to this issue at the end.) The nuSketch architecture uses a variety of geometric computations to visually construct qualitative representations, including RCC8 relations (Cohen 1996), Voronoi diagrams (Edwards and Moulin 1998) for approximating proximity, and polygon operations to capture domain constraints. A central feature of nuSketch is our use of analogical processing, based on Gentner’s structure-mapping theory (Gentner 1983). Analogy provides a powerful means of entering and testing knowledge. Currently sKEA enables users to compare two layers of a sketch, which enables the detection of similarities and differences. We use the Structure-Mapping Engine (SME) (Falkenhainer, Forbus and Gentner 1989) to perform the comparisons. SME is a general-purpose analogical matcher. sKEA’s analogies are based on both the visual and the conceptual material in a sketch. Some of our experiments also take advantage of the MAC/FAC system (Forbus, Gentner and Law 1994) which does a coarse matching using content vectors and then uses SME to narrow the results. SME produces candidate inferences, conjectures about one description based on its alignment with another. Candidate inferences are useful in knowledge capture because they suggest ways to flesh out a description based on similarities with prior knowledge. Since there is independent psychological evidence that structural alignment occurs in visual processing, and that SME captures many aspects of this processing accurately, it means that when our sense of similarity and our software’s sense of similarity about a sketch diverge, it is a sign that our representations have failed to capture something crucial about the sketch. This provides a powerful constraint that drives the visual and conceptual representations we compute. Even if one does not care about modeling human performance, a sketching system will be a better partner if you and it agree on when things look alike. Spatial Processing in nuSketch and sKEA While we do not use recognition techniques on our sketches, we do compute some simple spatial properties of the ink (Forbus and Usher 2002). We focus on the spatial relationships between the glyphs rather than doing detailed analysis of the structure of the glyphs themselves; we call this approach blob semantics. When a glyph is added, moved, or resized, sKEA computes a set of spatial attributes and relationships. This process is described in detail in (Forbus, Tomai, and Usher 2003). The spatial attributes and relationships that make up the visual structure of the sketch are: groupings, positional relationships, size, and orientation. sKEA automatically computes two kinds of groupings: contained glyph groups and connected glyph groups. A contained group consists of a single container glyph and the set of glyphs that are fully contained within it, possibly tangentially so. The contained group does not include glyphs that are contained within other glyphs in the group. A connected glyph group consists of a set of glyphs that overlap ink strokes with one another. Articulation points can be computed over connected glyph groups and tangentially connected pairs of glyphs can be noted as such. Positional relationships are computed pair-wise and expressed in a viewer-oriented coordinate system of left/right and above/below. They are not computed between all pairs of glyphs, but rather in local neighborhoods based on adjacency, as determined via a Voronoi diagram. Positional relationships are computed only between glyphs on the same layer of a sketch. Glyph size in sKEA is assigned as either tiny, small, medium, large or huge. Sizes are based on the area of a glyph’s axis-aligned bounding box, a coarse but empirically useful approximation. Glyph areas are normalized with respect to either the area of the bounding box around all glyphs on all layers or the users view port, whichever is larger. The normalized areas are then clustered into qualitative size values based on a logarithmic scale of the square root of the area. Additional Spatial Reasoning As part of our ongoing research we are developing more spatial reasoning capabilities to accomplish different research goals. Currently we are working on adding the ability to articulate the “important” points and segments on different objects as well as to more specifically define the relationship between adjacent glyphs. Another area where we need to expand our available spatial reasoning abilities is curvature. We are working on different techniques to qualitative summarize degree of curvature.

/pdf/open-domain-sketch-understanding-the-nusketch-approach-3hua7uq36t.pdf

Kate Lockwood

Papers

The value of reputation on eBay: A controlled experiment

CogSketch: sketch understanding for cognitive science research and for education.

The Value of Reputation on eBay: A Controlled Experiment

CogSketch: open-domain sketch understanding for cognitive science research and for education

Open-Domain Sketch Understanding: The nuSketch Approach.