Body movements communicate affective expressions and, in recent years, computational models have been developed to recognize affective expressions from body movements or to generate movements for virtual agents or robots which convey affective expressions. This survey summarizes the state of the art on automatic recognition and generation of such movements. For both automatic recognition and generation, important aspects such as the movements analyzed, the affective state representation used, and the use of notation systems is discussed. The survey concludes with an outline of open problems and directions for future work.

/pdf/body-movements-for-affective-expression-a-survey-of-33hgn5emm8.pdf

Body Movements for Affective Expression: A Survey of Automatic Recognition and Generation

Probability as an Alternative to Boolean LogicWhile logic is the mathematical foundation of rational reasoning and the fundamental principle of computing, it is restricted to problems where information is both complete and certain. However, many real-world problems, from financial investments to email filtering, are incomplete or uncertain in nature. Probability theory and Bayesian computing together provide an alternative framework to deal with incomplete and uncertain data. Decision-Making Tools and Methods for Incomplete and Uncertain DataEmphasizing probability as an alternative to Boolean logic, Bayesian Programming covers new methods to build probabilistic programs for real-world applications. Written by the team who designed and implemented an efficient probabilistic inference engine to interpret Bayesian programs, the book offers many Python examples that are also available on a supplementary website together with an interpreter that allows readers to experiment with this new approach to programming. Principles and Modeling Only requiring a basic foundation in mathematics, the first two parts of the book present a new methodology for building subjective probabilistic models. The authors introduce the principles of Bayesian programming and discuss good practices for probabilistic modeling. Numerous simple examples highlight the application of Bayesian modeling in different fields. Formalism and Algorithms. The third part synthesizes existing work on Bayesian inference algorithms since an efficient Bayesian inference engine is needed to automate the probabilistic calculus in Bayesian programs. Many bibliographic references are included for readers who would like more details on the formalism of Bayesian programming, the main probabilistic models, general purpose algorithms for Bayesian inference, and learning problems. FAQsAlong with a glossary, the fourth part contains answers to frequently asked questions. The authors compare Bayesian programming and possibility theories, discuss the computational complexity of Bayesian inference, cover the irreducibility of incompleteness, and address the subjectivist versus objectivist epistemology of probability. The First Steps toward a Bayesian ComputerA new modeling methodology, new inference algorithms, new programming languages, and new hardware are all needed to create a complete Bayesian computing framework. Focusing on the methodology and algorithms, this book describes the first steps toward reaching that goal. It encourages readers to explore emerging areas, such as bio-inspired computing, and develop new programming languages and hardware architectures.

Bayesian Programming

While single-accelerometers are a common consumer embedded sensors, their use in representing movement data as an intelligent resource remains scarce. Accelerometers have been used in movement recognition systems, but rarely to assess expressive qualities of movement. We present a prototype of wearable system for the real-time detection and classification of movement quality using acceleration data. The system applies Laban Movement Analysis (LMA) to recognize Laban Effort qualities from acceleration input using a Machine Learning software that generates classifications in real time. Existing LMA-recognition systems rely on motion capture data and video data, and can only be deployed in controlled settings. Our single-accelerometer system is portable and can be used under a wide range of environmental conditions. We evaluate the performance of the system, present two applications using the system in the digital arts and discuss future directions.

/pdf/designing-for-movement-evaluating-computational-models-using-5ffyrz7ikz.pdf

Designing for movement: evaluating computational models using LMA effort qualities

• A submitted manuscript is the author's version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers.

Ethics and aesthetics in intelligent product and system design

In this paper, we present a Knowledge Based System for describing and storing dances that takes advantage of the expressivity of Description Logics. We propose exploiting the tools of the Semantic Web Technologies in representing and archiving dance choreographies by developing a Dance Ontology in OWL-2. Description Logics allow us to express complex relations and inference rules for the domain of dance movement, while Reasoning capabilities make it easy to extract new knowledge from existing knowledge. Furthermore, we can search within the ontology based on the steps and movements of dances by writing SPARQL queries. The constructing elements of the ontology and their relationships to construct the dance model are based on the semantics of the Labanotation system, a widely applied language that uses symbols to denote dance choreographies.

A labanotation based ontology for representing dance movement

We present the implementation of computational Laban Movement Analysis (LMA) for human-machine interaction using Bayesian reasoning. The research field of computational human movement analysis is lacking a general underlying modelling language, i.e., how to map the features into symbols. With such a semantic descriptor, the recognition problem can be posed as a problem to recognise a sequence of symbols taken from an alphabet consisting of motion-entities. LMA has been proven successful in areas where humans are observing other humans' movements. LMA provides a model for observation and description and a notational system (Labanotation). To implement LMA in a computer, we have chosen a Bayesian approach. The framework allows us to model the process, learn the dependencies between features and symbols and to perform online classification using LMA-labels. We have chosen the application 'social robots' to demonstrate the feasibility of our solution.

https://ap.isr.uc.pt/archive/2009-10-21_IJRIS_final_JRETT.pdf

Bayesian reasoning for Laban Movement Analysis used in human-machine interaction

Tese de doutoramento em Engenharia Electrotecnica (Instrumentacao e Controlo) apresentada a Faculdade de Ciencias e Tecnologia da Universidade de Coimbra

Robot-human interface using laban movement analysis inside a bayesian framework

This paper presents a framework for gesture recognition by modeling a system based on Dynamic Bayesian Networks (DBNs) from a Marionette point of view. To incorporate human qualities like anticipation and empathy inside the perception system of a social robot remains, so far an open issue. It is our goal to search for ways of implementation and test the feasibility. Towards this end we started tlie development of the guide robot 'Nicole' equipped with a monocular camera and an inertial sensor to observe its environment. The context of interaction is a person performing gestures and 'Nicole' reacting by means of audio output and motion. In this paper we present a solution to the gesture recognition task based on Dynamic Bayesian Network (DBN). We show that using a DBN is a human-like concept of recognizing gestures that encompass the quality of anticipation through the concept of prediction and update. A novel approach is used by incorporating a marionette model in the DBN as a trade-off between simple constant acceleration models and complex articulated models.

Gesture Recognition Using a Marionette Model and Dynamic Bayesian Networks (DBNs)

This paper presents a framework for gesture recognition by modeling a system based on Dynamic Bayesian Networks (DBNs) from a Marionette point of view. To incorporate human qualities like anticipation and empathy inside the perception system of a social robot remains, so far an open issue. It is our goal to search for ways of implementation and test the feasibility. Towards this end we started the development of the guide robot ’Nicole’ equipped with a monocular camera and an inertial sensor to observe its environment. The context of interaction is a person performing gestures and ’Nicole’ reacting by means of audio output and motion. In this paper we present a solution to the gesture recognition task based on Dynamic Bayesian Network (DBN). We show that using a DBN is a human-like concept of recognizing gestures that encompass the quality of anticipation through the concept of prediction and update. A novel approach is used by incorporating a marionette model in the DBN as a trade-off between simple constant acceleration models and complex articulated models.

Gesture recognition using a marionette model and dynamic bayesian networks (DBNs)

This paper presents a novel approach to analyze the appearance of human motions with a simple model i.e. mapping the motions using a virtual marionette model. The approach is based on a robot using a monocular camera to recognize the person interacting with the robot and start tracking its head and hands. We reconstruct 3-D trajectories from 2-D image space (IS) by calibrating and fusing the camera images with data from an inertial sensor, applying general anthropometric data and restricting the motions to lie on a plane. Through a virtual marionette model we map 3-D trajectories to a feature vector in the marionette control space (MCS). This implies inversely that now a certain set of 3-D motions can be performed by the (virtual) marionette system. A subset of these motions are considered to convey information (i.e. gestures). Thus, we are aiming to build up a database which keeps the vocabulary of gestures represented as signals in the MCS. The main contribution of this work is the computational model of the IS-MCS-Mapping. We introduce the guide robot “Nicole” to place our system in an embodied context. We sketch two novel approaches to represent human motion (i.e. Marionette Space and Labananalysis). We define a gesture vocabulary organized in three sets (i.e. Cohen’s Gesture Lexicon, Pointing Gestures and Other Gestures).

Jörg Rett

Papers

Bayesian reasoning for Laban Movement Analysis used in human-machine interaction

Robot-human interface using laban movement analysis inside a bayesian framework

Gesture Recognition Using a Marionette Model and Dynamic Bayesian Networks (DBNs)

Gesture recognition using a marionette model and dynamic bayesian networks (DBNs)

Visual based human motion analysis: mapping gestures using a Puppet model