DEPARTMENT OF ANTHROPOLOGY- HUMAN BEHAVIOR RESEARCH | human motion

Fuzziness of Nonverbal Courtship Communication: Unblurred by automatic movie analysis

Grammer, K., Honda, M., Juette, A. and Schmitt,.A

Fuzziness Of Nonverbal Courtship Communication. Unblurred By Motion Energy Detection. Journal of Personality and Social Psychology. Journal of Personality and Social Psychology, 1999, Vol. 77, No. 3, 509-524

Abstract

A new method for the assessment of qualitative description of nonverbal behavior (automatic movie analysis) is introduced. This model-free method does not use any assumptions on the structure and organization of nonverbal behavior. Cross-cultural comparison (Germany, Japan) of unobtrusively filmed initial interactions between 2 opposite-sex strangers revealed no consistent courtship repertoire of directly observable behavior categories. Furthermore, an extensive analysis of gaze behavior and speech revealed differences between the countries but also showed no consistent relation to interest. Motion energy detection demonstrated in both cultures that female movement quality score (number of movements, duration, size, speed, and complexity) covaries with female interest. This effect is in concordance with the theory that in early stages of interactions manipulative efforts occur in order to avoid possible deception in high-risk situations.

Supplement to the publication

Cross-cultural comparison (Germany, Japan) of unobtrusively filmed initial interactions between two opposite-sex strangers reveals no common courtship repertoire of directly observable behaviour categories. None of 17 tested categories covaries with interest in the other person. This accords with published literature. In contrast, automatic Detection of Motion Energy (MED) in the digitized films (Automatic Movie Analysis, AMA) demonstrates in both cultures that female movement qualities (duration, size, speed, complexity) covary with interest, whereas male movement qualities do not. This sex difference is predicted by theory where in early stages of interactions manipulative efforts occur in order to gain information and to avoid possible deception. AMA throws new light on the functioning of nonverbal (courtship) behaviour by suggesting that movement qualities, not movement forms or patterns, influence the receiver cumulatively: (i) All movement patterns ("categories") are equivalent and convey information through their qualities. This runs counter to the "body-language" view that postulates discrete and meaningful movement patterns. (ii) Movements cumulate their effects over time.

The pictures above show the first frame of two movies created by e-MOTION. The first movie is a Quicktime movie of a female Head-Toss. The second movie shows the difference pictures. Differencing simple calculates the difference between two pictures of the movie: where you find movement it will be visible, where you find no movement the picture will be white. This can be done for different body regions in different persons. The result is a continous data-stream which can be plotted. This data can then be analyzed with EXPRESSION.

The results are discussed in terms of assymmetric investment in the offspring. Females try to elicit information from the male, without disclosing their intentions. This happens under the pressure of possible deception of philandering males, which becomes more likely when the male recognizes the intentions of the female.
Other results indicate that human non-verbal behaviour is highly structured and rythmic. This is clearly demonstrated by the analysis of a two-minute interaction between a male and a female.

An analysis of speech behaviour shows that these qualitative changes are not related to the amount of speech nor to the number of speech acts themselves, thus we are actually looking atr qualitative changes which have their own signal value: when interested females change the quality of their behaviour. Movements are slower, there are more movments and they become smaller. In addition the information content gets higher.

AMA ignores the form of a movement. This may be a strength and a weakness. On the one hand, our results on courtship behaviour corroborate the AMA approach. The absence of reliably identifiable courtship behaviours may be a specificum of mate-choice. On the other hand, behaviour types expressing courtship intentions may have been missed by researchers; the current absence of evidence does not preclude future evidence. In general then, a cautionary conclusion is that both quality and form/Gestalt (and of course context) of nonverbal behaviour may convey meaning. Since quality (speed, size, complexity etc.) and form are not mutually exclusive as information carriers, a key development in future nonverbal behaviour research should be the combined use of AMA and RA approaches. These results also have an impact on communication theories and research on nonverbal behaviour. They show that approaches using only a defined "signal" and effect approach are basically invalid because they do not consider the possibilities of qualitative changes and the tonic effects of repeated signals or repeated changes. Thus traditional communication theories deal only with a small subset of communicative reality. Nonverbal behaviour research recognizes that nonverbal behaviour is difficult to control consciously and that spontaneous expressions differ from posed expressions. We propose a level of communication that is even more difficult to assess and thus the ideal tool for manipulative efforts in communication. This also solves the communicative paradox in which intentions should be signalled without letting the receiver actually know what is going on.

UNIVERSITY OF VIENNA