*Machine Learning and Pattern Recognition with Multiple

 Modalities Hyungil Ahn and Rosalind W. Picard

 

 This project develops new theory and algorithms to enable

 computers to make rapid and accurate inferences from

 multiple modes of data, such as determining a person's

 affective state from multiple sensors--video, mouse behavior,

 chair pressure patterns, typed selections, or

 physiology. Recent efforts focus on understanding the level

 of a person's attention, useful for things such as

 determining when to interrupt. Our approach is Bayesian:

 formulating probabilistic models on the basis of domain

 knowledge and training data, and then performing inference

 according to the rules of probability theory. This type of

 sensor fusion work is especially challenging due to problems

 of sensor channel drop-out, different kinds of noise in

 different channels, dependence between channels, scarce and

 sometimes inaccurate labels, and patterns to detect that are

 inherently time-varying. We have constructed a variety of

 new algorithms for solving these problems and demonstrated

 their performance gains over other state-of-the-art methods.

 

 http://affect.media.mit.edu/projectpages/multimodal/