The Neem Platform: An Evolvable Framework for Perceptual Collaborative Applications

This paper introduces the Neem Platform, a generic research test bed for the development of a novel class of adaptive intelligent collaborative applications. These applications provide support for groups of people working together by facilitating their communication and by reacting adaptively to the perceived contexts of an ongoing interaction. Applications in Neem explore the multimodal context of mostly human-to-human communication, by “overhearing” conversations, monitoring user actions and reacting to perceived opportunities for augmentation via intelligent system interventions.The Neem Platform is designed to facilitate the integration of functionality for capture and reification of user communicative actions over a variety of modalities (e.g. speech, text, gestures), as well as facilitate analysis and reasoning based on this reified context. The platform promotes an exploratory and evolutionary style of development by affording rapid development of components, enactment of human-controlled real-time (Wizard of Oz) experiments, and remote collection of data for off-line analysis of interactions.This paper contributes the presentation of a novel coordination mechanism and shows how this underlying mechanism can be used to implement systems that support intelligent reaction to multimodal group interactions.This work was developed at the University of Colorado at Boulder.