Experience fusion: The next challenge for CARPE

Abstract

The CARPE research community has been addressing the very interesting question of how to effectively capture, archive, and retrieve personal experiences. The vision has been to provide people with a super-diary or Memex device that captures all one's experiences in multimedia form and provides easy ways to index, search, navigate, and even virtually relive relevant portions of experiences. The primary technology trend that led to the emergence of this research community has been the availability of storage, sensor, and processing technologies to the point that it is possible to continuously archive all media related to personal experiences with small devices at reasonable cost. Good strides have been made in recent years on the central themes of indexing, searching, and presenting personal experiences.However, another major challenge is now emerging for the CARPE community, driven by the ability to share personal experiences. Recent years have seen an exponential increase in weblogs or blogs where people write their diaries online and share them with other people and invite comments. While blogs and their collaborative cousins - wikis - started in text form, people are increasingly adopting multimedia for the experiences they publish and share online. Including images has become commonplace in blogs and sharing videos over the web is the latest trend with hundreds of millions of videos now being downloaded and accessed on a daily basis. What do these trends mean for CARPE research? Effectively dealing with hundreds and thousands of shared experiences is at least as big a challenge as dealing with the lifelong multimedia capture of an individual's experiences. We argue that the next big challenge and opportunity for CARPE research is experience fusion. Experience fusion is the combination of numerous multimedia experiences into a unified and integrated whole. This includes unification and presentation of multiple perspectives on events that happened (what and when), observations of environments (where), people involved (who), procedures followed for tasks or problem solving (how), and explanations or reasoning (why). Experience fusion has the potential to profoundly change many applications - how personal events like weddings and graduations are relived, how history is recorded and reviewed, how situational awareness is gained for security purposes, how enterprises capture best practices, how emergency relief and disaster recovery procedures are performed, how learning and training is done both in school and on the job, and how diagnosis and treatment is performed in healthcare.In this talk, we will present examples of our work in capture and retrieval of personal experiences, illustrate how our work is evolving into the realm of experience fusion, and discuss the host of research challenges that come to the forefront as we attempt to make experience fusion a reality. Copyright 2006 ACM.