Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review
Celeste Groenewald, Middlesex
3D gesture based systems are becoming ubiquitous and there are many mid-air hand gestures that exist for interacting with digital surfaces. There is no well-defined gesture set for 3D mid-air hand gestures which makes it difficult to develop applications that have consistent gestures. To understand what gestures exist we conducted the first comprehensive systematic literature review on mid-air hand gestures following existing research methods. The results of the review identified 65 papers where the mid-air hand gestures supported tasks for selection, navigation, and manipulation. We also classified the gestures according to a gesture classification scheme and identified how these gestures have been empirically evaluated. The results of the review provide a richer understanding of what mid-air hand gestures have been designed, implemented, and evaluated in the literature which can help developers design better user experiences for digital interactive surfaces and displays.
Celeste is a second year PhD student at Middlesex University and is working on the VALCRI project (WP3): Insight and Sense-making in Criminal Intelligence Analysis.
Common ground in collaborative intelligence analysis: an empirical study
Dr Sean Xavier Laurence
In this talk, I will briefly cover an empirical exploration of how different configurations of collaboration technology affect peoples’ ability to construct and maintain common ground while conducting collaborative intelligence analysis work. Unlike prior studies of collaboration technology that have typically focused on simpler conversational tasks, or ones that involve physical manipulation, in this talk, the tasks in this study presented focuses on the complex sensemaking and inference involved in intelligence work. The study explores the effects of video communication and shared visual workspace (SVW) on the negotiation of common ground by distributed teams collaborating in real time on intelligence analysis tasks. We theorize that the effect and value of communication cues, or visual cues and awareness nuances, is attenuated when communication is mediated via video, more so than it is in shared visual workspace mediated-communications. In this sense, teams utilizing a remote collaborative framework with fewer visual cues might be expected to work harder to maintain common ground. The experimental study uses a 2×2 factorial, between-subjects design involving two independent variables: presence or absence of Video and SVW. Two-member teams were randomly assigned to one of the four experimental media conditions and worked to complete several intelligence analysis tasks involving multiple, complex intelligence artefacts. Teams with access to the shared visual workspace could view their teammates’ eWhiteboards. Our results demonstrate a significant effect for the shared visual workspace: the effort of conversational grounding is reduced in the cases where SVW is available. However, there were no main effects for video and no interaction between the two variables. Also, we found that the “conversational grounding effort” required tended to decrease over the course of the task.
Dr Sean Xavier Laurence has recently taken up a lecturing position at the Joint Intelligence Training Group, Royal School of Military Survey, THATCHAM, UK, where he leads the MSc teaching modules in Human-Computer Interaction & Information Systems.
Towards an Approach for Analysing External Representations Created During Sensemaking Using Generative Grammar
Efeosasere Okoro and Simon Attfield (presenting)
During sensemaking, users often create external representations to help them make sense of what they know, and what they need to know. In doing so, they necessarily adopt or construct some form of representational language using the tools at hand. By describing such languages implicit in representations we believe that we are better able to describe and differentiate what users do and better able to describe and differentiate interfaces that might support them. Drawing on approaches to the analysis of language, and in particular, Mann and Thompson’s Rhetorical Structure Theory, we analyse the representations that users create to expose their underlying ‘visual grammar’. We do this in the context of a user study involving evidential reasoning. Participants were asked to address an adapted version of the IEEE VAST 2011 mini challenge 3 (interpret a potential terrorist plot implicit in a set of news reports). We show how our approach enables the unpacking of heterogeneous and embedded nature of user-generated representations and allows us to show how visual grammars can evolve and become more complex over time in response to evolving sensemaking needs.
Dr Simon Attfield is Associate Professor of Human Centred Technology at the Interaction Design Centre, Middlesex University. His research involves understanding how people think about and work with information, processes involved in sensemaking, and implications for interactive systems design, including the design and evaluation of information visualisation. He has conducted user-research in military signals intelligence and patterns of life analysis, crime-analysis, news writing, corporate investigations and healthcare. He teaches Human Computer Interaction and Interaction Design. He received a B.A. in Philosophy and a BSc. in Experimental Psychology from Sussex University, and a PhD in Human Computer Interaction from University College London.