The malleable surface touch interface combines a deformable input surface and video processing to provide a whole-hand interface that exhibits many attributes of conventional touch interfaces, such as multi-point and pressure sensitivity. This interface also offer passive haptic feedback, which can be effective with applications such as sculpting or massage.

Downloads

Overview

Imagine massaging someone with your hands and fingers. Now imagine massaging a tabletop in the same way. Not quite the same, right? We have worked with a touchpad interface that is precisely this: hard and unyielding (FlowField Caress). We want to create an interface that can capture complex whole-hand interaction like some interfaces can, but also with a deformable surface that can provide passive haptic feedback.

We use a rubber glove stretched over a rigid frame as the input surface to our interface, and use video capture to track whole-hand interaction. The surface is patterned with dots on the underside, and a webcam is positioned underneath. The video information is captured and processed with plugins written for ImageJ.

The incoming video images are thresholded so the dots can be accurately extracted. Positional displacement of the dots are determined, as well as depth information derived from the change in size of the dots. The resulting set of (x,y,z) coordinates for each dot are then sent to a simulation.

The simulation is built using the KineticsKit prototyping toolkit for Visual Python. A spring mass model is used to represent the dots. Our interface allows for capturing actions such as squeezing, stretching, and twisting, that would be difficult to achieve with a hard-surface interface.

Publications

Contact Information