Enabling Feedback in Touch Free Appliance Controls
Learn more about enabling greater feedback for gesture controls in the appliance and white goods markets.
The combination of internet connectivity and touchscreen consumer electronics devices has already influenced product design in the white goods sector. Consumers are looking for styles that fit both a modern, sleek image and their operational requirements and are embracing the use of touchscreens and gesture-based interfaces in ovens and other kitchen, bathroom and utility-room appliances.
Manufacturers of white goods products can now integrate the core technologies used in tablets into their hardware, which shows how far computers have come in interacting with people. Easily understandable gestures are used to zoom in and out of pictures by making a pinching movement with the fingers. Swipe gestures make it easy to scroll through options to find the right information or controls. The touchscreen interface enables designers of white goods to embrace the sleek lines of the modern smartphone or tablet, in the process removing protruding knobs and buttons that tend to act as traps for grease and dirt. The surface is easier to wipe down, helping to keep the product in apparent good condition for longer.
Appliances are becoming smarter, too. They offer more customizable settings. These precise adjustments require a more sophisticated user interface, whilst not compromising on intuitive ease of use. Oven-control modules are now available that combine a high-resolution display with touch controls that let the user swipe through different settings and use apps to program different temperature profiles. These profiles adjust temperature and airflow during cooking to improve the quality of roasts and baked foods.
Refrigerators have begun to adopt touch-based displays to show internal temperature and even connect to the internet to help put shopping lists together and monitor other home tasks, such as displaying a family calendar. Washing machines can show a more detailed set of programs to let the user ensure they have picked the right one for their collection of clothes. The screen can also provide feedback on energy usage and detergent requirements.
But touchscreen technology has its limitations in the white goods environment because of their requirement for physical contact. The interfaces are susceptible to contamination from foodstuffs, dirt, germs and bacteria. Touchless interfaces provide the next step in design and will allow far more flexibility in product design and avoid the problems of smudging on the surface that tablets and smartphones still have. That, in turn, will lead to greater innovation in design with which manufacturers can differentiate themselves in the market and create more attractive products for consumers.
The importance of feedback
Human interactions depend on feedback to confirm engagement. A conversation is a simple example of the requirement for feedback. When delivering a monologue as part of a conversation the speaker relies on visual and audio cues from the listener. Nods of agreement and verbal interjections all make up part of the interaction. This is apparent on a phone call, visual cues eliminated, when there is no verbal feedback the speaker must invite it to ensure the listener is still there.
To use today’s touch-based interfaces, the user depends primarily on visual feedback, which requires controls to have direct line of sight to the user’s eye, and be large enough to read. Although devices can also be set to beep or produce other sounds, these are often less desirable than the touch feedback that consumers have become accustomed to when using conventional buttons and knobs.
One key advantage of a traditional control surface on an appliance is the user’s ability to determine what state the unit is in. Detents on a hob temperature control provide haptic feedback, in the form of mechanical resistance, on what level has been set—and whether that particular hob ring is on or off. There is less need to keep checking at which position the knob has been set or the state of a button. The differences in tactile or haptic feedback let the user know how far through the range of control they have moved. Increasing resistance towards the end of a range of movement helps the user understand how intense a heat the hob ring or oven will produce.
Similarly, tactile clicks around a set of selections quickly let users know into which mode they have put the oven—and reduce the need to keep looking at the dial. This is often important in real-world use where the user may need to quickly turn down the temperature to deal with water in danger of over-boiling or to add more heat after taking the lid off.
Haptic feedback in consumer technology
Haptic feedback has been adopted to a limited extent in consumer electronics. It has been achieved so far through comparatively simple implementations that have become practically synonymous with touch technology. Most users will be familiar with the vibration-only ringtone of a mobile phone. Inside the handset is a small motor attached to a slightly off-center wheel that gently rocks the unit when it turns quickly. Phone makers have moved beyond the vibration alarm to provide apps with the ability to feedback information to the user. For example, rather than an intrusive beep, vibration lets the user know they have entered the wrong password or acknowledge that they have pressed a button.
In automotive applications, haptic feedback has begun to appear mainly in advanced driver assistance systems that alert the driver to potential problems. Some of these systems vibrate the steering wheel when the car begins to stray from its lane on the road or use actuators in the seat to simulate the effect of driving over the rumble strips that often lie to the side of the physical roadway. In studies, lane-departure warnings have been shown to improve lane-keeping by more than 30%, helping to demonstrate the efficacy of haptic feedback.
These applications have given users a taste of the power of haptic feedback but they are relatively simple in terms of implementation. The range of messages that a vibration alarm can send to a user are restricted and often context-specific, which means the message often has to be accompanied by some other form of feedback—visible, audible or both.
A drawback of the motor-driven haptic technologies is that they rely on touch, which can be troublesome in the typical kitchen environment. When cooking, people often find they need to adjust controls with hands covered in flour or oil after preparing the foods they are about to put into the oven or a pan sitting on a hob ring. Touch-based interfaces suffer more from this type of contamination because a layer of flour will cover vital information on the display. And liquids spilled onto the surface can disrupt the ability of the touch controller to properly register the movement of a finger or hand.
Moving the interaction into free space above or in front of the surface of the appliance is one way to avoid the problem of the user having to clean food and water off the surface of the control panel after use. They can turn virtual knobs and activate simulated buttons on the screen using turning and pointing movements, respectively. Such gestural interfaces have been trialled in car dashboards and other systems, taking advantage of the ability of some touchscreen technologies to detect the presence of a finger just above the surface of the display itself.
The problem that the automotive industry has found with gestural interfaces is that users find they often need to repeat the gesture to make sure the device understands it or, as with touch-based control, keep their eyes focused on the display to check. A similar problem emerged in the hands-free faucets used in sink tops. There is often a delay between the sensor recognizing the approach of a hand or a gesture intended to start or stop the water. This small but perceptible delay can be irritating to the user because they often feel they have to wave repeatedly at the sensor.
Haptic feedback technology now exists that can complete the touchless user interface and usher in a new generation of innovative, attractive product design. Born out of years of research at the University of Bristol, a new technology emulates the feel of physical controls in free space—removing the need to touch or handle the display to get haptic feedback.
The technology uses ultrasound generated by transducers mounted under the fascia of a product, or on a separate control panel, to provide the sense of touch up to a meter away from the surface. The key to the technology’s ability to create haptic feedback lies in the interaction between focused ultrasound and the skin.
When the ultrasound is focused onto the surface of the skin, it then reflects off the skin’s surface, creating a vibration in the form of a shear wave in the skin tissue that neuroreceptors pick up and channel to the brain—creating the sense of that part of the hand having touched an object.
The technology uses an array of ultrasound transducers to create one or more focal points of energy on the user’s hand that can be less than 1cm across. The transducers can modulate frequency of the wavelength rapidly to create different types of sensation. The combination of these two approaches lets software generate a wide range of touch-based sensations.
Algorithms have been developed to control the volumetric distribution of the acoustic radiation force field, to form a variety of 3-D shapes. A close focused beam might be used to simulate the curve of a knob of button. A rapidly modulated beam of energy may simulate the feeling of rubbing the hand across a ribbed surface.
Developers and user interface designers can easily experiment with the sensations needed to implement intuitive feedback for gesture using the API supplied with the evaluation system software development kit. Users of the API can generate multiple focal points and work with pre-defined gestures and control sensations. Tools are also under development to allow the customer to generate and modify standard and custom shapes.
To determine where the user’s hands and fingers are located and so determine which virtual controls are being accessed, the technology works with one or more input sensors. One option is to use a camera sensor to allow the recognition of complex hand-based gestures made over the appliance. But other technologies are possible. Radar-like techniques are being explored and there are combinations of rangefinding and image sensors used in products such as the Microsoft Kinect.
One issue with the location of touch controls on appliances such as hobs is the potential for damage by hot saucepans and scalding liquids. Increasing the interaction zone, to up to a meter away from the controls, gives designers greater degrees of freedom in the location of the sensors and actuators. This will give rise to greater innovation in the design of white goods.
Technology for smart goods
The technology provides the flexibility to mount user-interface components so they don’t need to be in close proximity to the hob. For example, by taking advantage of wireless networks to control the appliance a control panel may be mounted on a wall in a location that is more convenient to the home user. This is an approach to design that is likely to become more common as white goods become an integral part of the smart home.
As a sensory-feedback technology, the technology supports the trend of white goods becoming smart goods. The control panel need not be dedicated to just appliance but act as a kitchen hub providing easy access to other functions that range all the way from interactive recipes through to music and media control. The haptic technology can be used to make it more obvious to the home user the mode that the panel is in and which controls have been activated using subtle changes in shapes and sensation. For example, the sensation of a turned knob may be reserved for the hob and oven; sliders may be used for music volume or scrolling through the instructions of a recipe.
Because the sensors around the panel have a better understanding of the position of the user’s hand than traditional user-input technologies such as touchscreens, the context-sensitivity of the overall system can be greatly enhanced. If the system detects the user’s hand moving towards a hot area on an induction hob—which may not appear to be switched on—it can subtly alert them to its state using a gentle buzz against the hand, much like the rumble strips of a lane-departure system in a car. Actuators mounted around the oven and hob can be used to provide detail on the state of each hob ring as well as the oven.
The sensor technology can allow the user, as they sense the alert signal, to then turn their hand to bring the temperature down if they realize they left the ring on accidentally. Haptic feedback can be used to indicate how close the oven is to its target temperature while it is warming up. Similarly, the sense of resistance to a turn can indicate how close to maximum a hob control is.
The technology allows a large degree of freedom in terms of form factor when it comes to mounting the actuators. The form factor need not be a flat rectangle—the mount can be curved or angled. Simulations can be provided to indicate the three dimensional interaction zone area for custom designs.
The support for curved surfaces will allow the technology to support changes in design as tastes change. Although the trend has been towards flat, angular surfaces in recent years to fit the rise of consumer devices such as the tablets and smartphones, device manufacturers are experimenting with curved displays and products. Similarly, in the bathroom, more organic curves are being introduced into design to visually soften lines and surfaces.
As in the kitchen, panels in the bathroom can give the user more fine-tuned control over room temperature as well as that of the water filling the bath. The panel, whether flat or curved to fit the lines of the overall design, can adjust other comfort settings on appliances such as lavatories. The support for haptic feedback means the user need not look at the panel directly to make an adjustment. Haptic actuators and motion sensors over the sink make it possible to turn water on and off without touch but still achieve the haptic feedback of turning a traditional faucet.
The touch interface has helped begin the process of transforming white goods into smart goods. Through easy-to-understand gestures, users can gain much more intuitive control over the products they use. Touchless interfaces that support haptic feedback take gestural interfaces to another level. The touchless nature of the technology is a natural fit for products. The technology not only supports the requirements of safety, intuitiveness and responsiveness, it promotes the ability of manufacturers to create innovative and stylish products that inspire and entice consumers.