Ambient Kitchen (CELS)

Jump to: About | Build | Software | Team | Technology | Dataset

Ambient Kitchen - About

Funder: CELS (£30K)

Newcastle research team: Patrick Olivier, Cuong Pham, Tom Bartindale, Dan Jackson, Cassim Ladha, Karim Ladha, Jack Weeden, Thomas Ploetz, Nils Hammerla.

The kitchen provides a platform for explaining and exploring the application of pervasive computing technology in a domestic setting. Building on previous technical developments at Newcastle University in which we have used pervasive computing to support older people, elements of the physical environment and appliances are instrumented, both with sensors and displays, which allows both wireless collection of activity data and the display of helpful information.

Ambient Kitchen - Build

Our commitment to developing a useable kitchen area overrided factors that would have made it easier to install the technology. We aimed to create an environment that is free from the usual home-automation "gadgets". In sympathy with Mark Weiser's vision of the "computer of the 21st century", the projectors, wireless sensor network, RFID readers, cameras, and floor sensors had to disappear into the fabric of the kitchen itself.

Douglas Lamond, an independent artist and craftsman, designed and built the physical structure. After understanding the limitations of the technology Doug developed an environment that has both the aesthetics and functionality that we look for in a kitchen. The physical structure can even be dismantled, and reassembled at another location.

From floor to frame...

The frame is built, and the once quiet empty corner of our office (Space 2 in Culture Lab) becomes a building site. The panels on the walls will eventually conceal the cameras, and the pressure sensitive flooring will be placed between the floor visible here, and the laminate which will provide the final finish.

Some assembly required...

With the frame assembled, it's time to add a small roof area for spot lights, and paint the walls. The kitchen cabinets are assembled by the army of helpers, who despite being intelligent people, find it impossible to follow IKEA's instructions. Finally, everything is done, the kitchen is finished, dressed, and ready for the installation of the software.

The finished kitchen

Under the hood...

The RFID readers and projectors are now safely hidden beneath the work surface. The display is projected through small uplighter-style openings in the work surface onto a long mirror fixed under the overhead cabinets. The mirror reflects the projected display onto the walls. RFID tags are concealed in the packaging of all food products, replacing the barcodes as they are likely to do in a few years time. The nodes of the sensor network are placed in the cupboards, on the kitchen utensils and in the food containers. Where it's been impossible to conceal these Mark McLeish and Rachel Colley (students of Jayne Wallace, the group's digital jeweller) have designed stylish covers. 

Ambient Kitchen - Software

What it Does...

Since the Ambient Kitchen is a research platform, the software that we are developing is in a continuous state of development and redevelopment. Our initial system demonstrates both contextual information displays for general users. In the future we plan to use the kitchen to explore ways of prompting people through multi-step cooking tasks. These are not "interfaces" in the traditional sense as there are no keyboards, mice or conventional input devices. Instead, the displays respond to the activities that take place in the kitchen. The RFID tags attached to everyday household items trigger media, produce recipes, or prompt users about scheduled events. The concealed projectors (and speakers) project all this on an 80" display across one wall of the kitchen. Simple color themes, frameless designs, and gentle floating text yield a calm ambient display, allowing the presentation of contextual information without the look and feel of a conventional interface.

Recipe recommender

The recipe recommender reacts to the items the user places on the work surface, selecting recipes which most closely match the items chosen. The placement of the chopping board initiates the recommender, which then responds to food items taken out of the cupboards. The Ambient Kitchen can sense which ingredients the user has available, presents the nutritional value of the recipe they have chose, as well as step-by-step preparation instructions.

The left image shows the initial state of the recipe software. Adding items to the work surface changes the recipes, which at this stage are simply ingredients, and a brief description. The red and green lights indicate whether the ingredient is in the kitchen or not. 

Once the user stops adding items to the work surface, the system uses the position of the chopping board to determine which recipe the user would like to use. This can be changed by sliding the board along the bench in front of the preferred recipe. When the recipe is chosen, more information about it is displayed. This extra information includes the preparation instructions and its nutritional value.

Medication reminder

People with dementia often struggle to remember to take the right medication at the right time. The medication reminder displays reminders when a medication is due. The Ambient Kitchen tracks any items involved in the reminder, and resets the reminder only when the item has been removed from a cupboard and replaced again. 

Once the user stops adding items to the work surface, the system uses the position of the chopping board to determine which recipe the user would like to use. This can be changed by sliding the board along the bench in front of the preferred recipe. When the recipe is chosen, more information about it is displayed. This extra information includes the preparation instructions and its nutritional value.

Situated media

With widespread use of RFID technology rapidly approaching, household items such as food products could be used to trigger media relating to virtually anything. In the case of the Ambient Kitchen, we have attached media to a number of objects, though currently we simply use adverts for food items and trailers for DVDs.

Media can be displayed on a single section of the display, or utilise the entire display area available. In single display mode the media migrates with the object as the user moves it along the work surface.

Sensor display

Software has also been provided to test the output of each of the devices embedded in the kitchen. This includes the view from each of the cameras, the RFID readers output, and the sensor network values. This application is not assistive (other than for our software developers), but offers people the opportunity to interact with the devices, and understand the complexity of the underlying system.

As with the recipe recommender, the sensor display uses a control item to set the context (for convenience we use the spaghetti jar!). Each of the screens above displays output from each of three device types: (1) cameras around the kitchen; (2) the sensor network (acceleration and light values); and (3) the RFID antenna readings.

Ambient Kitchen - Team

• Pervasive computing (Newcastle University): Patrick Olivier; Wayne Smith (Lead Developer); Robin Firth; Paul Dunphy; Cassim Ladha; Steven Lindsay (with help from Rachael Colley, Dan Jackson, Christian Kray, Mark McLeish, John Shearer, Jayne Wallace)
• Design and construction: Doug Lamond
• Psychology (University of York): Andrew Monk; Joe Wherton
• Clinical (Newcastle University): Louise Robinson
• Contact: Patrick Olivier

The Ambient Kitchen was funded by CELS ATL (Assistive Technology Laboratory) which is exploring how technology can meet the challenges raised by a rapidly ageing population. The project is committed to developing commercially viable products that bring real benefit to users. Through collaborative projects, CELS ATL brings together industrial and academic expertise across a wide range of disciplines, essential when designing products for consumers with specific needs. Key to this is the involvement of consumers themselves, ensuring that the products developed are truly "fit for purpose". In addition to university and industrial partners, CELS ATL is working with major charities and public sector bodies in the UK to bring real improvements to the lives of elderly and chronically ill people. CELS ATL is jointly funded by the Regional Development Agency; One NorthEast and European Regional Development Fund (ERDF).         

Ambient Kitchen - Technology

The goal of Ambient Kitchen is to build an environment that can provide calm support for older people in a kitchen using technology that has effectively disappeared. This has been achieved by embedding the RFID readers within the work surfaces and the cupboards; concealing the miniature DLP projectors inside the cabinets; placing pressure sensors under the flooring; and attaching wireless light and acceleration sensors to anything that can be moved. The Ambient Kitchen's software is able to monitor not only what objects are manipulated, but the actions people are trying to perform (although this is quite basic at present). Using this information gentle hints and advice is displayed to help people in their everyday kitchen activities.

Radio Frequency Identification

Radio Frequency Identification (RFID) is a technology that consists of a reader, a number of antennas (multiplexed to the reader), and numerous tags that we embed in objects and food packaging. The reader communicates with a tag by sending a radio frequency signal to it. We use passive tags (i.e. not powered) and so the tag must convert that signal into power in order to send its reply back to the antenna. 8 antennas are embedded in the kitchen: 6 in work surfaces and 2 in cupboards. The antennas are connected to the reader via an 8-port multiplexor, which cycles through each antenna, reading the data from each tag in an antenna's field. The data identifies the object that the tag is attached to. All this data is then communicated to a central computer.

Wireless sensor network

Our wireless sensor network allows us to monitor acceleration and light values. Each sensor is integrated with a mote, small wireless devices that can pass information between themselves back to a base station and then onto the central computer. Using these mote-based sensors, the ways in which people manipulate kitchen objects (e.g. the kettle, food containers, and cupboard doors) can be tracked which makes reasoning about what people are doing a lot easier than simply using camera systems.

Pressure sensors

Our custom built pressure sensors contain a circuit board encased in memory foam which is in turn sandwiched between two sheets of metallic foil. The more pressure that is applied, the closer together the two foil sides move, resulting in a change in capacitance. The full floor consists of an array of these sensors, arranged in lines that transmit to mid-level controllers. These controllers in turn transmit readings to a master controller, which communicates the profile of pressure back to the central computer.

DLP projectors

Digital Light Processor (DLP) Projectors rely on individually controlled micro mirrors, 1 per pixel of the display, laid out in a matrix on a Digital MicroMirror Device (DMD). Light is split through a prism, and then each of the primary colors passed to an individual DMD. The use of DLP technology allows for much smaller projectors than usually available. The Ambient Kitchen currently uses 4 projectors each of which has dimensions 5"x4"x2".

Ambient Kitchen - Data set

http://di.ncl.ac.uk/publicweb/AmbientKitchen/KitchenData/CLAK_WiiUtensils/cuong_1.4/