Our environment is composed according to certain rules and characteristics which are so obvious to us that we are scarcely aware of them. Professor Melissa Lê-Hoa Võ, psychologist at Goethe University Frankfurt, is studying this “scene knowledge”, amongst other topics of visual cognition in a virtual reality laboratory. In the current issue of the “Forschung Frankfurt” research journal, journalist Jessica Klapp tells readers about her virtual trip to Italy and explains why we don´t look for the milk under the bed or for our pillow in the bathtub.
“When we search for a specific object in a scene, we seem to have developed a precise idea of where to look for and find certain things,” explains Melissa Võ. What she finds particularly interesting in her investigation of these naturalistic scenes is how we perceive our environment. In which circumstances are we especially attentive? And what will we remember later? To find this out, Professor Võ uses eye tracking and virtual reality scenarios in her laboratory, alongside measurement of brain potentials.
“We use eye tracking to measure which parts of a picture the observer finds interesting or important, how fast his gaze settles on specific objects in a scene and how long it dwells there,” explains Dr. Dejan Draschkow, a member of Professor Võ’s research group. Eye tracking is very important due to the close relationship between eye movement and cognitive processes. The video-based systems used by the researchers register eye movements with the help of a camera. Both head-mounted systems that resemble a pair of spectacles as well as remote eye trackers installed in a computer monitor together with a camera and infrared LEDs are used. With the mobile system, the test participants can move around in the room, search for objects and interact with them.
With the virtual reality headset, the computer simulates a virtual 3D world in which the test person moves about. With simulated settings, such as an Italian piazza in the centre of which brown boxes are unexpectedly floating around, the researchers check whether the results identified on two-dimensional monitors are also valid in a realistic, three-dimensional environment. They want to understand which rules people use to compose their environment and interact with the objects in it.
Investigating scene knowledge in children is one of the topics that the research group is tackling in particular depth. The aim of the “SCESAM” project, which was launched with the support of IDeA, an interdisciplinary research centre, is the early detection and treatment of possible cognitive disorders such as dyslexia. Tests are conducted in a mobile laboratory right outside the nursery school: The researchers show children “ungrammatical” pictures, for example a shoe on the cooker instead of a saucepan, and watch their reactions with the help of an eye-tracking camera. If one of the children in a large group behaves differently, they are interested to see whether there is a correlation between language development and attention behaviour.
Other fields, such as medicine, also benefit from the results. For example, the researchers measured the eye movements of radiologists when studying x-rays and analysed which strategies they use to recognize tumours and how successful these strategies are. The research results are also important for the security control of hand baggage at airports. How do staff decide which baggage needs a more thorough check? Why wasn’t a dangerous object found? Didn’t the baggage inspector look at this area? Or did he look but didn’t consider the object important?
And finally, people with dementia could also profit from scene knowledge research, since Võ and her team have discovered that the ability to recall pictures in a scene increases if the test persons have previously looked for and found individual objects. In a surprising memory test, they performed far better than participants who were supposed to memorize objects explicitly. “What this means for us is that in the case of a visual search there is more interaction with the scene and the participants commit objects better to memory,” explains Professor Võ.
Journalists can order the current issue of “Forschung Frankfurt” free of charge from Helga Ott, ott@pvw.uni-frankfurt.de.
Online: www.forschung-frankfurt.uni-frankfurt.de.
“Forschung Frankfurt” subscriptions: http://tinygu.de/ff-abonnieren
Source: Press Release 08/12/2017
