Project

Back to overview

The interaction of perception and language in memory: How do labels shape visual working memory?

English title The interaction of perception and language in memory: How do labels shape visual working memory?
Applicant da Silva Souza de Carvalho Alessandra
Number 169302
Funding scheme Project funding
Research institution Psychologisches Institut Universität Zürich
Institution of higher education University of Zurich - ZH
Main discipline Psychology
Start/End 01.02.2017 - 30.09.2020
Approved amount 351'756.00
Show all

Keywords (4)

long-term memory; language; visual working memory; labels

Lay Summary (German)

Lead
Als Menschen sind wir täglich visuellen Reizen ausgesetzt. Gleichzeitig werden wir mit verbalen Beschreibungen dieser Reize konfrontiert. Der Einfluss solcher Beschreibungen bei der Verarbeitung von visuellen Reizen im Arbeitsgedächtnis ist bisher noch unerforscht. So können sie hilfreich, unbedeutend oder auch schädlich sein. Ziel dieses Projekts ist, den Einfluss sprachlicher Inhalte auf die Speicherung, die Verarbeitung und den Abruf von visuellen Inhalten im Arbeitsgedächtnis zu untersuchen.
Lay summary

Inhalt und Ziel des Forschungsprojekts

Das Arbeitsgedächtnis ist wichtig für die kurzzeitige Aufrechterhaltung von Informationen und hat eine begrenzte Kapazität. Unsere Fähigkeit komplexe Aufgaben zu bewältigen ist somit beschränkt. Bis anhin wurde die Rolle der Sprache innerhalb des visuellen Arbeitsgedächtnisses vernachlässigt. Die folgende Projekt soll diese Lücke anhand empirischer Untersuchungen erstmals schliessen.

Ziel dieses Projekts ist, den Einflusses verbaler Beschreibungen auf das visuelle Arbeitsgedächtnis anhand von fünf verschiedenen Hypothesen zu untersuchen. Diese Beschreibungen können (1) konzeptuelles Wissen im Langzeitgedächtnis aktivieren, (2) eine verbale Spur auf Kosten der visuellen Spur im Arbeitsgedächtnis abspeichern, (3) zusätzlich zur visuellen Spur im Arbeitsgedächtnis eine verbale kreieren, (4) einen Abrufreiz zur Beschreibung hinzufügen und (5) die Abstraktion und das Lernen von Chunks fördern. Die Hypothesen sollen in verschiedenen Verhaltensexperimenten untersucht werden.

Wissenschaftlicher und gesellschaftlicher Kontext des Forschungsprojekts

Die Schlussfolgerungen dieser Studie sollen zu verbesserten theoretischen und praktischen Kenntnissen über die Kapazität des visuellen Gedächtnisses im Zusammenhang mit der Sprache führen. Darüber hinaus leiten sich aus diesen Kenntnissen erfolgreiche Interventionen ab, die zu einer gezielteren Anwendung der beschränkten Kapazität des visuellen Arbeitsgedächtnisses führen.

Direct link to Lay Summary Last update: 11.01.2017

Responsible applicant and co-applicants

Employees

Publications

Publication
The interplay of language and visual perception in working memory
Souza Alessandra S., Skóra Zuzanna (2017), The interplay of language and visual perception in working memory, in Cognition, 166, 277-297.
No Evidence That Self-Rated Negative Emotion Boosts Visual Working Memory Precision
Souza Alessandra S., No Evidence That Self-Rated Negative Emotion Boosts Visual Working Memory Precision, in Journal of Experimental Psychology: Human Perception and Psychophysics.

Datasets

No evidence that self-rated negative emotion boosts visual working memory precision

Author Souza, Alessandra S.
Publication date 23.03.2020
Persistent Identifier (PID) 10.17605/OSF.IO/WZVMX
Repository Open Science Framework
Abstract
This repository contains materials, data and analysis scripts related to the paper of the same title in-press in Journal of Experimental Psychology: Human Perception and Psychophysics

The interplay of language and visual perception in working memory

Author Souza, Alessandra S.
Publication date 10.05.2017
Persistent Identifier (PID) 10.17605/OSF.IO/TF93Q
Repository Open Science Framework
Abstract
This page has all the data, materials, and analysis scripts published in the paper of same title in the Journal Cognition.

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
60th Annual Meeting of the Psychonomic Society Poster Evidence Against All-Or-None Short-Term Consolidation: Consolidation Speeds up with Prioritization and Is Interrupted by a Distractor Task 14.11.2019 Montreal, Canada da Silva Souza de Carvalho Alessandra; Overkott Clara;
60th Annual Meeting of the Psychonomic Society Talk given at a conference Improving visual working memory with verbal labeling. 14.11.2019 Montreal, Canada da Silva Souza de Carvalho Alessandra; Matyja Marta; Overkott Clara;
9. European Working Memory Symposium (EWOMS) Poster From color terms to color memory: Does labeling in working memory boost episodic long-term memory? 29.08.2019 Pavia, Italy Overkott Clara; da Silva Souza de Carvalho Alessandra;
59th Annual Meeting of the Psychonomics Society Poster Is Consolidation in Visual Working Memory an All-or-None Process? 15.11.2018 Louisiana, United States of America da Silva Souza de Carvalho Alessandra; Overkott Clara;
9. European Working Memory Workshop (EWOMS) Poster Is consolidation an all-or-none process? Examining the interaction between encoding and recall order in visual working memory. 29.08.2018 Pavia, Italy da Silva Souza de Carvalho Alessandra; Overkott Clara;
58th Annual Meeting of the Psychonomics Society Poster Filtering of irrelevant information in visual working memory: The effect of labeling object features. 09.11.2017 Vancouver, Canada da Silva Souza de Carvalho Alessandra; Overkott Clara;


Associated projects

Number Title Start Funding scheme
187671 How do Verbal Labels Influence Children’s Visual Working Memory? 01.09.2019 Doc.Mobility

Abstract

In our everyday interactions, we are exposed - among other things - to streams of visual inputs and verbal descriptions of these inputs. Whether these descriptions are self-generated or provided by others, language is involved in most of our activities. What is the impact of describing our visual experiences as they unfold? Are these descriptions helpful, inconsequential, or harmful? The present proposal is concern with how verbal labels modulate the storage, maintenance, and retrieval of visual information in working memory (WM). WM is the cognitive device enabling the maintenance of information in an easy-to-retrieve state for ongoing processing. WM capacity is severely limited, thereby constraining our ability to carry out complex cognitive tasks. Despite the growing number of investigations on how language affects visual perception, learning, and episodic memory, the role of language has been largely neglected in the field of visual WM. The present proposal will fill this gap by providing a first systematic empirical and theoretical examination of the effect of labels on visual WM. Based on the extant literature on language and cognition I have delineated 5 hypotheses about the labeling effect for visual WM: (1) labeling activates conceptual (non-verbal) knowledge in long-term memory (LTM); (2) labeling results in the storage of a verbal trace in WM at the expense of the visual trace; (3) labeling generates a verbal trace in addition to the visual trace; (4) labeling adds a retrieval cue to the labeled item; and (5) labeling fosters abstraction and chunk learning. The experiments planned in the present proposal aim at testing predictions derived from these hypotheses. The project uses established continuous-reproduction visual WM tasks. These tasks will be combined with controlled verbalization protocols to assess the impact of specific labeling strategies. In combination, these methods will allow the examination of the format of the representations in WM when participants combine visual and verbal inputs, and whether doing so is beneficial, inconsequential, or harmful to performance. This knowledge will deepen our understanding of the mental codes available to guide our thoughts and actions over the short-term, and will provide further theoretical and practical insights regarding the conditions that hinder or foster the best use of the capacity of visual WM.
-