Reading Illustrated Texts: How Pictures Influence Activation, Integration, and Validation Processes

Abstract

Reading and comprehending texts illustrated with pictures is an essential part of our daily lives. Newspaper articles, internet blogs, and textbooks are just some examples of written works that use illustrated texts to communicate information. In order to facilitate and improve comprehension of illustrated texts, it is important to understand the underlying cognitive processes (Mayer, 2021b). This dissertation project aimed to specify the ongoing cognitive processes when processing illustrated texts. To this end, the research presented in this project focused on the Resonance-Integration-Validation model of text comprehension (RI-Val, O’Brien & Cook, 2016a, 2016b) in combination with the Cognitive Theory of Multimedia Learning (CTLM; Mayer 2021a, 2021b). The RI-Val model assumes that readers passively activate information stored in long-term memory that overlaps with the current content that is being read, integrate the activated and currently read content, and validate the newly formed linkages, that is, check them for consistency. The CTML does not yet consider these passive processes. The overarching research questions investigated in this dissertation were whether activation, integration, and validation processes apply to illustrated texts, and, if so, whether pictures might influence these processes. To answer these questions, three research packages (A-C), each consisting of three experiments, were conducted (total N = 2,496). All nine experiments used adapted versions of what is known as the contradiction paradigm (Albrecht & O’Brien, 1993). In this paradigm, reading times for a target and the following sentence (i.e., spillover sentence) are measured. It was varied whether the information presented beforehand was consistent or inconsistent with the target sentence and whether the previously mentioned consistent or inconsistent information was illustrated with a picture or not. Overall, the results show that activation, integration, and validation processes apply to illustrated text and that these processes occur across modalities (i.e., text and picture). Furthermore, a summary with meta-analytic measures showed that pictures when presented beforehand slightly accelerated the reading times of the target sentences. These findings broaden the scope of the RI-Val model and further specify the cognitive processes identified within the CTML, by showing that passive activation, integration, and validation processes apply to illustrated texts and that they occur across modalities (i.e., for information from both texts and pictures).

Dissertation ist gesperrt bis 25. März 2027 !