To achieve the objectives of PIAAC, the international PIAAC Consortium has developed a special survey design and corresponding survey instruments for PIAAC Cycle 2. The frameworks for the assessment domains have been revised, extended, or newly developed by the international expert groups. When developing the instruments, care was taken to maintain an adequate number of questions and items from Cycle 1 to allow for measurement of trend as well as to cover new constructs (e.g., with a view to the increasingly ubiquitous digitalization).
The PIAAC interview comprises two main modules, which are delivered on tablet computers in Cycle 2: first, an interviewer-administered background questionnaire, which collects important sociodemographic and other background data on the respondent; second, a self-administered direct assessment of cognitive skills, in which the respondents carries out everyday tasks on their own. In PIAAC Cycle 2, the cognitive assessment again includes literacy and numeracy. To allow for a differentiated measurement at low proficiency levels, reading components and numeracy components are assessed. An important innovation in PIAAC Cycle 2 is that the domain of adaptive problem solving is assessed for the first time.
The background questionnaire collects demographic and other key information about the respondent. It includes, for example, questions about age, educational background (e.g., highest educational qualification, non-formal learning and training activities), work history, household and family background (e.g., household size, language most frequently spoken at home). The respondents are also asked to report how often they use various skills in their everyday lives. In addition, employed persons are asked how often they use various skills at work. There are also questions about respondents' personality traits and attitudes.
This information can be used, for example, to explore important factors that influence the development and maintenance of skills in adulthood.
The key elements of the PIAAC background questionnaire are:
Literacy is the ability to access, understand, evaluate, use, and reflect on written texts to accomplish one’s objectives, develop one’s knowledge and potential, and participate in society. Literacy is an important prerequisite for the acquisition of further skills. The concept of literacy in PIAAC Cycle 2 also takes into account that reading behavior has changed as a result of the increasing digitalization of modern society. Therefore, there is an increased focus on digital reading of texts that are distributed across different locations and linked with each other. Thus, navigation, strategic scanning, and increasingly the evaluation of the trustworthiness of text sources are required. Literacy items consist of everyday tasks in a wide variety of contexts. For example, respondents might have to search for specific health-related information on different websites, search customer reviews in electronic media, or answer questions about an article.
The aim of the basic components of reading competence is to obtain a more accurate picture of adults with lower reading skills. The tasks measure basic skills required for reading and comprehension of written texts, such as comprehension of individual simple sentences and paragraphs.
Numeracy is the ability to access, critically reflect on, and use different kinds of mathematical information to navigate the demands of different everyday life situations. At a time when people are frequently confronted with numerical and mathematical information at work and in their everyday lives, numeracy is of particular importance. Possible assessment tasks may include, for example, calculating material requirements in a crafts context or interpreting numerical information in graphs and tables. The tasks include different areas of everyday life and cover a broad range of demands.
The basic components of numeracy competence aim to obtain a more accurate picture of adults with rather low levels of everyday mathematical competence. The tasks with a focus on numerical comprehension have been newly developed for cycle 2.
Adaptive problem solving denotes the ability to achieve one’s own goals in a dynamic situation in which a method for reaching a solution is not directly available. Cognitive and metacognitive processes are necessary to define the problem in diverse environments and contexts, to search for information, and to implement the solution. This domain and the corresponding instrument were newly developed for PIAAC Cycle 2 and constitute an important innovation. The assessment tasks are embedded in everyday scenarios within an interactive environment. Possible tasks in this domain may include organizing trips or celebrations in coordination with other (virtual) persons, or planning, controlling, and monitoring systems (e.g., production machines or plants).
The survey instruments were tested in a field trial in 2021. Based on the results of this field trial, the final set of questions for the background questionnaire and items for the cognitive assessment were selected for the main survey. The selection took into account that, in contrast to Cycle 1, the survey is not delivered on laptops but rather on tablet computers on which touch functionalities have been implemented. Using the data from the field trial, the international PIAAC Consortium examined, for example, whether mode effects occurred in comparison with Cycle 1. For the field trial, the survey instruments were translated by all participating countries in compliance with rigorous translation standards. The field trial data allowed for the examination of the equivalence of the translations and the possibility of implementing corrections for the main study.
