Measuring quality of preprimary education — Raikes et al.

Measurement of quality in early childhood education (ECE) helps shape policy and practice. This study reports on the adaptation of the Measure of Early Learning Environments scale (MELE-A) in one sub-Saharan African country. The sample included 250 public and private preprimary schools and 979 children from all regions of the country. Categorical confirmatory factor analysis supported the three factors hypothesized: health/safety, materials/activities, and teacher/child interaction. Association between these factors and child development and learning and teacher characteristics where following: only materials/activities demonstrated significant associations with children’s learning, while teacher education was associated with all three factors. Adapting measures for the use in low- and middle-income countries is a multi-faceted process and it is important to document psychometric properties of these adapted measures to improve accuracy of ECE measurement to inform policy and practice.

Author: A. Raikes, N. Koziol, D. Davis & A. Burton

Source: Raikes, A., Koziol, N., Davis, D., & Burton, A. (2020). Measuring quality of preprimary education in sub-Saharan Africa: Evaluation of the Measuring Early Learning Environments scale. Early Childhood Research Quarterly, 53, 571-585. https://doi.org/10.1016/j.ecresq.2020.06.001.

• In the United Nations’ Sustainable Development Goals (SDGs), Target 4.2 states that by 2030, “all children have access to quality early childhood care, development and preprimary education so that they are ready for primary education.”
• Accordingly, access to early childhood education (ECE) is expanding globally, however, the quality of the ECEC remains a concern.
• Several aspects of young children’s environments contribute to quality in ECE, including the physical environment, teacher/child interaction, the content of the curricula, the teachers’ skill in scaffolding children’s development, and the ability of the teacher to individualize instruction.
• However, in many low- and middle-income countries data on quality in ECE is not consistently available.
• Measurement forms the basis of monitoring systems to inform governments and stakeholders on the status of their investments in preprimary education, to inform parents, and to support teacher knowledge and education.
• In response to measurement needs as part of Target 4.2 and to support national implementation of ECE, the Measuring Early Learning Quality & Outcomes (MELQO) Initiative was begun by four organizations, UNICEF, UNESCO, World Bank and UNICEF.
• MELQO’s Consortium created two sets of open-source tools focused on preprimary education addressing child development and learning, and quality in preprimary settings.
• These tools were designed to facilitate measurement in low- and middle-income countries by integrating commonly-articulated global concepts of ECE quality and child development with local adaptation processes, with emphasis on creating feasible, cost-effective measurement.
• The present study reports on the development and testing of a recent open-source ECE observational quality measure, the Measure of Early Learning Environments (MELE), as it was adapted and tested in one country in sub-Saharan Africa.

ECE quality in low- and middle-income countries

• Positive impacts of ECE on child development and learning have been reported across countries and also in some of the poorest areas of the world including in East Africa.
• ECE has been shown to have greater positive impacts as quality increases.
• However, there is little research directly documenting quality of ECE in many parts of the world, especially in sub-Saharan Africa, an important focal regional area for early childhood development where ECE could make a substantial contribution.
• There is considerable variation in the types, settings, and quality of ECE, e.g. with 87% of classroom time spent with young children sitting in rows of desks facing forward in a sample of preschools in Nairobi, while only 18% of time was spent this way in Johannesburg.
• Variations in ECE quality may also reflect investments: resources for ECE are limited in sub-Saharan Africa, with just 2% of education budgets devoted to ECE.

Evaluating psychometric properties of ECE quality measures

• Evidence based on test content, or the extent to which test content adequately samples the underlying construct, e.g. test content is the ideas, beliefs and behaviors that comprise definitions of quality in ECE
• Evidence based on internal structure, or the extent to which the data support the hypothesized factor structure
• Evidence of internal consistency, or the extent to which responses among items are highly correlated, e.g. Cronbach’s alpha
• Evidence based on relations to other variables, or the extent to which the MELE scores are associated with child outcomes and teacher and school characteristics

The study

• This study describes the adaptation and resulting psychometric properties of MELE scores in one sub-Saharan African country (denoted as MELE-A).

Hypothesis:

• The MELE-C constructs and items would be generally aligned with the content of national standards, but that adaptations would also be required to improve contextual fit to create a revised version, MELE-A.
• The MELE-A items would factor into three correlated but distinct constructs: health/safety, materials/activities, and teacher–child interactions.
• The MELE-A scores would demonstrate adequate internal consistency (reliability) evidence.
• The MELE-A scores would be associated with teacher and school characteristics, and positively predict child development.

Measuring ECE quality in low-resource settings

• Culturally and contextually sensitive measurement is important for long-term usefulness and applicability of results.
• Adaptations of quality measures have included changes at the domain, construct and item levels.
• The “core” MELE (MELE-C) served as the starting point for country adaptation. It shares many elements with other measures, including a focus on teacher/child interactions; materials; and physical environments such as health and safety.

The measures

Classroom observation tool

• The adapted version of Measure of Early Learning Environments (MELE-A) was used to observe classroom quality.
• MELE-A consists of several items including teacher-child interactions (e.g., teacher or child engagement), health and safety (e.g., drinking water or toilet facilities) and materials/activities (e.g., writing utensils or free play).

Child development and learning and social/emotional development

• Parents (n=906 children) and teachers (n=473 children) were interviewed on social/emotional development of the child with a 10-item questionnaire, e.g. How often does the child follow instructions?, with response options 0=never, 1=sometimes, 2=often/always.
• Children’s development and learning was also assessed by a trained assessor with a battery of tasks relating to spatial vocabulary, verbal counting, producing a set, number identification, letter identification, expressive language, listening comprehension, name writing, head toes knees shoulders, and pencil tap.

Participants

• The sub-Saharan African country has over 2 million people of which about half are children under 17 years-old. About 70 percent of population lives in rural areas and most live below the international poverty line. About 33 percent of children are enrolled in preprimary education.
• A sample of 250 schools, of which 85 % were private, were recruited to participate for the study.
• Children were nested in classrooms (one classroom per school), with one main teacher serving as the informant on children’s social/emotional development.
• Approximately four children in each classroom were randomly selected to the study, thus the total sample of children were 979 (46% male, mean age 5.49 years).
• The 222 teachers were on average just under 40 years-old, mostly females, with variable levels of education (e.g. only one-fourth had some college or higher and about one-fourth only primary education).

Conclusions

• Measurement of ECE quality is an important component of building an equitable early childhood system,
• Using the MELE-A, three hypothesized constructs of ECE quality –health/safety; materials/activities; and teacher/child interactions– were supported by confirmatory analyses.
• The lessons from the adaptation process include the value of including a wide range of stakeholders; the importance of clarifying what specific behaviors describe “quality” in that context; and the value of revising, testing, and revising again.
• Teachers’ education levels were associated with all constructs of quality, and health/safety and materials/activities scores were higher on average for teachers who were less stressed.
• After accounting for a wide range of child, family and school characteristics, children who were in better organized classrooms had significantly higher scores on average on measures of child development using direct assessment tool, but not on parent or teacher reported social/emotional development.
• Our study was not able to fully investigate whether MELE-A meaningfully differentiated between high- and low-quality ECE and on which dimensions the scale was most sensitive.
• The data from MELE-A might have also failed to detect associations between quality and child development because the scale was not sensitive enough to identify thresholds of quality.
• The MELE represents a novel approach to quality measurement, by taking a “core” of items to adapt to better match local context.
• Adaption of ECE quality measures is commonplace and likely inevitable to ensure both feasibility and appropriate test content, and more work should systematically document adaptation processes and resulting psychometric properties of measures.