The expansion of digital technologies in education has intensified debates on infrastructure, connectivity, and pedagogical innovation in Brazilian basic education. Recent initiatives, such as the Política Nacional de Educação Digital (PNED) and the Estratégia Nacional de Escolas Conectadas (ENEC), aim to coordinate investments in connectivity, devices, and digital resources while promoting curricular integration and teacher development (Brasil, 2023a, 2023b). These policies intersect with the Base Nacional Comum Curricular (BNCC) and its Computing Education complement, which emphasize digital competencies for student learning (Brasil, 2018; Brasil, Ministério da Educação, Secretaria de Educação Básica, 2022).
To support implementation, a national technical advisory process helped state departments elaborate Digital Education and Pedagogical Innovation Plans. Twenty-three systems participated, producing drafts with objectives, goals, and action plans related to connectivity, curriculum, teacher development, and school-level innovation. These plans aim to guide investments, coordination, and capacity-building across state networks.
Despite growing attention to digital education policy, little empirical research has examined how M&E is embedded in subnational planning. Existing studies focus on policy design or outcomes, paying less attention to evaluability, that is, how clearly plans articulate results chains, indicators, targets, and responsibilities (Markiewicz, 2014; Nazare et al., 2024; World Bank, 2013). This study addresses that gap by investigating how Brazilian state systems structure M&E in their Digital Education and Pedagogical Innovation Plans. The guiding question is: How do Brazilian state education systems structure monitoring and evaluation arrangements with respect to objectives, targets, actions, and indicators?
Monitoring refers to the continuous tracking of progress and implementation fidelity, providing managers with an understanding of whether the project is on track, whereas evaluation provides deeper judgments about results and causal mechanisms (OECD, 2013; UNESCO, 2024; World Bank, 2013). Robust M&E in educational planning typically involves explicit results chains, SMART targets, defined indicators, timelines, and responsibilities. When these elements are absent or weak, plans tend to function as formal statements rather than management tools. Analyzing these elements offers insight into both evaluability and the institutionalization of M&E practices.
This study adopts a qualitative content analysis of Digital Education and Pedagogical Innovation Plans produced by Brazilian state education departments (Bowen, 2009; Cellard, 2008; Ravitch & Carl, 2021; Saldaña, 2013). Following recommendations to assess the credibility and completeness of documents before analysis (Bowen, 2009; Cellard, 2008), 13 of the 23 plans initially considered met the inclusion criteria: consolidated versions with all required sections completed and no tracked revisions. These documents represent all five Brazilian regions. The analysis focused on sections where M&E logic is operationalized: general objective (3.1), specific objectives (3.2), goals (5), and action plan and schedule (6). Diagnostic and contextual sections were excluded.
An analytical checklist was developed drawing from M&E and results-based management literature (Instituto Jones dos Santos Neves, 2018; Markiewicz, 2014; Nazare et al., 2024; World Bank, 2013), comprising six criteria: (1) linkage between objectives, outcomes, targets and actions, (2) coherence between output and success indicators, (3) presence of goal-level targets, (4) targets for action-level indicators, (5) clarity of timelines and responsibilities, and (6) feasibility of action volume for monitoring. These criteria guided deductive coding (Saldaña, 2013).
Plans were transcribed into an analytical matrix and scored from 1 (incipient) to 3 (full) based on textual evidence. All codes were reviewed for consistency with operational definitions. Composite scores supported cross-case comparison. The analysis prioritized manifest content and avoided inference beyond what was documented (Bowen, 2009).
The analysis indicated significant variation in how the 13 plans articulate M&E. While all included objectives, goals, and an action plan, their internal structure and evaluability varied. Strategic goals were often clearer than operational elements, consistent with studies showing stronger goal definition than measurable results and implementation logic (Markiewicz, 2014; Nazare et al., 2024; World Bank, 2013).
A key distinction lay in target quality. Higher-scoring plans aligned with the SMART model (Doran, 1981), featuring quantifiable targets, clear time horizons, and links to strategic objectives. In contrast, intermediate- and lower-scoring plans often lacked numeric thresholds or relied on vague temporal markers, undermining monitoring. The presence or absence of SMART attributes was a major differentiator.
Plans also differed in how they articulated the results chain. One higher-maturity plan translated the curricular update goal into a sequence of actions with resources, timelines, responsible units, and deliverable criteria. It defined the committee as a product and set “100% of the committee instituted by May 2025” as the success indicator, followed by actions such as preparing a guideline and creating a virtual learning environment. This structure made the pathway from objectives to actions visible and evaluable. A lower-maturity plan, by contrast, expressed the same goal broadly and listed heterogeneous initiatives—acquiring books, offering hybrid support, forming partnerships—mostly without measurable indicators. The logic of change remained opaque.
Indicator quality reinforced these contrasts. Mature plans included thresholds like “80% of teachers with active access to the training platform” or “100% of students accessing the virtual learning environment,” enabling precise tracking of outputs and early outcomes. Intermediate- and lower-maturity plans listed indicators without targets or used vague terms like “improved practices,” which are insufficient for judging progress.
Timelines and responsibility further distinguished plans. Higher maturity plans named specific offices or individuals for each action, with defined start and end dates. Intermediate- and lower-maturity plans used generic timelines and vague references, weakening accountability and reducing their utility for implementation.
Monitoring feasibility was another dimension. Mature plans linked each goal to a small set of structured actions that addressed objectives and could be realistically tracked—some intermediate plans listed granular tasks. Though detailed, the volume and overlap of actions hindered systematic monitoring without substantial technical and managerial capacity.
Based on this analysis, plans were grouped into three maturity clusters. High-maturity plans articulated a clear results chain, SMART targets, defined responsibilities and timelines, and a distinction between product and success indicators. Intermediate-maturity plans partially operationalized these elements. Low-maturity plans had generic objectives, limited measurability, missing success indicators, and fragmented implementation structures. Here, maturity is understood analytically—drawing on M&E and results-based management literature—as the extent to which a plan makes its logic explicit, coherent, and usable for implementation, learning, and accountability (Instituto Jones dos Santos Neves, 2018; Markiewicz, 2014; Nazare et al., 2024; World Bank, 2013).
Overall, differences in plan quality reflect not only context but also the degree to which M&E routines are internalized within state departments. Some systems use the plan as a managerial tool, while others treat it as a formal requirement. Regional and socioeconomic factors offer background but do not fully explain the variation. Some lower-income states produced structured, evaluable plans, while some higher-income systems showed gaps linked to institutional fragmentation. Leadership continuity, internal coordination, and established planning routines were more decisive than regional characteristics in shaping the evaluability and maturity of Digital Education and Pedagogical Innovation Plans.
The analysis reveals substantial variation in the quality of monitoring and evaluation components across the Digital Education and Pedagogical Innovation Plans. This suggests that future policy and capacity-building efforts should consider the degree of structure in planning and M&E routines and adapt the approach to each context. When objectives, targets, indicators, and responsibilities are clearly defined, it's easier to move toward more sophisticated M&E arrangements. In systems where these elements are still incipient, the priority is to ensure a minimum level of internal coherence that makes the plan trackable and evaluable.
Academically, the results open three avenues for further research. First, expanding the sample to all 23 state plans would offer a broader national view and enable comparisons across system profiles. Second, studies could examine how technical advisory processes, such as the one described in the introduction, support planning and M&E capacity development, using documentary data alongside meeting transcripts, feedback records, or new data collection with technical teams. Third, the analytical checklist used here, still exploratory, could be refined as both a research tool and a formative instrument for state departments. Future studies may adjust their criteria and judgment levels, test reliability across evaluators, explore their use for self-assessment and iterative revision, and investigate whether their systematic application correlates with improvements in M&E quality and digital education policy implementation.
This study is part of the project “Digital Education: Teacher Development and Curriculum Innovation”, conducted by the EduTec research lab at the Federal University of Mato Grosso do Sul (UFMS), in partnership with the Secretariat for Basic Education of the Brazilian Ministry of Education (MEC). It received ethics approval from the UFMS Ethics Committee (ref. 7.317.243). Data collection was collaborative; analysis and writing are the authors’ independent responsibility. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001.
EdTech Archives