Professional development for decision makers – our policy learning journey so far
EVIDALI Development Diary #3
EVIDALI is not the first EU project that focuses on fostering educational data literacy. As a matter of fact, this project follows the steps and lessons learned from Agile EDU, an Erasmus+ funded project that explored the key factors in the meaningful and responsible use of education data for decision making. Based on evidence from different education ecosystems, Agile EDU showed that there is a need to contextualise policy recommendations versus a one-size-fits-all approach, because countries and regions vary greatly in terms of educational governance, culture, history and aims.
Therefore, the EVIDALI project adopted a co-construction and peer learning approach to building policy recommendations, through its policy learning lab (PLL) series. Having recently visited Trento for the 3rd policy learning lab of the series, we will take the opportunity to talk about this approach in this entry of the diary.
Figure 1 We were hosted by the Autonomous Province of Trento at ‘Casa Moggioli’ a renaissance style building which houses the Trentino headquarters of the European Grouping of Territorial Cooperation (EGTC).
Educational authorities build a shared understanding of data literacy
Throughout the project, EVIDALI partners that are also representatives from Malta, the province of Trentino (Italy), and the regions of Castilla y Leon (Spain) and Madeira (Portugal) participate in a series of nine Policy Learning Lab workshops.
The Policy Learning Labs (PLLs) are capacity building and knowledge sharing/peer learning activities. One of the main outputs of the series will be for each territorial entity to write a mock policy scenario addressing a territorial priority for development of data literacy for teaching (DLFT).
Now that the first three policy learn labs of the nine-lab series have been concluded, participants have built a common understanding of data literacy for teaching (DLFT), and defined the challenges and opportunities for promoting it in their own schools.
The 3rd PLL took place onsite in the beautiful Province of Trentino. A lot of ground was covered in just two and a half days:
- Participants learned about the data-informed decision making practices adopted by various educational actors in the province;
- They met educators from five provincial schools in Trento and Rovereto;
- IRVAPP presented the preliminary results from the DALIS survey on teachers’ and school leaders’ data literacy and their beliefs and perceived challenges towards it. IRVAPP then engaged participants in an exercise of data interpretation;
- Reflected on the barriers and enablers in their own contexts.
Figure 2 During the 3rd PLL in Trento, a visit to the Castle Buonconsiglio offered a beautiful view of the Alpine peaks of the region.
Reflection and co-construction
The PLL series can be thought as a professional development programme. And like a professional development programme, they have a final assignment: to write a mock policy scenario. The four scenarios will be the basis for the project's final policy recommendations.
The first three labs helped participants set the ground and build a shared understanding of data literacy, while also identifying ‘wicked problems[1]’ in their own context.
While working on the first three lab editions, we took note of challenges and things that could be improved by introducing new mechanisms in the series.
One of the challenges was the difficulty of remaining open and self-critical about one’s own education system, to remember that this series is not for sharing good practices, but developing solutions to existing problems in dialogue with peers representing other education systems. To address this challenge the Learning for Well-being Institute (formerly EIESP), the coordinator of the PLL series, and the Autonomous Province of Trentino (PAT) jointly created a background paper prior to the 3rd lab. The paper described the goals of the lab to make the peer learning aspect clearer for the participants. The paper also included a description of Trentino’s enablers and ‘wicked problems’ in promoting data literacy for teaching. By being open about their own education system, PAT encouraged and enabled participants to reflect on their own ‘wicked problems’.
Another challenge faced is the span of time between each lab in the series. Although there are nine sessions, there is still a long time between each event, which might make it difficult for participants to retain the discussions and the key takeaways they had after each. Therefore, two exercises were introduced: the reflection (learning) diary; and the maturity matrix (described below).
Figure 3 IPRASE and schools presented their data-informed inquiry projects. PLL participants had the opportunity to ask questions and comment on each initiative, while the Scientific Advisory Board members of EVIDALI moderated the discussions and offered commentary, further enriching the discussions.
The reflection diary
The reflection diary is a metacognitive, formative evaluation exercise. After each lab, participants will make a new entry in their diary, based on a template with questions. Since each edition has a different focus, some of the diary questions will be adapted to each event.
The diary entries are also a way for participants to take ownership of key concepts, reflect on learning needs, and identify opportunities for further development and improvement.
The benefits of keeping a reflection diary are:
- Participants can record what they learned in each event in their own words. This will consolidate key takeaways for their context.
- Participants may return to earlier entries both to recall and reinforce their own reflections, as well as feedback received from other participants.
- The diary prompts the participants to reflect on next steps they can take to prepare for future sessions and progress toward the mock policy scenario.
After their diary entry, participants will arrange at least one collaborative online meeting across territories to promote mutual learning by sharing diary entries and giving peer feedback. The outcomes of these reflections will be presented and discussed among all PLL participants at the beginning of the next session.
The maturity matrix exercise is a continuation of the shared understanding developed among participants. It is basically a rubric to be co-created by the partners and provides a common thread across the nine PLLs.
The matrix recognises that development occurs in progressive stages (foundation, developing, established, advanced), although not necessarily in a linear fashion (i.e., territories may be more advanced in some areas, and less in other areas). The progressive structure is similar to SIVON’s privacy norms framework, which was explored in Agile EDU, but while this framework focused on the dimension of data privacy at school level, the maturity matrix covers multiple dimensions of the educational data ecosystem (such as classroom assessment, school improvement, professional learning, governance, and data infrastructure).
The matrix includes all levels of the education ecosystem - at micro-, meso- and macro-levels. Its purpose is not to rank territories or prescribe a single linear pathway. Rather, the matrix is a tool for partners to define the different levels of maturity that may exist within a system. They may then identify strengths, gaps, and tensions, and clarify what steps are needed to move forward in their own system, and where elements of the overall data ecosystem may be better aligned (i.e., macro-level polices and school- and classroom-level implementation).
Figure 4 One example from the first co-construction exercise, showing one dimension of the matrix.
In the 3rd PLL participants worked in groups to develop three dimensions of the matrix – focusing specifically on school and classroom level development. The matrix will be further developed through subsequent Policy Learning Labs and the final version will be shared as a tool alongside the policy recommendations.
Active involvement in the programme
The Maturity Matrix and Reflection Diary complement each other. The Maturity Matrix provides a shared framework and common understanding of development stages across macro-, meso- and micro- levels of the school systems common to all territories. Each territorial entity may identify their specific challenges and steps that may be taken to advance to the next level as development. The reflection diary will enable participants to organise their ideas, lessons learned, and how their reflection evolves with every PLL event.
To give more ownership of the PLL series to the participants, we also collect their feedback after each PLL to understand what they would like to explore in the next. Finally, a steering committee will be formed, which is made up of members representing each territorial entity. This committee will provide input on the programme of upcoming sessions, based on their identified needs. The next PLL sessions will invite existing DLFT initiatives in countries that are not involved in the project, and the partners will co-author a policy brief that analyses the policy implications of these initiatives.
[1] The phrase "wicked problem" was coined by Horst Rittel and Melvin Webber in 1973 to characterize a type of social-system issue that is poorly defined, muddled by unclear information, and involves numerous stakeholders and decision-makers with conflicting values. Its effects throughout the system are complex and confusing. "Wicked" here means tricky and hard to manage, not evil.
Rittel, H.W. & Webber, M.M. "(1973). Dilemmas in a General Theory of Planning," Policy Sciences, v 4, (1973), pp. 155-169,
Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor EACEA can be held responsible for them.
