To measure impact, school leaders and teachers need to be clear about whose impact on what. Many factors influence student performance but the teacher is the largest in school variant on student performance (30%) (Hattie 2003). In this context the teacher’s impact on the students’ learning is considered and as an extension; what professional learning do the students need their teachers to undertake. This will shift over time and needs to be an embedded and iterative part of a dynamic school evaluation approach to improving student outcomes through building teacher capacity in a High Impact Professional Learning environment.
Teacher impact and identifying the right data
Determining teacher impact on student learning relies on strong evaluative practice and reliable use of data. Schildkamp (2019) states that the effective use of data for school improvement does not happen in isolation – its use is influenced by system, organisation and individual/ team factors. She goes on to note that there needs to be a consistent approach to defining goals for data use, the collection of different types of data, sense-making, taking improvement actions and evaluation; and that some of the most important enablers and barriers to effective data use are data literacy and leadership.
To measure impact effectively:
- ascertain the outcomes that students need to achieve
- establish a relevant and reliable baseline of student capability in relation to set outcomes
- plan a logical (and evidence based) intervention for teacher behaviour
- execute the planned intervention (and evaluate the quality)
- conduct post-test of student progress towards the outcome
- compare pre- and post-test data to determine teacher impact.
"There is a need to acknowledge the student voice about teacher impact… student voice is often highly reliable, rarely includes personality comments and can hence provide meaningful insights into understanding and promoting high impact teaching and learning." Hattie, 2015a
Focus on continuous improvement for impact
A culture of continuous improvement requires teachers and school leaders to plan for, and articulate, the processes which will demonstrate the impact of their own learning on student progress and achievement. When this is done well, teachers can demonstrate the impact of their professional learning on student progress and achievement. This allows teachers and leaders to answer the question: ‘How do I know it has worked?' (Hattie, 2015a; Timperley, Wilson, Barrar & Fung, 2007).
Link to a cycle of inquiry, action learning and improvement
Many high performing systems use a version of an improvement cycle to structure their improvement models and the professional learning of their teachers (Jensen, Sonnemann, Roberts-Hull & Hunter, 2016). A relevant, collaborative, and future-focused improvement cycle supports teachers to reflect on, question and consciously improve their practice. The challenge is to determine how the evidence of impact is then linked to school improvement within the context of a clearly understood cycle of improvement. What are the key components of an evaluation cycle that will ensure the successful impact of professional learning aligned to a particular intervention? The five levels of evaluation set out by Guskey (2000a) describe a series of interwoven evaluation points where evidence gathering becomes increasingly complex through each stage. The levels are:
- Participants’ Reactions: registering satisfaction with the learning
- Participants’ Learning: measuring growth in knowledge and skills
- Organisation Support and Change: gathering information on the organisation’s characteristics and attributes necessary for success
- Participants' Use of New Knowledge and Skills: typically gained from observations, this level allows for restructuring professional learning for more consistent implementation
- Student Learning Outcomes: has the professional learning impacted student growth and achievement?
Begin with the end in mind
In high performing schools, professional learning is backward mapped to valued student learning outcomes, and the evidence that will best reflect those outcomes (Guskey, 2001). Measures of student learning typically include cognitive indicators of student performance and achievement, such as standardised tests and assessments. However, there are other indicators of student progress and achievement that can assist in understanding and evaluating professional learning, such as students' self-concepts, study habits, school attendance and classroom behaviours and observations. Backward mapping professional learning against valued student outcomes means a clear vision for professional learning, where all activities are measured in terms of their impact on these outcomes (Le Fevre, Timperley, Twyford & Ell, 2020).
Learning by doing – evaluating the impact of professional learning in the classroom
Teachers can draw on the expertise of their colleagues in developing new ideas and interventions to address their own specific learning needs. Leveraging the expertise of their colleagues is effective and increases the engagement of colleagues whose expertise is being privileged. This enables teachers to directly apply their learning in the classroom, evaluate the impact of ongoing development, and use this formative data to guide ongoing improvements in professional learning design, implementation and follow up (Guskey 2016). In high performing systems, schools make space for teachers to engage in school-based research to inform ongoing development of practice (Jensen et al., 2016) and focus on content directly relevant to the specific curriculum (Darling-Hammond et al., 2017).
