Educational Designer

“Educational design research is the systematic study of designing, developing, and evaluating educational programs, processes, and products.” (Van Den Akker, Gravemeijer, McKenney & Nieveen, 2006a, p. i)

Situated in a theory of social learning, design research is suitable for working with “procedural ambiguity, ill-defined problems, and open systems that are socially multi-level, and multi-timescale” (Kelly, 2006, p. 166) – hence the applicability of this approach to the design of our large-scale teacher professional learning and leadership development framework. As indicated previously, our development and subsequent analysis of this framework represented a deliberate attempt at a specific type of design research – ‘engineering research’ design, the primary aim of which is the development of a new or better product or process. Direct impact on practice provides the main measure of quality, with new insights about the practice serving as an additional outcome (Burkhardt, 2006, p. 121). For the MEPC programme, we sought new insights into whether a particular way of integrating quality assurance, leadership development and professional learning activities (the ‘improved product’) would lead to long-term changes in the quality of leadership, teaching and learning experiences in subject mathematics (the ‘main quality measures’). In addition, and consistent with the aims of design research methodology (Edelson, 2006, p. 101), we also wanted to explore if our design approach could be transformed into a generalisable theory that transcends the subject boundary of mathematics to have application across a range of school-improvement activities.

Our approach embodied the following characteristics deemed common to design research projects (Kelly, 2006; Van Den Akker, Gravemeijer, McKenney & Nieveen, 2006b):

1. Interventionist: the research was aimed at designing an intervention in the real world.

2. Iterative: the design process incorporated cycles of design, evaluation and revision, with insights from ongoing needs analysis activities with involved schools used to directly inform the contents of professional learning activities.

3. Process- and Utility-oriented: the design targeted an improved approach to school improvement and an understanding of the usefulness and effectiveness of this approach for supporting schools with sustainable school-improvement activities.

4. Theory-oriented: the design process and consequent programme framework were based on theoretical propositions; and field testing of the design contributed to theory building about school improvement and validation of the framework.

5. Collaborative: the design process facilitated ongoing collaboration and input from varied parties, including programme designers, participants (teachers, school leaders, support staff), commissioned school-improvement and quality assurance specialists, and commissioned subject-specialists.

6. Multi-level: the design process explicitly targeted a high degree of coordination of activities targeting all levels of the school organisation.

We engaged these characteristics in the design process using a combination of both craft-based and research-based approaches (Burkhardt, 2006), drawing on both professional experiences of working in schools and recommendations of best practice drawn from research literature.

To support our engineering design approach, we drew on Burkhardt’s (2009) three aspects of educational design – technical, tactical and strategic. These provided a helpful structure for directing our attention to different ‘levels’ of the design process, and, so, for enabling a high degree of coordination and coherence between programme activities. Technical design is the design of individual elements of a product or programme. Tactical design is the overall internal structure of the product or programme and includes specification of the core design principles as informed by research. Strategic design is the overall structure of the complete set of products or programmes, considers how the programme will relate to and impact on the end-user, and incorporates relevant role-players from across the education spectrum. Although there is no hierarchy of importance between the aspects, all three need to be carefully coordinated to ensure that the designed product works effectively. This understanding influenced our first design principle for the MEPC programme framework.

Design Principle 1: Strong alignment between strategic, tactical and technical levels of design

Figure 2 provides a tactical-level overview of the MEPC programme elements and sets out the key strands around which programme activities were organised. These strands were operationalised at a technical-level via a series of targeted activities, including twenty-seven professional development offerings (Table 1), eight open-classroom experiences in both Primary and Secondary settings, and a number of in-school activities including audits of mathematics provision and analysis of needs by external specialists, and strategic support for subject and school leaders. A crucial strategic-level dimension of our approach was the coordination of the findings of the comprehensive whole-school audits of mathematics provision conducted in each of the 40 involved schools and the contents of the final professional development and open-classroom offering. This reflected our deliberate intention to synchronise forensic quality assurance activities (conducted with leadership) with professional development and learning activities (for all staff) to ensure targeted, coordinated and sustainable whole-organisation change. The audits were conducted by nationally recognised school improvement specialists together with relevant school, subject and phase leadership teams. They served three purposes. First, they identified the individual needs of each school, which was then used to directly influence and inform the focus of the supplied professional development offering. Second, they supported individual school leaders to identify strategic development and improvement proprieties. Third, they were used as a quality assurance measure, with an abridged post-programme follow-up audit used to assess the quality of engagement, gauge impact and identify further areas for development. Following the baseline audits, each school received personalised communication signposting to relevant professional development activities and in-school support opportunities. This approach ensured that the schools were directly engaged in professional development activities that were relevant to their local contexts and immediate priorities – a further strategic-level design element. By including subject, phase and senior leaders in the auditing process, this also served as a leadership development opportunity, giving all leadership levels insight into the school priorities and buy-in to the programme activities. Other strategic-level design elements included: ongoing dedicated support for school, phase and subject leaders to support and quality assure staff engagement in programme activities and to evaluate impact; follow-up in-school support for participants from external mathematics specialists to ensure transfer of learning from programme activities attended in out-of-school settings; and, a professional development offering catering for all members involved in mathematics provision, including specialist teachers and non-specialist support staff (tutors and teaching assistants), to target all aspects of the students’ mathematical learning experiences.

In keeping with our commitment to a research-based approach, the programme structure, strands, and activities were underpinned by research on school improvement, professional development and learning, leadership, and general and maths-specific pedagogic strategies (explored in more detail in the next section). This research established core philosophical ideas for application across all programme activities (tactical level), directly informing the focus of individual programme activities (technical level). Facilitator Training sessions ensured consistent messages and pedagogical practices between tactical and technical levels of the programme. Participants were also offered opportunities to participate in separate Lesson Study and Action Research communities to investigate issues directly relevant to their local contexts in a more rigorous way. Ongoing and regular evaluation of programme activities by the delivery team provided crucial insights as to areas of impact and areas for further development to inform future strategic-level improvement priorities.

In short, the MEPC framework was engineered as a highly coordinated approach to school improvement, with careful alignment between strategic, tactical and technical elements of programme design facilitating coherence of quality assurance, leadership development and professional learning agendas. This supported a comprehensive offering of professional development and support that directly targeted identified priorities at all levels of the school organisation – leadership, teaching and learning, and children’s experiences – and with all activities underpinned by common and consistent thinking and pedagogies. It is our view that this high degree of coordination supports long-term sustainable impact in school improvement activities, as consistent with other professional development frameworks that yield positive impact (Timperley, Wilson, Barrar & Fung, 2007; Cordingley et al., 2015)

School Improvement

Following a comprehensive survey of school academies, alliances and partnerships across England, Greany (2018) identified five key fundamentals common to school organisations that facilitate sustainable school improvement:

(F1) establish sufficient capacity
(F2) analysis of needs
(F3) deploy and support leadership
(F4) access to effective practice and expertise
(F5) monitor improvements in outcomes.

As will be discussed below, these fundamentals span a combination of, primarily, quality assurance and leadership development agendas which, when coordinated strategically, facilitate the design of professional learning activities that are closely aligned to individual and collective schools’ needs and priorities. As such, these fundamentals proved helpful for informing the focus of a number of our framework design principles targeting the ‘leadership’ priority area (Priority Area 1) in the MEPC framework.

Design Principle 2: Leadership development is prioritised and professional development activities are directly supported by school leadership

Strong leadership with clear vision is central to the success of any school improvement initiative (Day et al., 2009; McAleavy, Riggall & Fitzpatrick, 2016). This is particularly true for schools in poor socio-economic areas, with leadership – alongside high-quality teaching – being important determinants of whether children are afforded effective and safe learning experiences. Furthermore, our professional experiences and observations evidence that persistent issues with the quality of teaching and learning inevitably stem from leadership deficits at subject and/or senior leadership level. As such, school improvement initiatives in schools with ongoing low attainment require a substantial investment in leadership development alongside teaching and learning to ensure sustainable change and improvement.

Since all schools in the MEPC programme were characterised by high proportions of pupils from poor socio-economic backgrounds and with low attainment, a focus on leadership support and development were necessary design elements of the improvement framework (F3). This leadership development and support targeted all levels of leadership in the school, including senior leaders (headteacher and deputies or assistants) and mathematics subject leaders. To begin with, to ensure that leaders were able to fully commit themselves and staff to all programme activities, schools were given a substantial funding allocation to fund release time and cover. In addition, schools could also request further support from central programme team if sufficient staffing or leadership capacity was not available in school (F1). With capacity established, the analysis of needs (F2) involved senior and subject leaders working together with nationally accredited school improvement specialists (F4) to conduct the comprehensive whole-school baseline audits of mathematics teaching and learning provision. These audits sought out commentary from all members of the school community to agree on school improvement priorities. This ensured that offered professional learning and development opportunities were relevant to the needs of the school community and, so, would be actively supported by leadership – both of which are considered to be key components of effective professional learning experiences (Collin & Smith, 2021; Cordingley et al., 2015). These auditing experiences also served as an important source of leadership development, with the external specialists modelling a rigorous and comprehensive quality assurance and forensic analysis process (F3). Then, school and subject leaders were engaged in ongoing in-school support (9 days-worth) by external school improvement and mathematics subject specialists, facilitating the development of long-term school and subject development plans and monitoring activities (F3). Post-programme audits, supported by the external school improvement specialists, offered leadership the opportunity to evaluate overall impact and identify further improvement needs (F5). This auditing loop also facilitated a degree of quality assurance and accountability of each involved school for external reporting purposes to funders. Importantly, the external mathematics subject specialists maintained a supportive and developmental role in their work with subject leaders, acting as a coach to affirm and challenge their strategic plans and monitoring and quality assurance activities. This ensured that subject leaders could treat the experience as a development opportunity rather than an evaluative exercise. However, as part of our commitment to an iterative design process, formative inputs from external specialists were sought to inform the contents of the professional learning offer to ensure relevance to current and changing needs.

Design Principle 3: Professional learning activities are directly relevant to participants’ and students’ needs and focus on improving students’ outcomes and learning experiences

Schools with persistent low attainment need support on specific elements of teaching practice, and generic support that is not tied closely to the local context or to identified priorities does not lead to sustainable change (Cordingley et al., 2018; Stoll, Harris & Handscomb, 2012). By strategically designing the auditing dimensions into the MEPC framework, we ensured that we would be able to deliver in-school support and professional learning experiences (Priority Areas 2 and 3) that were directly linked to both collective and individual context-specific needs and priorities (F2). To this end, the auditing process included lesson observations and classroom visits, analysis of students’ work, and conversations with leaders, teachers, supporting staff (for example, teaching assistants), students and governors. This ensured a wide perspective on students’ and teachers’ learning needs and experiences. These audits then supported each school to map specific areas of priority, informed the programme and contents of the wider professional development offering, and signposted individual schools to specific professional development activities to ensure maximum relevance to participants’ needs (F4).

Design Principle 4: Professional development activities are part of a strategic and long-term improvement plan and target the whole school community

To ensure sustainable improvement from involvement in professional learning activities, these must be directly linked to school priorities and impact must be evaluated in relation to changes in those priority areas (DfE, 2016a). By designing in the deliberate coordination of audit findings with the contents of professional learning activities, and by including school and subject leaders in these quality assurance processes, we modelled a strategic process for integrating targeted professional learning activities into long-term improvement plans (F3).

A further strategic-level element of the framework was the design of programme of activities to target all members of the school community involved in supporting students’ mathematics learning (F4) – hereby supporting a ‘whole-school’ improvement focus. The varied audiences targeted in the baseline audits provided invaluable information about the specific needs and priorities of different groups within the school community (Priority Areas 2 and 3), and this information was then used to inform the programme offering. To this end, we distinguished between professional learning activities targeting specialist teachers and non-specialist teachers and non-mathematics specialist (such as teaching assistants and tutors). For specialists, professional learning activities foregrounded a focus on ‘Quality First Teaching’ (see Figure 2) by exploring innovative teaching strategies. For example, the Problem Solving at KS 1 and 2 activities (Table 1 – shaded cells) explored ways of supporting children to engage in problem solving activities across a range of topic areas. By contrast, for non-specialists the ‘Specialist Knowledge’ strand was prioritised. For example, the Specialist Maths for Teaching Assistants course ran over six half-days and focused primarily on supporting participants to develop a deeper conceptual understanding of topics. However, since many teaching assistants work with smaller groups of children who struggle in mathematics, pedagogic strategies for children with maths-specific learning difficulties were also foregrounded.

An omission from the final offer was an activity targeting support for parents. Given the importance of parental involvement for supporting children’s learning (Castro et al., 2015), we recognise this as a shortcoming of the programme offering. This is an area of focus we aim to research further and address in future school-improvement work.

Design Principle 5: Sustainable improvement is supported by ongoing research of practice and impact evaluation

Sustainable improvement requires regular evaluation of needs and impact and reflection on current practices. We operationalised this principle in two ways. First, the auditing activities modelled a form of forensic analysis and evaluation to identify needs and priorities – which is consistent with Greany’s (2018) fundamental principle of monitoring improvements in outcomes (F5). Second, we applied an expanded understanding of monitoring in our design process to include a focus on schools actively researching their own practices. Recognising the value of research for informing classroom practices (Brown & Zhang, 2017), we see the capacity to ‘particularise theory’ (Leinhardt, Young & Merriman, 1995) as an essential element of sustainable school-improvement. As such, we designed in opportunities for schools to engage in collaborative ‘Research Communities’ (see Figure 2) focused specifically on trialling and evaluating strategies for improving children’s learning experiences and independent learning behaviours (Priority Area 3). The Action Research Workgroup: Transition & Pre-teach Intervention community provided teachers with the opportunity to design and trial bespoke context-specific intervention strategies for students in their local settings. The Lesson Study Workgroup: Supporting Inclusive Classrooms community provided a collaborative space for teachers to develop shared resources targeting children with special educational and learning needs, to trial and observe the use of these resources in live teaching sessions, and to reflect collaboratively on learning and impact.

Professional learning and development

The professional learning agenda in the MEPC framework is embodied within a comprehensive offering of professional learning activities directly informed by schools’ individual and collective needs and priorities identified via the quality assurance and forensic analysis processes. This agenda was operationalised via design principles focused on giving participants agency and autonomy to transform their own practices, supporting participants to transfer learning to local contexts and settings, and enabling participants to develop varied knowledge and skills to enhance children’s learning experiences.

Transformative professional development

We make an important distinction between professional learning and professional development in our work with teachers, with the latter referring to the collaborative processes and activities that teachers engage in to develop, challenge, and reflect on their understanding of subject-specific content, pedagogic strategies, and more general aspects of educational practices. For us, the primary aim of professional development, then, is to facilitate deep and sustained professional learning, and to support teachers and school leaders to reflect and question their existing understandings and practices and the values and beliefs that underpin these. In other words, to challenge their professional identities (Sammons et al., 2007). This priority is consistent with others professional development activities that have effectively supported changes in teacher’s practices (Cordingley et al., 2015; Timperley, Wilson, Barrar & Fung, 2007).

This perspective foregrounds a ‘transformative’ agenda (Kennedy, 2005), ensuring that participants play an active role in the learning process and act with professional autonomy. This transformative agenda increases capacity for professional autonomy – or what Hargreaves and Fullan (2015) refer to as ‘professional capital’, which we believe is a necessary condition for supporting schools and school communities to take ownership of school improvement processes. As shown in Table 2, this agenda is more likely to be facilitated through specific types of professional development activity that enable participants to have autonomy and ownership of their learning and development. Consideration of the above prompted our sixth design principle.

Model of Professional Development (PD)	Outcome	Purpose
Training	Skills-based approach where passive participants receive training from an ‘expert’ to update their competence.	Transmission	Increasing capacity for professional autonomy
Award-bearing	Relies on completion of an award-bearing programme that is externally validated.
Deficit	Designed to address an externally perceived deficit in teacher performance.
Cascade	Teachers attend training and cascade learning to colleagues in local contexts.
Standards-based	Teachers develop their teaching around a collection of professional actions which evidence that a certain standard of teaching has been achieved.	Transitional
Coaching/mentoring	Centrality of a one-to-one relationship between professionals to supports professional learning.
Community of practice	Similar to coaching but involving a wider collection of professionals who support each other’s professional learning.
Action research	Participants engage collectively in a research process to investigate issues relevant to their practice.	Transformative
Transformative	“ … the key characteristic of the transformative model is its effective integration of the range of models described above, together with a real sense of awareness of issues of power, i.e., whose agendas are being addressed through the process.” (Kennedy, 2005)

Design Principle 6: Professional development activities enable and support participants to evaluate their practices and to have autonomy and ownership over (changes to) these practices

The far right-hand column of Table 1 lists the professional development model associated with each programme activity in the MEPC framework and demonstrates how we operationalised this principle in the framework design to prioritise development in teaching and in-class support (Priority Area 2) and children’s learning experiences (Priority Area 3). By engaging this principle in the design process, we ensured that participants had access to a collection of professional learning experiences embodied within a combination of cascade, coaching, community of practice, action research, and transformative models of professional development, since these are more likely to facilitate a transformative agenda (Kennedy, 2005). By contrast, training, award bearing, deficit and standards-based models of professional development were downplayed. The potential for once-off sessions to function as training rather than cascade models was offset by the in-school support element that directly aided participants to cascade learning to wider staff teams. Furthermore, this extensive in-school support element, reinforced by several community-of-practice research activities, and quality assured through the follow-up audit process, spearheaded a transformative agenda for equipping teachers and leaders with the capacity for increased ownership and autonomy over school-improvement activities.

Enabling transferability of learning to local contexts

Joyce and Showers (2002) argue that it is the combination of theory, modelling by experts, opportunities for practice and application in local contexts, and coaching, that affords the greatest potential for the transferability of learning from training environments to classroom practice (see Table 3). Theory provides a common language and understanding around which to frame a learning experience, giving insight into the underpinning rationale of the professional development approach and an evidence-base of tested practices shown to impact learning. Experts help to mediate and translate this theory into workable classroom activities, support participants to improve their understanding of the evidence, and offer alternative perspectives that challenge teachers’ beliefs and expectations about how students learn (DfE, 2016b). Opportunities to apply learning in local contexts are vital for enabling participants to translate learning into practices relevant to local needs and priorities, followed by opportunities for reflection, discussion and deconstruction (McDonald, 2014). Experts support this process by modelling effective practice and coaching participants via observation and feedback (Cordingley et al., 2015).

	OUTCOMES (% of participants who demonstrate knowledge, demonstrate new skills in a training session, and use new skills in the classroom)
Training Components	Impact on Knowledge	Impact on Skill	Transfer to Practice in the Classroom
Theory	10%	5%	0%
Theory + Modelling	30%	20%	0%
Theory + Modelling + Practice	60%	60%	5%
Theory + Modelling + Practice + Coaching	95%	95%	95%

Design Principle 7: Professional development activities include, where possible, a combination of theory, modelling by experts, opportunities for practice and application in local contexts, and coaching

For shorter courses (for example, half-day content courses), the ‘coaching’ dimension was again facilitated via the in-school support element. All sessions, irrespective of duration, included theory on content and pedagogical knowledge (see Principle 8 below) and modelling of classroom strategies. Exposure to additional expert modelling was facilitated via carefully orchestrated open-classrooms experiences, top-and-tailed by pre- and post-lesson discussions. Dedicated Facilitator Training sessions prior to the launch of the programme offering ensured collective understanding of the importance of this combination of elements, with the first author quality assuring all materials prior to each professional learning activity.

Combined focus on content knowledge, pedagogy and independent learning strategies

The understanding that gave rise to our final design principle was summarised by Cordingley.

“All the reviews [of effective professional development] found that pedagogy and subject knowledge were equally important; the strongest single review went further to state that CPDL [continuous professional development and learning] focussed on generic pedagogic strategies is insufficient, particularly in maths, and that it is important to consider several alternative pedagogies for specific pupils too.” (Cordingley et al., 2015, p. 5)

Design Principle 8: Professional development activities prioritise both subject-specific content knowledge and pedagogical knowledge, with some additional focus on more general strategies for supporting students to become independent learners

Given the mathematics-specific focus of the MEPC framework, we drew heavily on Ball, Thames, and Phelps' (2008) work on domains of mathematical knowledge for teaching to frame activities targeting the development of teachers’ practices (Priority Area 2). This ensured a dedicated focus in all programme activities on both subject matter knowledge (SK) and pedagogical content knowledge (PCK). The extent to which elements of SK or PCK were prioritised was influenced by the target audience of each programme activity. Activities targeting subject specialists (e.g. teachers and maths subject leaders) prioritised the development of PCK over SK. By comparison, activities targeting non-subject specialists (e.g. teaching assistants and tutors) prioritised SK over PCK.

Additionally, and specifically in support of the priority for impacting children’s learning experiences (Priority Area 3), the MEPC framework included a selection of professional learning activities focussed on general and mathematics-specific strategies for supporting independent and reflective student learning, drawn from research about cognition, self-regulated learning, and positive dispositions and mindsets (Boaler, 2015; Cragg & Gilmore, 2014; Kilpatrick, Swafford & Findell, 2001; Quigley, Muijs & Stringer, 2018; Willingham, 2009). Our motivations here stem from an understanding that most curricular, teaching and assessment processes foreground the domains of mathematical content, knowledge forms, and competencies; however, underlying cognitive challenges and deficiencies (including with working memory, central executive functions, visuospatial and language systems) can have a significant impact on mathematical learning and experiences (Cragg, Keeble, Richardson, Roome & Gilmore, 2017). In addition, both teacher’s and children’s attitudes about mathematics directly influence their learning experiences (Boaler, 2015; Ramirez, Hooper, Kersting, Ferguson & Yeager, 2018). As such, sustainable improvements in children’s learning experiences in mathematics require teachers and leaders to put systems and strategies in place that reduce emphasis on memorisation, support children’s independent learning behaviours, recognise the impact of cognitive structures on learning, and engender positive mathematical mindsets. This commitment is reflected in the suite of professional learning and research activities in the MFEC framework specifically targeting children’s learning experiences:

1. Using learning environment and bookwork structure to support children’s learning

2. Building positive attitudes and attributes in mathematics

3. Tackling misconceptions, collaboration, independent learning and problem solving in secondary maths

4. Action research workgroup: Transition & pre-teach intervention

5. Lesson study workgroup: Supporting inclusive classrooms