Educational Designer

Educators today are questioning the viability of instructional approaches and assessments that do not directly support students to flexibly transfer their learning to new contexts. It is important to develop educational resources that align instruction with assessment, which in turn enable students to think as scientists and engineers as they encounter phenomena and design solutions to problems. To overcome these challenges, Principled Assessment Design (PAD) and Understanding by Design (UbD) frameworks have been used to show how clarity and coherence within and across standards-based assessment and classroom contexts can be achieved. However, there are challenges in mapping the two approaches due to the domain-specific nature of PAD as opposed to the more comprehensive nature of the UbD process. In this paper, we discuss how transfer and alignment to learning goals informed design decisions that were made to integrate the benefits present in each model. The resultant resources are geared at providing equitable and accessible learning opportunities and assessments for all students. Further recommendations about the quality and usefulness of these resources are made based on feedback received from parents, educators, and administrators across several states.

Since their introduction in 2013, twenty states have adopted the Next Generation Science Standards (NGSS). An additional 24 states have also adapted rigorous and challenging science standards influenced by A Framework for K-12 Science Education (Openscied, n.d). However, many educators are struggling to teach the required knowledge, skills, and abilities in a way that incorporates students’ prior learning opportunities and engages them in local phenomena that encourage sensemaking (Schwarz et al., 2017; Haverly et al., 2022). Through the SIPS ((Stackable, Instructionally-Embedded, Portable Science Assessments), we have developed the necessary foundation for an integrated instruction and assessment approach by creating tools that educators, students, and parents need to leverage high-quality assessment to achieve the following goals:

1. Establish a collection of instructionally-supportive science assessment tasks aligned with clearly defined learning goals.
2. Use the developed resources to build state and local educators’ capacity to offer high-quality science instruction, evaluate students’ learning, and make data-based instructional decisions; and
3. Engage educators, students, and parents in a partnership for student success in science across a range of implementation circumstances.

This paper is focused on the resources that were developed through the SIPS project. Table 1 and Table 2 describe these resources, and the relevant terminology from the NGSS. The project produced year-long model coursework grounded in phenomena and NGSS topic bundles at grades 5 and 8, as well as process guides articulating the design approach and process. All the resources are freely accessible via a web-based resource center (SIPS, 2021). The goal is to facilitate collaboration and communication among teachers at the local level to share best practices and useful resources.

Enlarge…

Enlarge…

Curriculum and assessment development initiatives are all too often carried out independently from each other. As a result, educators lack science curricula and assessments that work together to support teaching and learning. This often leads to a lack of alignment between the curriculum goals, assessment, and instruction. Additionally, it does not ensure students receive meaningful and adequate learning opportunities that they can apply to real-life situations. We provide a solution to this problem by building a coherent system that aligns curriculum, assessment, and instruction.

Our design philosophy is centered on the Assessment Triangle (Pellegrino, Chudowsky & Glaser, 2001) and the necessary coherence among its three elements: cognition, observation, and interpretation (Nichols et al., 2016). This philosophy enabled us to (a) carefully define the expectations for learning in relation to standards as well as research on how students gain competency toward those standards and (b) design observations (assessment tasks) that can elicit information about students’ learning status and progress so that (c) this information can be interpreted and used to support and to evaluate student learning.

Principled Assessment Design (PAD) was also applied to articulate expectations and design decisions to guide claims about the meanings of student scores (Mislevy & Haertel, 2006). PAD approaches have been successfully used in the past to design assessment tasks and learning activities (Next Generation Science Assessment, n.d; Gane et al., 2024). This approach begins with a deep analysis of the target domain for assessment. PAD approaches have three characteristics in common. The first is that they are construct-centered, which makes all design decisions centered on the original definition of the constructs to be assessed. Secondly, PAD approaches are geared towards solving design challenges by collecting evidence to support the intended interpretation and uses of the assessments. Finally, the need for collecting evidence necessitates making all design decisions and rationales explicit and transparent, which is best facilitated by the collection of documentation to support them (Ferrara et al., 2016). Out of all PAD approaches, Evidence Centered Design (ECD; Mislevy & Haertel, 2006) has gained the most widespread use and implementation. Its key features (domain modeling, domain analysis, conceptual assessment framework, assessment implementation, and assessment delivery) were used as a guide to develop our tasks and resources.

The resulting tasks and instructional resources were also embedded within Understanding by Design (UbD; Wiggins & McTighe, 1998) instructional frameworks that used a backward approach by first designing the goals to be assessed as a means of informing the instructional process. As such, the learning goals are contextually grounded in ways that support students’ optimal demonstration of their knowledge, skills, and competencies (Fischer et al., 1993; Yan & Fischer, 2002).

Both ECD and UbD approaches have similarities in the ways that they both look for evidence present in the learning outcomes of students, driven by the curriculum and assessment goals. In the section that follows, we demonstrate the procedure involved in mapping both approaches through the use of alignment and transfer of learning goals.

The design and development of the instructional frameworks and assessments was an iterative process that revolved around overcoming the design challenges of combining the domain-specific nature of ECD (Mislevy & Haertel, 2006) with the more comprehensive approach of UbD. We began the process by seeking answers to the following questions:

1. What knowledge, skills, and abilities can be assessed for a selected three-dimensional science performance expectation?
2. What student products (what students produce in a task) can provide evidence of learning the selected knowledge, skills, and abilities?
3. What tasks or situations should elicit products that can be teacher-scored across a range of student abilities?

Each instructional framework was divided into three sections as follows: Stage 1, the desired results, provides the expected learning outcomes of the unit. These include a) the performance and learning expectations covered in the unit; (b) major concepts and questions that students would explore (i.e., the enduring understanding and essential questions); and (c) the breakdown of specific content knowledge and skills that students would need to master the learning expectations. Stage 1 outlines what students need to know and be able to do by the end of the unit. Using the components of the topic bundles, a student profile was generated for each unit. The profile documents the overall measurement target that integrates the core ideas, practices, and crosscutting concepts into a single statement representing what is to be taught and assessed in each unit. These profiles outline the prior learning and future learning opportunities for each unit guided by the NGSS and the National Research Centre (NRC) Framework. They also outline how the content of the performance expectations could be developed into a structure for the development of core ideas, practices, and crosscutting concepts throughout the unit.

Stage 2, the assessment evidence, describes the means of assessing the concepts, knowledge, and skills from Stage 1. This includes:

1. graphic organizers, exit tickets, and discussion prompts defined as “in the moment” assessment opportunities that identify student challenges and lack of knowledge or alternate conceptions;
2. formal assessments (e.g. performance tasks, projects) that measure how well students perform when engaging in more complex tasks that require integration of the three dimensions (core ideas, practices, crosscutting concepts) in the service of sensemaking. They are administered at specific, intentional points in time along an instructional plan before or after a lesson or a series of lessons; and
3. the end-of-unit assessment, comprised of three tasks that support educators in evidence-driven planning for subsequent instruction after each unit in combination with evidence from the informal and formal instructionally-embedded assessments.

Stage 2 explains how teachers would evaluate the level of student understanding based on the information taught. This section of the instructional frameworks makes a direct connection between the learning expectations and content to be delivered in a unit and how students and teachers would evaluate learning and mastery of those expectations. The instructional framework offers embedded links to sample instructionally-embedded assessments that provide data to inform instructional planning.

Stage 3, the learning plan for each unit, aligns with Stage 1 and Stage 2 of the instructional framework. It provides a roadmap that teachers can use to design a plan of instruction that helps students attain the unit’s learning goals (Stage 1) and respond effectively to the assessment and evidence targets designated in Stage 2 for the unit.

The above perspective enabled us to achieve coherence at the unit level by guiding coordination and alignment across the three major components of curriculum, instruction, and assessment. By referring to the Stage 1 learning goals and engaging teachers and other educators in the development processes, the tasks, and ultimately the assessments composed of them, yield evidence regarding critical aspects of what happens in the classroom. The next section outlines and discusses the design decisions made to build coherence within and across instructional frameworks

Connecting ECD and UbD approaches enabled us to incorporate transfer into the learning process and also align curriculum, instruction, and assessment within and across units. We were able to incorporate the benefits of each approach into the following phases of the design process: 1) Unpacking of the core ideas, practices, and crosscutting concepts; 2) Developing big ideas; 3) Identifying enduring understandings and essential questions; 4) Developing acquisition goals to support transfer and alignment of learning; and 5) Aligning the instructional sequence with stage 1 and 2.

Each phase was guided by design decisions geared towards achieving coherence between the learning goals, assessment tasks, and instruction as described below:

Development of Big Ideas

Big ideas were developed to describe what students would work on during the delivery of a coherent sequence of lessons and opportunities to learn. The process involved describing the disciplinary knowledge best aligned to the unit’s Performance Expectation bundle that students would acquire in the context of a phenomenon or design problem. The development of the big ideas also considered how students would transfer their knowledge to different situations as they integrate the dimensions to answer specific questions about the natural and designed world. Table 3 shows the big ideas developed for Unit 1 as aligned to the core ideas of the Performance Expectation bundle.

Identification of Enduring Understandings and Essential Questions

Following the UbD model of developing Stage 1 goals, we identified enduring understandings and essential questions that were aligned to the dimensions present in the topic bundle. An enduring understanding is an overarching statement that reflects a deeper internalization of a topic and may connect to a real-life issue or a larger understanding of the world for both students and teachers. It reflects an important idea that has lasting value beyond the classroom and should be transferable beyond the scope of a particular unit. On the other hand, an essential question is an open-ended question that provokes sustained inquiry and meaningful reflection that leads the student to enduring understandings (Wiggins & McTighe, 1998). Overarching essential questions point beyond the particulars of a unit to the larger skills and understandings. Topical essential questions address the specific core ideas in focus for the unit. Both enduring understandings and essential questions guided assessment and instruction design decisions incorporated a principled design approach, ensuring evidence to support the intended purpose and use of the entire curriculum. This approach guided the scoring and interpretation of students’ scores. Table 4 shows the essential questions and enduring understandings used to develop the first unit for Grade 5.

Use of Acquisition Goals to Support Alignment and Transfer of Learning

Stage 1 of each unit includes a set of learning goals. These encompass acquisition goals reflecting different combinations of the core ideas, practices, and crosscutting concepts that comprise the topic bundle. The acquisition goals aim to enable students to flexibly use their knowledge with a range of practices and crosscutting concepts not only within the unit but also in other contexts/situations. Thus, providing students opportunities to learn the acquisition goals can prepare them to demonstrate the associated transfer goal at some point in time.

The acquisition goals did not reference specific phenomena and design problems. Therefore, the instructionally-embedded assessments that were developed in Stage 2 of the UbD process can be flexibly used with other instructional frameworks (e.g., Open Educational Resources state-developed resources) besides the Stage 3 learning plan.

Using Ruiz-Primo et al.’s (2000) levels of knowledge transfer, two claims were made about how knowledge acquisition, in the context of a phenomenon or phenomenon-rooted design problem based on the topic bundle, may occur:

1. transfer as demonstrating sense-making through the flexible application of knowledge through integrating the original combinations of dimensions within the performance expectations from the unit bundle, i.e., close transfer.
2. transfer as the flexible application of knowledge through integrating new/different combinations of the dimensions represented by the performance expectations in the bundle/unit, i.e., proximal transfer.

The design of the tasks determines how close or proximal those assessments are to instruction. A family of tasks could be created in which a task is close to instruction (same phenomenon, context/situation) and other tasks are proximal (somewhat close, but slightly more distal to instruction).

Table 5 presents sample acquisition goals developed to address transfer goals aligned to the 3 dimensions (core ideas, practices, crosscutting ideas) present in the unit.

The unit’s storyline begins with students exploring an anchor phenomenon. They attempt to make sense of the phenomenon by connecting it to what they already know about it and identifying what they are curious to know. The anchor phenomenon provides a context to raise questions that initiate a sequence of investigations for a unit. Students uncover what needs to be figured out. As noted by Reiser (2021), “it is not the sole thing that a class will try to explain and may not even be the most central one" (p. 815). In a unit storyline model, related phenomena are also identified. Over the course of the unit teachers and students are introduced to a related phenomenon that could help them know more about the anchoring phenomenon. These related phenomena are unique to a particular instructional segment or are used only within a single lesson. As part of the storyline design, students identify questions and ideas for investigations. As the unit progresses, students are given opportunities to answer questions generated when they first explored the anchoring phenomenon and generate more throughout the subsequent instructional segments.

An instructional segment articulates a portion of the unit’s learning goals as defined in Stage 1 of the UbD process. These learning goals are used in a corresponding Stage 2 segment to identify and describe the assessments that would be used to collect evidence of students’ learning throughout the unit and instruction. The lessons in each instructional segment are designed to ensure students have opportunities to acquire and apply the learning goals from Stage 1. Each instructional segment that was developed has the following characteristics: (a) an outline of four to eight lessons; (b) at least two out of the five phases of the BSCS 5E model per lesson (Bybee, et al., 2006; (c) a description of what students are expected to do during each lesson, and (d) a list of acquisition goals covered in each lesson. The complete set of lessons in an instructional segment covers all the acquisition goals previously identified (in Stage 2) for that segment and all five phases of the 5E model.

Within each unit’s instructional segments, sample lessons were developed for use by state and local administrators and teacher leaders (e.g., curriculum directors, instructional facilitators, and professional learning specialists) to: (1) illustrate examples of instructional lessons developed using an ECD approach, and (2) support accompanying process documentation about how to use the instructional frameworks to intentionally design high-quality lessons in an aligned curriculum, instruction, and assessment system. Figure 4 is an outline of the components of a sample lesson, illustrating how learning goals, formative assessment opportunities, resources, and core text connections are integrated into the learning process.

In summary, we have demonstrated the importance of ensuring that each phase aligns with the learning goals to ensure they measure the original knowledge, skills, and abilities as defined in the topic bundles. In doing so, students can transfer knowledge within and across instructional frameworks, ensuring coherence throughout their learning process.

We have demonstrated the importance of aligning curriculum, instruction, and assessment that will be useful for state and district science specialists and classroom teachers. At a global level, our approach will be helpful to educators struggling to adopt rigorous and challenging science standards. Our design approach is also useful to curriculum developers struggling to design and implement assessments that adequately reflect science standards and meet technical quality and validity requirements. We believe in a coherent research-to-practice model with rigorous evidence to support design and evaluation decisions which in turn supports better classroom instruction.

The design decisions that resulted in developing the instructional frameworks show that it is possible to create a coherent system of curriculum, instruction, and assessment guided by ECD and UbD approaches applied to NGSS. This was made possible by incorporating acquisition goals developed using ECD design principles throughout the three stages of the UbD approach. Acquisition goals proved useful in ensuring alignment of the learning goals throughout the unit. They also support the transfer of learning by combining different dimensions to provide opportunities for close and proximal transfer within the unit.

Feedback from educators shows that SIPS instructional frameworks are best used to supplement other curriculum materials that have been used in the past, which can be adapted to align with the acquisition goals. The effectiveness of the entire curriculum in improving student achievement and understanding of the dimensions is still under investigation. However, there is considerable evidence that shows that this approach to designing coherent systems of curriculum, instruction, and assessment will give students the agency they need to solve real-life problems as they engage in three-dimensional science learning.