Evaluating Edtech for Efficacy, Standards-Alignment, and Student Growth

A 5-Step Approach to Choosing Digital Curriculum the Wichita Way

People who’ve never been to Wichita think it’s a rural area—some vestige of the Wild West, replete with tumbleweeds and free-roaming cattle. Not so. Today, Wichita is quite urban—it’s an industrial hub and the largest city in the state of Kansas. In fact, the Wichita Public Schools district is the biggest between the Mississippi and the Rockies, and it serves a richly diverse population of more than 50,500 students from 86 different countries. For this reason, Wichita’s recent success implementing digital curriculum district-wide provides a valuable use case that will resonate with many looking to make the leap to edtech.

Back in April, Debbie Thompson, mathematics curriculum specialist for the Wichita district, presented an EdWeek Webinar, “Going to Scale: Implementing Evidence-Based Personalized Learning for Math Intervention.” In this popular 60-minute session available now on demand, Thompson neatly outlined the step-by-step process her district took to identify, evaluate, implement, and scale a successful intervention program. We briefly recap the Wichita approach below—up to and including the decision stage.

    1. Step 1: Assemble a dedicated team of stakeholders. The first thing Thompson and her colleagues did was assemble a math curriculum work group—a team that included curriculum specialists, classroom teachers, special education teachers, math intervention teachers, head principals, assistant principals, and English for Speakers of Other Languages (ESOL) teachers. Because administrators and educators at every level district-wide would be expected to implement the new solution, it was important that their interests be represented at every step in the decision-making process. To that end, a key stakeholder from each population participated in the math curriculum work group.
    2. Step 2: Identify evidence-based solutions. Next, Thompson’s math curriculum group had to do a little homework. They sought quality intervention programs that had been vetted by trusted educational research organizations, including What Works Clearinghouse, the Institute for Educational Sciences, experts affiliated with the National Council of Teachers of Mathematics (NCTM), and others. Thompson explains, “We needed to make sure we weren’t just getting any program.”
    3. Step 3: Evaluate against established criteria. Once the group compiled a list of viable and proven digital curriculum for consideration, it was time to narrow the field. They measured each program against carefully crafted criteria including this set of “non-negotiables:”
      • Alignment to content standards and math practices
      • Support of conceptual understanding for students
      • Teacher support to plan and provide effective learning experiences
      • Backing of a leading expert/organization within the field of mathematics

Programs that didn’t satisfy these stringent criteria were immediately rejected. Those that passed were next measured by a series of “essential/critical considerations” that could impact implementation, including the ability to:

      • Support quality instruction
      • Provide evidence of student learning
      • Demonstrate accessibility and responsiveness

The complete Mathematics Intervention Evaluation Criteria document that the Wichita Public Schools district used is part of a larger packet of reference materials available to webinar attendees. We encourage you to view the recorded session to access your free copy of these helpful resources.

At the end of the evaluation cycle, the team already knew their top choices would impact student achievement. Next they had to determine which program was the most practical. Could teachers easily implement it? Could they use it to get their students where they needed to be?

  1. Step 4: Set up a pilot process. Once the team narrowed their choices based on the aforementioned criteria, they created a pilot process to demo each program under consideration that included three 4-week cycles. During this phase, each teacher piloted at least three different programs, and administrators and curriculum leaders made classroom visits to observe students and teachers using each program. At the end of each cycle, the pilot teachers met with the curriculum department to provide feedback. When the entire pilot process was complete, all participating teachers completed a comparison evaluation form and ranked each program thinking about how they met those “essential/critical considerations” identified during the preliminary evaluation stage. NOTE: While most programs were available for free trial, some required a nominal fee to implement and test.
  2. Step 5: Decide based on the data. After the pilot phase was complete, the mathematics curriculum work group compiled, processed, and evaluated all the feedback and made an informed recommendation to district leaders.

In the end, the entire intervention adoption process took the Wichita Public Schools district about two years to complete. Thompson and her team piloted five intervention programs, but one clearly rose to the top. Tune in to the recorded session now to see which digital platform they chose, why it was so important to test multiple options, how they arrived at their decision, and how it’s all working out a year later. Remember, when you view the webinar you can also download a free packet of tools and resources the Wichita team used to conduct their evaluation. Enjoy!

Jennifer Lee

Jennifer Lee

Jenny Lee is a passionate marketer at DreamBox Learning, where she manages national events, webinars, and social media. A lifetime learner, Jenny is currently pursuing her MBA at the University of Washington's Foster School of Business. In her free time, she enjoys outdoors adventures with her dog Allie in the beautiful Pacific Northwest, and chasing creative whims through visual art.
Jennifer Lee