Agile Development to Support State Assessment Analyses

Intern Update #1

Brian Harrold and Erik Whitfield
June 20, 2025

Project Overview

Brian Harrold and Erik Whitfield

Description

Create a suite of R packages to:

  1. Support policy-relevant analysis of state educational data
  2. Develop reproducible R functions for common analytic tasks
  3. Use an iterative development process to build, test, and refine tools
  4. Empower state analysts and researchers to produce high-quality, replicable reports

Goal

Develop robust functionality to:

  • Facilitate routine analysis
  • Streamline standard analytic workflows
  • Enable deeper, exploratory data investigations
  • Generate clear, reproducible reports and presentations
  • Provide comprehensive supporting documentation

Workflow

  • R and RStudio – Programming and package development
  • GitHub – Version control and website hosting
  • Quarto – Website creation and dynamic report generation

Brian Harrold

cohortED badge

Cohort Analysis

https://stat-brain.github.io/cohortED/

Erik Whitfield

bueller badge

Participation and Chronic Absenteeism Analysis

https://erwx.github.io/bueller/

cohortED package

Brian Harrold

Focus of the Package

  • Cohort Analysis for Educational Research
    • Demographics
    • Student Mobility
    • Longitudinal Analysis (tracking cohorts over time)
    • Cross-Sectional Analysis (comparing cohorts)

Why cohortED?

  • How do students move between grade levels over time?
  • Who is present in one year but not the next?
  • When and where do enrollment patterns shift?
  • How do these trends vary across districts or subgroups?
  • How does student mobility impact metrics, such as performance or graduation rates?
  • How do different cohorts compare to each other over time?

Grade-to-Grade Transitions

Grouping students based on movement across adjacent grades:

  • Leave - in the earlier grade but not the next
  • Stay - in both grades
  • Join - in the later grade but not the previous

To support measures of:

  • Persistence - proportion of students who stay
  • Churn - combined rate of students who leave or join

Looking for Your Insights

Please consider:

  • What additional features or capabilities would be valuable?
  • Are there any specific challenges or questions that states want addressed?
  • Are there datasets that could help improve and validate the package?

Ideal Dataset

  • Longitudinal Student-Level Data
    • Example: sgpData_Long from SGPdata package
    • Requires: Student ID, Grade Level, Academic Year
    • Optional Demographics, Subgroups, Performance, School/District, etc.

Current Analysis

  • Data Manipulation and Cohort Construction
  • Grade-to-Grade Transition Tracking
  • Cohort Demographic Comparisons
  • Cross-Cohort Performance Summaries
  • Analysis of Mobility’s Impact on Performance

Future Analysis

  • Cross-District and Cross-School Mobility
  • Advanced Subgroup Analysis
  • Expanded Longitudinal Cohort Tracking
  • Improved Visualization of Churn and Persistence

Report Generation

Currently, reports are user-built with Modular Functions that have list output

  • Updated or refined datasets
  • Formatted table objects and summary statistics
  • ggplot2 plots
  • Auto-generated captions
  • Designed to be embedded in R Markdown or Quarto

Future versions will offer pre-built templates with dynamic text and visual explanations.

Example Report

View Example Reports

Persistent Challenges

  • Designing flexible function arguments
  • Creating cohesive, user-friendly functionality
  • Handling data limitations
  • Developing dynamic reporting capabilities

Next Steps

  • Build in more functionality and analysis features
  • Expand dynamic reporting capabilities
  • Test with additional datasets (as available)
  • Obtain feedback

Desired Feedback

  • Functionality: Are there any features you think would be useful to add?
  • Reporting: Are there any challenges from states that this package could better support?
  • Datasets: Do you know of any datasets that can be used to test and validate the package?