Agile Development to Support State Assessment Analyses

Intern Update #3

Brian Harrold
July 25, 2025

cohortED package

cohortED badge

In the last two weeks

  • Refined primary report (State Level Mobility Report)
  • Expanded package functionality
  • Developed new report (Grade-Based Performance Report)
  • Analyzed real data
  • Drafted NCME 2026 Proposals

Refinements

  • Improved narrative summaries
  • Enhanced visualizations and tables
  • Updated modular report structure
  • Strengthened logic and data handling

Refined Watch Gears Image

Primary Report

View Example Reports

Package Improvements

  • Integrated grade change analysis
  • Enhanced participation visualizations
  • Developed additional performance tools
  • Expanded narrative reporting
  • Increase parameter flexibility

Image of Computer Code

Developed Grade-Based Performance Report

  • Focuses on a single grade level across multiple years
  • Utilizes adjacent grade levels for cohort analysis
  • Describes participation, performance, and persistence
  • Supports reproducible, customized reporting with minimal code changes

Working towards reproducibility

  • Require fields:
    • (Unique) Student ID
    • (Academic OR Numeric) YEAR
    • GRADE
  • For the report:
    • Differences in sgpData and real data
    • Focusing on target_grade (and appropriate subset)

YAML Parameters

params:
  content_area: "Mathematics"
  target_grade: 5
  grades: [3, 4, 5, 6, 7]
  dataset: "math"
  proficiency_levels:
    - Proficient
    - Advanced
  advanced_levels:
    - Advanced
  achievement: "ACHIEVEMENT_LEVEL"
  output_file: "cohort_report_basic.html"

🔁 Change these values to generate a custom report.

✨ A New Report

✨
🌟
💫
✨
  • Early version of report
  • Built using real performance data
  • Explores grade-level achievement trends
  • Fully reproducible and flexible
  • Expands the scope of the cohortED package

Performance Report (with sgpData)

📄 View Example Reports

Working with Real Data

  • Change in variables and variable names
  • Handle non-unique student identifiers
    • New students vs. Existing students
    • Based on same test, same grade, same year
    • Found a potential solution
  • Separate re-testers [only affected 9-12]

Reporting with Real Data

See other reports

Not embedded for confidentiality reasons

Internship Highlights

  • Developed and documented the cohortED for R
  • Version control and website hosting through GitHub
  • Built automated, reproducible reports using Quarto
  • Applied workflows to real data
  • Improved reproducibility and flexibility

I’m excited about the potential of this work!

Desired Feedback

  • Usefulness: Are the reports and visuals informative and actionable?
  • Conceptual Fit: How should grade-level trends complement cohort analysis?
  • Potential: In what ways do you see this package being useful?
  • Future Direction: How best to turn this into a blog?
  • Writing Support: Suggestions on the NCME Innovation Demonstration draft?
    (I’d be happy to share it.)

Brian Harrold

🎉 Thank You!

It’s been a privilege to learn and grow with the Center for Assessment.

💬 Let’s stay connected
📧 bharrold@email.sc.edu
🔗 linkedin.com/in/brianharrold6

📦 Project Site: stat-brain.github.io/cohortED/

Center for Assessment logo