Student Impact Evaluation: Methodology

The Student Impact Evaluation is Rustic Pathways’s multi-phase measurement system for tracking student growth before, during and after every program. It combines pre- and post-program surveys, long-term alumni tracking, daily reflection tools, leader observations, parent feedback, and focus groups, all operated under IRB-approved research standards.

This page documents the impact report methodology. For the learning outcomes the evaluation measures and the results from recent program years, see 10 Student Learning Outcomes.

Disclosure: Rustic Pathways developed and operates this methodology in-house, with input from PhD candidates and experts in social-emotional learning and youth development. The same methodology is applied across all Rustic Pathways programs.

What is the Rustic Pathways Student Impact Evaluation?

The Student Impact Evaluation (SIE) is the measurement system Rustic Pathways uses to track student growth across its ten student learning outcomes. It collects data at four points in a student’s program lifecycle and pairs that data with complementary inputs from program leaders, parents, and alumni focus groups.

The four collection points are:

1. Before the program — baseline assessment
2. During the program — daily reflection and leader observation
3. Immediately after the program — post-program evaluation
4. 6–9 months post-program — long-term impact study

The SIE is one of three measurement systems Rustic Pathways operates. The other two are client satisfaction surveys and community impact reports.

Why does Rustic Pathways evaluate student impact?

When students grow more confident, globally aware, and self-directed, those changes ripple forward into college, careers, and their lifelong potential to lead and serve.

Rustic Pathways evaluates student impact to ensure every program delivers measurable, lasting growth in students’ skills, mindsets, and worldviews. These data-driven insights guide program refinement, curriculum design, and staff training, so transformation is not a promise but a measured outcome.

See recent results from the SIE

What makes this evaluation different from a post-trip survey?

Unlike a standard post-trip survey, the Student Impact Evaluation:

• Measures change, not just satisfaction. Pre-program baselines are compared to post-program responses to quantify growth.
• Tracks sustained impact. Alumni are re-surveyed 6–9 months later to measure which changes persist.
• Uses multiple data sources. Student self-report is paired with leader observations, parent feedback, and focus groups.
• Follows research-grade methodology. Instruments are developed by academic researchers and the protocol is IRB-approved.

The result is a system that measures growth that is academic, practical, and interpersonal, not feel-good testimonials.

How does Rustic Pathways measure student growth?

The evaluation has five components.

1. Pre-Program Assessment

What we collect: Baseline surveys completed by students before they travel.

What it measures: Confidence, leadership, independence, and cultural awareness.

Why it matters: Without a baseline, post-program responses cannot be interpreted as growth. The pre-program assessment establishes the comparison point for every other phase.

2. During-Program Tools

What we collect: Daily and weekly inputs throughout the program.

Tools used:

• Daily reflection journals with guided prompts
• Peer feedback sessions
• Leader observations using standardized rubrics
• Photo narratives and visual storytelling
• Debriefs after meaningful challenges

Why it matters: Capturing students’ experience in real time produces richer qualitative data than a single post-trip survey, and gives program leaders signal to adjust facilitation mid-program.

3. Post-Program Evaluation

What we collect: Validated assessments completed immediately after the program ends.

What it measures: Growth across all 10 Student Learning Outcomes.

Why it matters: The post-program assessment is the primary growth measurement, comparing student responses against their own pre-program baselines. These assessments are developed by academic researchers to ensure reliability.

4. Long-Term Impact Study

What we collect: Alumni surveys conducted 6–9 months after a student returns home.

What it measures: Which growth in learning outcomes persists beyond the immediate post-program window.

Why it matters: Short-term post-program responses can reflect emotional intensity rather than lasting change. The delayed survey distinguishes durable growth from temporary reaction and informs long-term student success strategies.

5. Complementary Data Collection

What we collect: Qualitative and external data points that round out student self-report.

Sources:

• Focus Groups — deep-dive student conversations
• Leader Interviews — frontline insights from program staff
• Parent Surveys — family-observed changes after a student returns home
• Alumni Tracking — long-term academic and career outcomes

Why it matters: Self-report alone cannot triangulate change. Combining student, leader, and parent perspectives produces a more accurate picture of program impact.

What research standards does this methodology follow?

The Student Impact Evaluation is developed and operated under three standards.

IRB-Approved

Research Protocol

PhD Researchers

Academic Oversight

Standardized Instruments

Replicable Measurement

• IRB approval: The research protocol is reviewed and approved through an Institutional Review Board, ensuring participant protections and methodological soundness.
• Academic researchers: The evaluation team includes PhD candidates and experts in social-emotional learning and youth development. The methodology has been validated by educational institutions.
• Standardized instruments: Post-program assessments are validated tools developed by academic researchers, not custom marketing surveys. Leader observations use standardized rubrics rather than open-ended notes.

What survey instruments does the evaluation use?

Survey items are calibrated to the 10 Student Learning Outcomes. Students rate their agreement with each statement before and after the program. Growth is measured by comparing pre- and post-program responses.

Sample survey items:

• “I regularly think about my passions and what drives my life.”
• “When I visit a new place, I find it easy to fit in.”
• “I often volunteer to help people who look lost.”

What focus group questions does the evaluation use?

Focus group prompts are designed to elicit qualitative insights that quantitative surveys cannot capture. They complement survey data with student stories, surfacing themes that inform program design.

Sample focus group questions:

• “Tell us about a moment that changed your perspective.”
• “Describe a meaningful interaction with a local family or partner.”
• “What was your biggest challenge, and how did you respond?”

Measured impact snapshot

A sample of impact metrics observed across Rustic Pathways programs:

How is evaluation data used to improve programs?

Evaluation results feed back into four program improvement processes:

• Curriculum refinement based on skill development trends across cohorts
• Program Leader training so leaders can better facilitate the growth the data shows students need
• New reflection tools aligned to emerging needs identified in the data
• Improved pre-departure preparation for student success

Each year, Rustic Pathways selects one to three Student Learning Outcomes as focused improvement targets based on the prior year’s evaluation results.

What does this evaluation not measure?

The Student Impact Evaluation focuses on student growth in the 10 Student Learning Outcomes. It does not measure:

• Client satisfaction — tracked through separate client satisfaction surveys
• Community impact — tracked through separate community impact reports
• Absolute student ability — the evaluation measures change from a student’s own baseline, not where students start relative to one another

How can I contact the evaluation team or opt my student out?

For questions about the Student Impact Evaluation methodology, or to opt your student out of the evaluation, contact:

studentimpact@rusticpathways.com

Opt-out requests do not affect a student’s participation in or experience on a Rustic Pathways program.