How do you know if your after school program is meeting the needs of the youth and community it serves? Like the students it serves, you measure its strengths and challenges using a variety of tests designed to show you where your program is succeeding and where you need to improve.
Before you measure, however, you must clearly define your goals and set a benchmark for each, so you have something against which to measure.
While many programs focus on the changes in school test scores and crime rates, you need to consider a broader range of desired outcomes, especially in the areas of social and emotional development.
This variety of outcomes steers the goals you set, the way you structure your program, and how you measure the success of your childcare service. Every evaluation method has multiple measures, and you must find the ones pertinent to your program and community.
Key performance indicators, or KPIs, are metrics used to evaluate the success of a given process or program. One example of a KPI measures homework completion rates; another is school attendance. Both of these are reported as a volume or percentage.
Other KPIs are reported along a spectrum or range. Many surveys designed to provide feedback ask participants to place their answers on a scale such as from "strongly agree" to "strongly disagree." Other evaluations ask observers to provide a number from one to ten indicating how often a measurement is met.
After school programs are assessed with a variety of methods providing reports with exact numbers as well as more subjective evaluations. While gathering data on staff attendance is easily measured with a time clock application, determining how effectively the staff supports youth is a bit more subjective.
Your organization must commit to collecting data as a routine part of operations to generate accurate and helpful metrics.
|
Example: Activity Structure and Management by Staff 1. Does staff arrive on time? OR What is the on-time arrival rate of the staff? Yes No Percentage 2. How well does the staff handle disputes among the students on a scale of 1-5? 3. Do daily tasks address the stated goals? Yes No
Examination of Relationships Between Youth and Staff 1. How responsive are the youth to the staff? Very Responsive Moderately Responsive Little to No Response 2. How effective is the staff in providing support to youth while they are accomplishing tasks? Very Effective Moderately Effective Not Effective 3. What is the emotional tenor of the relationship between youth and staff on a scale of 1-5?
Examine an Activity’s Level of Challenge 1. Do the students report they are challenged? Yes No Sometimes 2. Does the staff monitor a youth’s level of frustration and provide clear direction? Yes No Sometimes |
Numerous tools and instruments are available from a wide variety of associations, groups, and retailers that specialize in childhood development and childcare programming. A few of these tests are described below.
This tool is used in site visits to collect observational data. It measures the quality of interactions, opportunities, and resources at a particular program. An observer records indicator data during a sequence of five-minute observations, including:
The After School Environment Scale contains 36 items to measure three main components of an after school program’s environment as it is reported by the youth participants. It is a survey type of tool in which the students report on a frequency scale that ranges from “Never” to “Always” in answer to questions such as:
Through the reported reactions your program can judge the enthusiasm for and perceptions of the provided emotional support, the students’ belief in their autonomy and privacy, and the opportunity for peer affiliations.
From the Intercultural Center for Research in Education and the National Institute on Out-of-School Time (2005)
The APT-R measures the quality of an after school program by observing the overall program, each activity, and homework assistance and completion. It reacts to statements such as:
Most items are rated on a scale of one to four that ranges from “not true” to “very true.”
Observations on 26 different elements are used to rate a single staff member on his or her interaction quality with children. Sample elements include:
The elements are scored from 1 (never true) to 4 (often observed).
The NAA, a membership association for professionals who work with children in school and community settings, developed standards in the following care areas:
Assessment is according to the degree to which the program meets various quality standards, such as:
Observational responses are provided on a scale from zero (no evidence or not met) to 3 (fully met).
Once the observations have been recorded, the results are placed into a report. If an outside agency performed the assessment, that agency should provide an expert to help you translate the various answers into a format that shows clearly where your program is operating to standard and where it falls short.
Take the areas where a weakness is shown and clearly define each problem. Perform a root-cause analysis to determine exactly what is causing the issue and come up with a resolution to put into place to correct the deficiency. All stakeholders (program staff, a board of directors, school staff, and members of the community) should help formulate potential solutions.
After school programs provide wide-ranging benefits to the youth it serves and the community itself. In some areas, it is the only place for children to safely stay where they can be assisted with schoolwork and provided with real-world applications of what they learned.
You want to know that your program is delivering the level of quality the community, youth, and parents expect. If one area is deficient, the youth are not receiving the full benefits of the program. Use the results of the assessments and reports to optimize your after school program.