Quality improvement has been a watchword for most industries over the past 30 years. Today’s need to measure the success of and improve your after school program using data and evidenced-based improvements is an outgrowth of those programs, which have begun to mature.
Feedback and reiterative improvement systems to increase program quality have become major building blocks of the quality movement. Relationship building and improvement in communications have become additional tools in creating successful after school programs.
Build in quality from the beginning
As you put an after school program in place, there are several steps you can take to start off on the right foot and begin to build rapport and communication with the principal and school day teachers.
First, establish policies to help set expectations and provide a framework for the program’s introduction and growth in the future. Let the school and parents know:
- When students can add or drop from the program
- How refunds are handled
- Deadlines for registration, refunds, and cancellations
Connect with the principal or head of the school where the program is located. Have an introductory meeting to begin building your relationship and set up regular meetings going forward. Send a brief meeting report to the principal afterward.
Develop a formal agreement, often call a Memo-of-Understanding or MOU, to address the roles and responsibilities of the school staff and get it signed by the principal and the after school program director if that is not you. The MOU describes:
- Expected achievement outcomes
- Timeline for meetings with dates
- Specific roles and responsibilities of key individuals
- The time and financial resources required
- All information about sharing resources, space, and materials
At mid-year, review the MOU and make any needed changes. During the first half of the school year, you need to document challenges and successes to present at future meetings.
Develop and maintain relationships by joining school committees with the intent on highlighting how the teachers and the program can serve each other to improve the effectiveness and outcomes of the program. Come to the meetings prepared with a progress report and ideas for working together.
Learn about and support school events by having your students:
- Participate
- Attend
- Support
…the events as appropriate.
Communicate clearly and often to school day teachers, school staff, and after school program staff as well as parents about the policies and practices of the program. Create a guide for families and one for after school staff with critical information pertaining to their participation. Update it and provide a new one each year.
Visit classrooms and learn curricular agendas so you can create complementary activities for after school. Create student portfolios to show parents and teachers what the students are doing in the program. Support continuing education for your staff and invite teachers to workshop with staff or suggest educational opportunities to learn more about specific subjects.
Data collection and analysis
The most common data for after school program performance is observational data, which can be both qualitative and quantitative. Use this data to focus your priorities when you are creating a training agenda for your staff.
Data quality review
Before analysis, review the data for completeness and accuracy during collection to ensure quality data is used for the analysis. If you select random observation tools as they are completed, you can reduce delays or problems that could disrupt regular measurement and study.
Check at least 20% of each batch of observation tools for clarity, completeness, and to make sure they reflect the data you need and want to capture.
Data organization
Organize paper copies or create an electronic database where the data can be entered directly. A database is easier to maintain and store than a paper file. In addition, the data are easily manipulated to create calculations, charts, and reports.
Comparison to benchmarks and goals
Review the data to determine if your program is meeting benchmarks and achieving the goals that were set. Check the details of each goal and how well the overall program is meeting goals.
Example Goal: Staff will read to students for a minimum of 15 minutes daily.
Data analysis
Look at:
- Range of compliance numbers for each goal or practice observed
- Average amount or number of times the goals were hit or missed
- Percent of the observations where the goal was achieved
- Watch for trends
You need at least three data points to define a trend. If you see the issue more than three times on three different occasions and three different individuals also report observing the issue, you can call it a trend. Being conservative and careful about trending data keeps you from wasting energy investigating and resolving non-existent issues.
Counts
Build a tallying system around your benchmarks, prioritizing the components in the order of their importance. As more data is acquired, you can refine your observational questions to look more deeply into your practices and processes.
Learn the context of an observation in comparison to a benchmark. Inquire when something appears different. Specify the focus, such as onto a single lesson. Validate that you measure the right concept.
Before creating changes, make sure you are seeing a true trend, pattern, or theme developing and define the scope of it. Use the data to focus future training efforts and to justify your training agenda. The presentation should be informative and help to build a complex picture of your after school programming.
Improvement cycle
Once you have analyzed your data and verified trends, prioritize the programming gaps and staff needed to address the issues. Integrate the instructional strategies, classroom practices, and innovation into the training by:
- Phrasing the trends as action statements
- Outlining activities
- Illustrating how innovations will be used
Get feedback on the training to improve for the next session. Also, use your data and any feedback on the program itself to build or adjust the performance goals for the program and how you will meet them. Thus begins the next iteration of quality improvement.
Parents want to know their children are getting the support they need for homework and that your program provides quality educational support. Teaching staff and principals want to know you are complementing their curriculum and events to deepen the educational advantage while exposing the children to activities the school day does not provide.
Parent and teachers need to see you setting goals, measuring how well you and your after school program are meeting them, and making improvements in areas where you are not. Use these data collection and analysis practices to begin measuring your success and improve your after school program.
