|
The CERES program was developed in California public schools in response to community and employer concerns that too many students were completing their education without a positive orientation to the world-of-work. Many could not demonstrate minimum competency in basic reading, writing, math, listening, and speaking, as well as academic thinking, problem solving, and comprehension, and did not demonstrate skills related to working on teams or serving customers. CERES was developed to provide students with opportunities to acquire workplace skills. The idea behind this project was that students would meet four main objectives such as being able to identify and acquire positive work habits, acquire knowledge of economic/free enterprise principles, acquire knowledge of diverse occupations including job training preparation and job duties, and apply basic skills to career decisions and actions, including job-seeking and job-retention.
The CERES program is widely used throughout the U.S. and Puerto Rico. A review over a five-year period (1993-97) showed that the program has been adopted by approximately 800 public and 35 non-public schools. In an attempt to evaluate the results of the CERES program, seven program schools and four control schools were studied over a two-year period. The results indicated that students at all grade levels performed significantly better than students in comparable test sites lacking the program. Although the evaluation did provide support for the use of the CERES program, there are still many questions that could be asked in evaluating a career curriculum. This article will provide a comprehensive model for evaluating the CERES program.
As mentioned above, several questions about the use of the CERES program remain unanswered. The first of these is the degree to which teachers are integrating CERES materials into the classrooms. If the program has been properly implemented, it is desirable to see if it effectively attains its objectives. If the objectives are being met the next question is whether those objectives are attaining the overall goal of having high school graduates that are more familiar with career choices and who are better prepared for the next step in their career, whether that is a job or further education. If the program is successful in attaining its overall desired goal, a final question is whether there are any side effects. Side effects could be higher levels of self-efficacy or an over burdened curriculum that takes away from other required lessons. Answering these questions will provide guidance to improving the CERES program, as well as overall information about the outcomes of the program on post-high school success.
In evaluating the CERES program, schools will be used as units of analysis. It is proposed here for discussion purposes that 15 schools that have adopted the CERES program and 100 schools that have not adopted the program throughout California will be used as the sample.
The first portion of the evaluation involves the extent to which the CERES program is being implemented and used in school curriculum.A qualitative evaluation using an interview technique where teachers are directly asked specific questions about how they use the program in their classroom will be conducted. The drawback to a qualitative evaluation is that it takes up time and resources. To solve this problem, only 5 of the 15 schools will be randomly selected for this part of the evaluation. A comparison of the two groups will be made to see if there are major differences. Within each of the five schools, two teachers will be selected randomly from each grade and will each participate in 30-minute individual interviews with the evaluator. Specifically, the schools will be compared on the outcome variables (discussed later in this article) to see if there are significant differences.
The second portion of the evaluation will answer the question of whether the program is meeting its objectives. The CERES program has eight well-defined objectives:
|
1. Self-awareness referring to the ability to acquire knowledge about one’s self-physical characteristics, interests, aptitudes, abilities and attitudes.
2. Attitude development is defined as the development of positive attitudes toward self and others, learning, working, leisure and individual and cultural differences.
3. Career awareness and exploration enhances the development of awareness of career options and opportunities and the ability to explore them in relation to interests, aptitudes and values.
4. Consumer competencies refer to the ability to acquire knowledge and skills to make wise consumer decisions.
5. Educational awareness is defined by the ability to recognize that all educational experiences are integrated with total career development and preparation.
6. Economic awareness is the ability to develop an awareness of theeconomic system.
7. Career planning and decision-making is the ability to acquire decision-making skills and apply them to personal career development.
8. Career preparation is the ability to develop competencies needed for entry into the next educational or occupational level.
|
The extent to which the CERES program meets each of these objectives will be tested by using all 15 randomly selected schools that have used the CERES program. A questionnaire will be passed out to students from random classes throughout the schools to measure the extent to which they perceive they have acquired each of the eight abilities. The questionnaire will ask the students to indicate the extent to which they acquired each ability on a scale of one to five (1 = not at all, 5 = to a great extent). Demographic information about the student’s grade level and age will also be collected.
The third portion of this evaluation will determine the extent to which the CERES program has accomplished its ultimate goal of having students graduate from high school who are prepared to enter the world of work or continue in their education.
In an effort to further evaluate CERES this proposal recommends that a random comparison group design be used. In this particular evaluation the treatment group will be schools that have implemented the program and the control group will be schools that have not.
The criterion group is used in the first step of analysis by estimating counterfactual weights for each variable thought to affect the outcome measure other than the program. A regression analysis will be used to examine the four outcome variables.
The Four Outcome Variables Are:
1. acquire positive work habits
2. show higher knowledge of economic/free enterprise principles
3. acquire higher knowledge of diverse occupations
4. apply basic skills to career decisions and actions
|
Research reviewed for this proposed evaluation indicates the possible moderator variables are: average socioeconomic status, size of school, teacher to student ratio, and location of school. Once the estimates for the betas in the regression are calculated, these estimates will be applied to the mean scores for the experimental group on all the possible confounding variables.
For the present proposed evaluation, twelfth grade students at both schools using CERES and those that are not will complete a questionnaire designed to measure vocational readiness. The items ask participants to rank how they feel on a scale of 1 to 5 (1 = very inaccurate, 5 = very accurate). In addition to the self-report questionnaire, archival data from the same schools will be collected from RAND California (http://ca.rand.org) and Internet sites that rate high schools on several different categories. These outcome variables will also be used in the random comparison regression equation to control for the same confounding variables discussed previously.
These data will be analyzed by comparing the expected outcomes to the actual outcomes. To control for possible Type-1 error a conservative estimate will be collected. This will be done because schools that have decided to implement the CERES program could be inherently different than schools that have not implemented the program. The final effect sizes for each of the outcome variables will be calculated based on these conservative estimates.
The final question this evaluation will address is in regard to any side effects the CERES program might have. These data will be collected through phone interviews with vocational and career guidance counselors who will be asked to describe any unforeseen side effects both positive and negative.
The means for the eight items on the survey will be compiled to measure the extent to which the program is meeting each of its objectives. Demographic characteristics will be used to control for moderator effects.
This article describes the suggested specific steps to be taken in performing an evaluation that tests the implement-ation, the theory, the results and the side effects of the CERES program. This thorough evaluation will provide additional information about the usefulness of the program and will help educators decide if the CERES program can add to the K through 12 curriculum. If your school curriculum uses the CERES program or would like to and would be willing to participate in an evaluation study such as that described here please contact EdITS.
References
Dee, T. S., (2002). Standards and Student Outcomes: Lessons from the “First Wave” of Education Reform. Cambridge, MA: Kennedy School of Government.
Farris State University Career Institute for Education and Workforce Development, (2002). Students embarrassed by vocational education. Career Tech Update, 3, 1-2.
Maxwell, N. L., & Rubin, V. (2001). Career Academy Programs in California: Outcomes and Implementation. Berkeley, CA: California Policy Research Center.
Project CERES. (2002). A comprehensive K-12 school to career education program. [Brochure]. San Diego, CA: EdITS.
Rand California, An Online Source for California and U.S. Statistics (n.d.) Retrieved June 17, 2003, from http://ca.rand.org.
Schwager, M. T., (2001). Building Foundations for Academic Success in School-to-Career Systems.
San Francisco, CA: West Ed.
-
|