Educational evaluation is the process of making judgments
about the merit, value, or worth of any aspect of an educational program. Eight
such aspects are described below that help to describe the range of influence
evaluation research has in the field of education.
Instructional Program
Evaluation Instructional programs are often evaluated to determine whether
they were implemented as intended and to asses their effectiveness in promoting
student learning.
Instructional Methods
Evaluation Instructional methods are usually evaluated to determine which
of a variety of teaching methods is most effective in teaching particular
subject matter. For example, lecture versus discussion method of teaching,
whole-language approach to teaching reading versus the phonics method, etc.
Instructional Materials
Evaluation - This type of evaluation relates to how well the instructional
materials (text books, workbooks, study guides, etc.) contribute to student
learning. One of the most serious faults with current instructional materials
is that their development is usually far removed from the classroom teacher and
the operational learning situation. Of particular interest when evaluating
instructional materials is the degree of relationship between the instructional
materials and the instructional goals of the program itself. In addition to
measuring the effectiveness of instructional materials, an evaluation should
also consider the ease with which they can be adopted into the curriculum as
well as the cost of adopting the materials.
Evaluation of Specific Groups
or Organizations Any educational group or organization can be evaluated
and its operation compared to its own organizational goals or to the operations
of other, similar organizations.
Evaluation of Educators
Teachers must take tests or otherwise demonstrate certain competencies to obtain
and retain certification or employment. Performance appraisals also routinely
occur in relation to salary reviews and promotions. These are all forms of
evaluation that focus on a persons merits in comparison to other persons
(normative) or to some standard (objective). Educators who make decisions about
educational programs, materials, and methods also need general information about
the qualifications of the educators who will administer such programs,
materials, and methods.
Evaluation of Students
Students are assessed regularly to guide decisions about their academic
performance, leading to assignment of school grades and/or granting of diplomas
or degrees. Except for standardized assessment instruments developed by
national testing organizations, such as the Educational Testing Service,
teachers are usually given the responsibility for designing, administering, and
scoring many of these tests. Due to its commonality, student evaluation is not
typically viewed as a form of evaluation research. But it is, in fact, a very
crucial one.
Evaluation of Tests and
Assessment Procedures Tests used to evaluate student performance must
themselves be evaluated as to their validity, reliability, objectivity,
fairness, and other relevant factors. Chapter 5 of the course textbook or
the Week 1 lesson on the research web site describe procedures used to assess
the validity and reliability of academic tests. Many software packages are
on the market that are designed to assess validity and reliability of academic
tests and a variety of related psychometric parameters of the test and the items
that make up the test.
Evaluation of Evaluations
(also known as Meta-Evaluation) It is critically important that evaluation
studies meet quality standards. Such determinations are made by rigorously
evaluating evaluation studies (known as a meta-evaluation).
Forms of Evaluation
Needs Assessment
Evaluation of discrepancies between the existing situation and the desired
state of affairs is also know by the name Needs Assessment. Another definition
of needs assessment is the process of identifying needs and deciding upon
priorities among them. In the context of an education and training program, we
can define a need as a situation in which there is a discrepancy between an
acceptable state of affairs and an observed state of affairs.
The extent of the discrepancy may
be measured either objectively or subjectively. An objective needs assessment
usually involves the following stages:
1.
Identifying the goals considered important to the educational system.
2.
Selecting or developing measures for those goals.
3.
Setting acceptable levels on the measures.
4.
Administering the measures.
5.
Comparing obtained levels against acceptable levels (gap analysis). If
a gap is found, then a need is identified.
A subjective needs assessment
usually involves the following stages:
1.
Identifying the goals considered important to the educational system.
2.
Selecting or developing measures for those goals (This stage is
optional)
3.
Developing a rating scale for judging the degree to which present
performance in a goal area is acceptable. If stage 2 is used, judges can be
given summaries of data collected to aid them in making their judgments.
4.
Obtaining ratings from a group of judges and averaging (and
rank-ordering) the ratings to identify the number and relative importance of
needs.
Its possible to have various
combinations of objective and subjective needs assessment. For example, judges
may be asked to consider formal evidence (objective) as well as to provide their
professional opinions or to seek opinions of colleagues, student comments, etc.
(subjective).
Regardless of the method used
(purely objective, purely subjective, or a combination of both), the next step
in the needs assessment process is to determine the relative priorities among
the various needs so that a program of action can be developed. Factors to be
considered in determining priorities are such things as:
- The perceived importance of each goal area where a
need has been identified
- The number of persons likely to be affected by the
need and the intensity of the need
- The feasibility (including resource costs) of
implementing a program to eliminate the need
Objectives-Based Evaluation
Objectives-based curriculum
evaluation Many research studies dealing with new instructional programs,
which typically involve the use of experimental designs, use quantitative
(objectives-based) evaluation to determine how well subjects in the experimental
group (those who received an innovative instructional program) perform compared
to subjects in the control group (those who received a traditional instructional
program or no program at all).
Cost-Benefit Analysis In
cost analysis studies, evaluators determine how well an educational program
achieves its objectives (benefits) relative to the costs of implementing the
program. The most efficient program is the one that shows the greatest benefit
per unit of cost.
Criteria for Judging Evaluation Research Studies
The Joint Committee on Standards
for Educational Evaluation created a set of standards for judging the adequacy
of evaluation research studies. The committee identified 30 standards for
program evaluation and grouped them under the following four criteria:
Utility This criterion
is designed to ensure that the evaluation study is informative, timely, and
influential.
Feasibility This
criterion is intended to ensure that the evaluation study is realistic, prudent,
diplomatic, and economical.
Propriety This standard
is intended to ensure that the evaluation study is conducted legally, ethically,
and with regard to the welfare of both those being evaluated and those who might
be affected by the results of the evaluation.
Accuracy This standard
is designed to ensure that the evaluation study conveys technically accurate
(and adequate) information about the merit and worth of the program being
evaluated.
