A.
Definition
1.
Scoring
‘A score is a number derived
from a measure...’ (Anderson in Sulistyo, 2007). In testing context, this follows
then that scoring is a process of utilizing a number to represent the responses
made by the test taker.
2.
Grading
Grading
rubrics are an effective tool for providing students with your expectations on
a given task as well as providing you with a standardized grading mechanism.
Rubrics identify the important traits (those to which you assign a score or
deduct points) of a task and articulate performance levels. Rubrics attempt to
make explicit the implicit criteria by which you grade or assess student performance.
Typically,
grading rubrics are created to correspond to an instructor’s student learning
objectives. In addition to providing students with clear expectations, a rubric
is a valuable assessment tool for faculty wishing to demonstrate direct evidence
of students learning.
A
grading rubric should be created with the following items:
ð
Specific qualities (or traits) to be assessed
ð
Standards or levels of performance
ð
Specific indicators describing what the various levels of
performance look like for each of the qualities being assessed
Be
cautious not to make the indicators so broad that they are non-descriptive;
they should be as focused as a student learning objective.
Barbara Walvoord and Virginia Anderson’s book, Effective
Grading: A Tool for Learning and Assessment has a wealth of information on
using rubrics to facilitate the grading process and as a formative assessment
tool for both the faculty member and the student.
3.
Rubric
A rubric
is a scoring guide that’s intended to help those who must score students’
responses to constructed-response items. The most important component in any
rubric is the rubric’s evaluative criteria, which are the factors a
scorer considers when determining the quality of a student’s response. For
example, one evaluative criterion in an essay-scoring rubric might be the
organization of the essay. Evaluative criteria are truly the guts of any rubric,
because they lay out what it is that distinguishes students’ winning
responses from their weak ones. A second key ingredient in a rubric is a set of quality
definitions that accompany each evaluative criterion. These quality
definitions spell out what is needed, with respect to each evaluative criterion,
for a student’s response to receive a high rating versus a low rating on that
criterion. A quality definition for an evaluative criterion like “essay
organization” might identify the types of organizational structures that are
acceptable and those that aren’t. The mission of a rubric’s quality definitions
is to reduce the likelihood that the rubric’s evaluative criteria will be
misunderstood.
Finally,
a rubric should set forth a scoring strategy that indicates how a scorer
should use the evaluative criteria and their quality definitions. There are two
main contenders, as far as scoring strategy is concerned. A holistic scoring
strategy signifies that the scorer must attend to how well a student’s response
satisfies all the evaluative criteria in the interest of forming a general,
overall evaluation of the response based on all criteria considered in concert.
In contrast, an analytic approach to scoring requires a scorer to make a
criterion-by criterion judgment for each
of the evaluative criteria, and then amalgamate those per-criterion ratings into
a final score (this is often done via a set of predetermined, possibly
numerical, rules).
B.
Types
1.
Dichotomous
Dichotomous scoring entails
viewing and treating the response as either one of two distinct, exclusive
categories. The number utilized in this kind of scoring is 0 (zero) and 1
(one).
2.
Continuous
Continuous scoring views and
treat the test taker’s response as being grade in nature. The test taker’s
response is considered as having a gradation or degree in it.
C. Strategy in Scoring
1.
Holistic
Holistic scoring considers the
test takers’ response as a whole totality rather than as consisting of
fragmented parts. In holistic scoring, the focus is laid on the global quality
of the test taker’s response. The benefits of this scoring are quick marking,
relatively high inter-rater consistency, easy interpretation due to use of
standards, showing the strength of the test takers, and a wide spectrum of use
among disciplines. The challenges are absence of diagnostic power, concealing
differences across sub-skills within each score, genre-limited application,
extensive training on the part of raters.
2.
Primary
Trait
Primary trait scoring focuses
on one specific type of features or traits that test takers need to
demonstrate. The key of primary trait is specificity of discourse to be
exhibited by the test takers.
3.
Analytic
The analytic scoring emphasizes on individual points or
components of the test taker’s response. In an analytic scoring plan,
linguistic and non linguistic features considered important in a discourse
become the building blocks of the scheme, and these features are commonly
evaluated separately as individual components.
D.
Component of
Rubric
A
scale of points to be assigned in scoring a piece of work on a continuum
of quality. High numbers are typically assigned to the best work.
1.
Descriptors for each level of performance that
contain criteria and standards by which the performance will be judged
2.
Indicators
are often used in descriptors to provide examples or signs of performance in each
level
3.
Criteria that describe the conditions that any
performance must meet to be successful
Five
categories to consider:
ð Impact – the
success of performance, given the purposes, goals and desired results
ð Work
quality/Craftsmanship – the overall polish, organization, and rigor of the work
ð Methods – the
quality of the procedures and manner of presentation, prior to and during
performance
ð Content – the
correctness of the ideas, skills or materials used
ð Sophistication
of the performance – the relative complexity or maturity of the knowledge used.
Should describe both strengths and errors (errors should be described
particularly in lower levels of performance)
4.
Standards that specify how well criteria must be
met
E.
Steps in
Designing Rubric
A
step-by-step process for designing scoring rubrics for classroom use is
presented below.
Information
for these procedures was compiled from various sources (Airasian, 2000 &
2001;
Mertler,
2001; Montgomery, 2001; Nitko, 2001; Tombari &Borich, 1999). The steps will
be
summarized
and discussed, followed by presentations of two sample scoring rubrics.
Step 1: Re-examine the
learning objectives to be addressed by the task. This allows
you tomatch your scoring guide with your objectives and actual instruction.
Step 2: Identify specific
observable attributes that you want to see (as well as those youdon’t want to
see) your students demonstrate in their product, process, orperformance. Specify the
characteristics, skills, or behaviors that you will belooking for, as well as
common mistakes you do not want to see.
Step 3: Brainstorm
characteristics that describe each attribute. Identify ways todescribe above
average, average, and below average performance for eachobservable attribute
identified in Step 2.
Step 4a: For holistic
rubrics, write thorough narrative descriptions for excellent workand poor work
incorporating each attribute into the description. Describe
thehighest and lowest levels of performance combining the descriptors for
allattributes.
Step 4b: For analytic
rubrics, write thorough narrative descriptions for excellent workand poor work
for each individual attribute. Describe the highest and lowestlevels
of performance using the descriptors for each attribute separately.
Step 5a: For holistic
rubrics, complete the rubric by describing other levels on thecontinuum that
ranges from excellent to poor work for the collectiveattributes. Write
descriptions for all intermediate levels of performance.
Step 5b: For analytic
rubrics, complete the rubric by describing other levels on the continuum that
ranges from excellent to poor work for each attribute.
Write descriptions for all intermediate levels of performance for each
attribute separately.
Step 6: Collect samples
of student work that exemplify each level. These will help you in designing scoring rubrics for your classroom.
Step 7: Revise the
rubric, as necessary.
Be prepared to reflect on the effectiveness of the rubric and revise it prior to
its next implementation.
F.
Criteria of A
Good Rubric
1.
Are
sufficiently generic to relate to general goals beyond an individual performance
task but specific enough to enable useful and sound inferences on the task. The
word “rubric” derives from the Latin word for “red.” It was once used to
signify the highlights of a legal decision as well as the directions for
conducting religious services, found in the margins of liturgical books—both
written in red.
2.
Discriminate
among performances validly, not arbitrarily—by the central features of
performance, not by the easiest to see, count, or score.
3.
Do
not combine independent criteria in one rubric.
4.
Are
based on analysis of many work samples, and based on the widest possible range
of work samples—including valid exemplars.
5.
Rely
on descriptive language—what quality, or its absence, looks like—as opposed to
relying heavily on mere comparatives or value language (e.g. “not as thorough
as,” or “excellent product”) to make the discrimination.
6.
Provide
useful and apt discrimination to enable sufficiently fine judgments—but not
using so many points on the scale as to threaten reliability (typically
involving, therefore, 6-12 points on a scale).
7.
Use
descriptors that are sufficiently rich to enable student performers to verify
their score, accurately self-assess, and self-correct.
8.
Highlight
the judging of the “impact” of performance—the effect, given the purpose—as
opposed to over-rewarding merely the processes, the formats, or the content
used; and/or the good faith effort made.
References:
Airasian,
P. W. (2000). Assessment in the classroom: A concise approach (2nd ed.).
Boston: McGraw-Hill.
Airasian,
P. W. (2001). Classroom assessment: Concepts and applications (4th ed.).
Boston: McGraw-Hill.
Mertler,
C. A. (2001). Using performance assessment in your classroom. Unpublished
manuscript, Bowling Green State University.
Montgomery,
K. (2001). Authentic assessment: A guide for elementary teachers. New
York: Longman.
Nitko,
A. J. (2001). Educational assessment of students (3rd ed.). Upper Saddle
River, NJ: Merrill.
Tombari,
M. & Borich, G. (1999).Authentic assessment in the classroom:
Applications and practice. Upper Saddle River, NJ: Merrill.
