research article critique grading rubric

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.71(4); 2007 Aug 15

A Rubric to Assess Critical Literature Evaluation Skills

To develop and describe the use of a rubric for reinforcing critical literature evaluation skills and assessing journal article critiques presented by pharmacy students during journal club exercises.

A rubric was developed, tested, and revised as needed to guide students in presenting a published study critique during the second through fourth years of a first-professional doctor of pharmacy degree curriculum and to help faculty members assess student performance and provide formative feedback. Through each rubric iteration, the ease of use and clarity for both evaluators and students were determined with modifications made as indicated. Student feedback was obtained after using the rubric for journal article exercises, and interrater reliability of the rubric was determined.

Student feedback regarding rubric use for preparing a clinical study critique was positive across years. Intraclass correlation coefficients were high for each rubric section. The rubric was modified a total of 5 times based upon student feedback and faculty discussions.

A properly designed and tested rubric can be a useful tool for evaluating student performance during a journal article presentation; however, a rubric can take considerable time to develop. A rubric can also be a valuable student learning aid for applying literature evaluation concepts to the critique of a published study.

INTRODUCTION

There has been increased interest over the past decade in using evidence-based medicine (EBM) as a basis for clinical decision making. Introduced in 1992 by the McMaster University-based Evidence-Based Medicine Working Group, EBM has been defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” 1 Current best evidence is disseminated via original contributions to the biomedical literature. However, the medical literature has expanded greatly over time. Medline, a biomedical database, indexes over 5000 biomedical journals and contains more than 15 million records. 2 With this abundance of new medical information, keeping up with the literature and properly utilizing EBM techniques are difficult tasks. A journal club in which a published study is reviewed and critiqued for others can be used to help keep abreast of the literature. A properly designed journal club can also be a useful educational tool to teach and reinforce literature evaluation skills. Three common goals of journal clubs are to teach critical appraisal skills, to have an impact on clinical practice, and to keep up with the current literature. 3 , 4 Journal clubs are a recognized part of many educational experiences for medical and pharmacy students in didactic and experiential settings, as well as for clinicians. Journal clubs have also been described as a means of teaching EBM and critical literature evaluation skills to various types of medical residents.

Cramer described use of a journal club to reinforce and evaluate family medicine residents' understanding and use of EBM concepts. 5 Pre- and posttests were used during each journal club to assess the residents' understanding of key EBM concepts related to the article discussed. Pretest scores improved over the year from 54.5% to 78.9% ( p < 0.001) and posttest scores improved from 63.6% to 81.6% ( p < 0.001), demonstrating the journal club's ability to help residents utilize EBM techniques. Linzer and colleagues compared a journal club to a control seminar series with regard to medical interns' reading habits, epidemiology and biostatistics knowledge, and ability to read and incorporate the medical literature into their practice of medicine. 6 Forty-four interns were randomized to participate in the journal club or a seminar series. After a mean of 5 journal club sessions, 86% of the journal club group improved their reading habits compared to none in the seminar group. Knowledge scores increased more with the journal club and there was a trend toward more knowledge gained with sessions attended. Eighty percent of the journal club participants reported improvement in their ability to incorporate the literature into medical practice compared to 44% of the seminar group.

Journal clubs have also been used extensively to aid in the education and training of pharmacy students and residents. The journal club was a major component in 90% and 83% of drug information practice experiences offered by first professional pharmacy degree programs and nontraditional PharmD degree programs, respectively. 7

When a journal club presentation is used to promote learning, it is important that an appropriate method exists for assessing performance and providing the presenter with recommendations for improvement. Several articles have listed important questions and criteria to use when evaluating published clinical studies. 8 - 11 However, using such questions or criteria in the form of a simple checklist (ie, indicating present or absent) does not provide judgments of the quality or depth of coverage of each item. 12 A rubric is a scoring tool that contains criteria for performance with descriptions of the levels of performance that can be used for performance assessments. 12 , 13 Performance assessments are used when students are required to demonstrate application of knowledge, particularly for tasks that resemble “real-life” situations. 14 This report describes the development and use of a rubric for performance assessments of “journal club” study critiques by students in the didactic curriculum and during an advanced pharmacy practice experience (APPE).

Two journal article presentations have been a required part of the elective drug information APPE at the West Virginia Center for Drug and Health Information for many years. For these presentations, students select a recent clinical study to evaluate and present their study overview and critique to the 2 primary drug information preceptors. Prior to rubric development, these presentations were evaluated using a brief checklist based upon the CONSORT criteria for reporting of randomized controlled trials. 15 Work on a scoring rubric for the student presentations began in 2002. The first step in its development involved identifying the broad categories and specific criteria that were expected from the journal club presentation. The broad categories selected were those deemed important for a journal club presentation and included: “Content and Description,” “Study Analysis,” “Conclusion,” “Presentation Style,” and “Questions.” The criteria in “Content and Description” involved accurate and complete presentation of the study's objective(s), rationale, methods, results, and author(s)' conclusion. Other criteria within the rubric categories included important elements of statistical analyses, analysis of study strengths and weaknesses, the study drug's role in therapy, communication skills, and ability to handle questions appropriately and provide correct answers. The first version of the rubric was tested in 2003 during the drug information APPE, and several rubric deficiencies were identified. Some sections were difficult to consistently interpret or complete, other criteria did not follow a logical presentation sequence, and a few of the levels of performance were based on numbers that were difficult to quantitate during the presentation. For example, the criteria under “Content and Description” were too broad; students could miss one aspect of a study's design such as blinding but correctly identify the rest, making it difficult to accurately evaluate using the rubric.

Version 2 of the rubric was reformatted to remedy the problems. The description and content categories were expanded to make it easier to identify the specific parts of the study that the students should describe, and the “Study Overview” category was divided into distinct parts that included introduction, study design, patients/subjects, treatment regimens, outcome measures, data handling method, dropouts per group, statistics, results, and conclusion. To facilitate ease of use by evaluators, a check box was placed next to each item within the individual parts. This format also allowed the student to see in advance exactly which criteria they needed to include during their presentation, as well as any that were later missed. The use of a checklist also aided evaluators when determining the overall score assigned to the subsections within this category. “Study Analysis and Critique” directed students to refer to the “Study Overview” category as a guide to the parts of the study they should critically analyze. “Study Conclusion” divided the scoring criteria into an enumeration of key strengths, key limitations, and the conclusion of the group/individual student. “Preparedness” included criteria for knowledge of study details and handling of questions. The “Presentation” category included criteria for desired communication skills. This rubric version was tested during 8 journal club presentations during the drug information rotation, and on a larger scale in 2003 in the required medical literature evaluation course for second-professional year students. During the second-professional year journal club assignment, groups of 2 or 3 students were each given 1 published clinical study to evaluate, which they later presented to 2 evaluators consisting of a faculty member plus either a fourth-professional year drug information rotation student or a pharmacy resident. The faculty members evaluating students included the 2 rubric developers as well as 2 additional faculty evaluators. The evaluators first completed the rubric independently to assess student performance; evaluators then discussed their scores and jointly completed a rubric that was used for the grade. The rubric was given to the students in advance to serve as a guide when preparing their journal club presentation. In addition, to provide students with actual experience in using the rubric, 2 fourth-professional year drug information APPE students each presented a journal article critique to the second-professional year class. The fourth-professional year students first gave their presentations to the drug information preceptors as practice and to ensure that complete and accurate information would be relayed to the second-professional year class. The second-professional year students then used the rubric to evaluate the fourth-professional year students' presentations; the completed rubrics were shared with the fourth-professional year students as feedback.

Based on student and evaluator feedback at the end of the journal club assignment, additional revisions to the rubric were needed. Students stated they had difficulty determining the difference between the “Study Analysis and Critique” category and the key strengths and weaknesses parts of the rubric; they felt they were simply restating the same strengths and weaknesses. Students also felt there was insufficient time to discuss their article. The evaluators had difficulty arriving at a score for the “Study Analysis and Critique” category, and students often did not know the important aspects to focus on when critiquing a study. Revisions to the rubric included expanding the presentation time from a maximum of 12 to a maximum of 15 minutes, explaining that the strengths and weaknesses should relate to the areas listed under “Study Overview,” and stating that only the key limitations that impacted the study findings should be summarized as part of the conclusion.

Version 3 of the rubric was tested during the 2004 journal club assignment for the second-professional year students. A brief survey was used to obtain student feedback about the rubric and the assignment as a tool for learning to apply literature evaluation skills. The rubric was revised once again based on the feedback plus evaluator observations. Through use of the first 3 versions of the rubric, the evaluators continually noted that students skipped key areas of the analysis/critique section when presenting their journal articles. Thus, for version 4, a list of questions was developed by the drug information faculty members to aid students in identifying the key considerations that should be included in their analysis (Appendix 1 ). To prepare this list, several sources were located that detailed questions or issues to take into account when evaluating a published study. 8 - 11 Specific questions were also added based upon areas that were consistently overlooked or inappropriately discussed during the journal club presentations. Version 4 of the rubric was used by the 2 primary drug information preceptors to evaluate the fourth-professional year student journal club presentations during the drug information rotation. Following each fourth-professional year student's journal club presentation, each evaluator independently completed the rubric. The evaluators then met together to briefly review their scores, discuss discrepancies, and modify their individual scores if desired. This was important because one evaluator would occasionally miss a correct or incorrect statement made by a student and score the student inappropriately lower or higher for a particular section. Based upon further feedback from students and evaluators, final revisions were made to the rubric. The final and current version (Appendix 2 ) was used for all subsequent fourth-professional year journal club presentations, for the second-professional year students' journal club assignments during 2005 and 2006, and for a new, similar journal club assignment added to the curriculum for third-professional year students in 2006. Feedback about the finalized rubric was obtained from the second- and third-professional year students.

To evaluate the rubric's reliability, 3 drug information faculty members used the final rubric to evaluate the journal club presentations by 9 consecutive fourth-professional year drug information experiential students. Intraclass correlation coefficients were calculated for each rubric section and the total score.

Five versions of the rubric were developed over a 3-year time period. The majority of the revisions involved formatting changes, clarifications in wording, and additions to the criteria. However, the change that appeared to have the greatest positive impact on the student presentations was the addition of the specific questions that should be considered during the study analysis and critique. Second- and third-professional year student feedback from the final version of the rubric is shown in Table ​ Table1 1 and is very positive overall. Representative comments from the students included: “Very helpful for putting the class info to use,” “Great technique for putting all concepts together,” and “This assignment helped me to become more comfortable with understanding medical studies.” The suggestions for change primarily involved providing points for the assignment (it was graded pass/fail for the second-professional year students), better scheduling (the journal club assignment was due at the end of the semester when several other assignments or tests were scheduled), and providing more pre-journal club assistance and guidance to students. A small number of students indicated they still found it confusing to critique a study after the journal club assignment, which was expected since literature evaluation skills take considerable practice and experience to master.

Pharmacy Students Feedback Concerning a Journal Club Assignment in Which the Rubric Was Used for Evaluation

*Items specific to rubric

† Based on a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree

‡ Positive response = agree or strongly agree

A survey of 7 recent fourth-professional year students who used the rubric to prepare for journal club presentations and who were also evaluated using the rubric found that all of the students agreed or strongly agreed with each item shown in Table ​ Table1. 1 . One representative comment was, “I was surprised at how articles appear to be good when I first read them but then after going through them again and using the form, I was able to find so many more limitations than I expected. I definitely feel that journal club has helped me to interpret studies better than I had been able to in the past.” Several fourth-professional year students took the rubric with them to use during other rotations that required a journal club presentation. After establishing that the rubric was user-friendly to evaluators and that students could clearly follow and differentiate the various sections, the reliability of the rubric in each of the 12 rating areas was determined (Table ​ (Table2). 2 ). The intra-class correlation coefficient demonstrated a high level of correlation between evaluators for each student for 11 of the 12 areas. A score of 0.618 was found for the section involving the students' response to questions. This was still considered acceptable; however, given that a fairly low variability in ratings affected the intra-class correlation coefficient due to the small scale (0-3 points) used in the rubric, with a relatively small number of observations. The intra-class correlation coefficient was calculated using the fourth-professional year students' journal club evaluations from the drug information rotation. Thus, by necessity, the evaluators consisted of the 2 primary faculty drug information preceptors and a drug information resident. These evaluators had previously used the rubric and the 2 faculty evaluators worked to develop the rubric. This may have increased the level of correlation between evaluators due to their familiarity with the sections of the rubric.

Rubric Intraclass Correlation Coefficients (N = 9)

*95% confidence interval

About 5 minutes are required for an individual evaluator to complete the rubric, with an additional 5 minutes needed for score comparison and discussion. In almost all cases, the reasons for any differences were easily identified through discussion and resulted from an evaluator simply missing or not correctly hearing what was said during the presentation. In general, evaluators found the rubric easy to use and did not require an extensive amount of time to consistently assess literature evaluation skills.

A rubric can be a useful tool for evaluating student performance in presenting and critiquing published clinical studies, as well as a valuable learning aid for students. However, developing a rubric that appropriately guides students in achieving the targeted performance, provides proper student feedback, and is user-friendly and reliable for evaluators requires a significant initial investment of time and effort. Multiple pilot tests of the rubric are generally required, with subsequent modifications needed to improve and refine the rubric's utility as an evaluation and learning tool. Once the rubric is developed, though, it can be used to quickly evaluate student performance in a more consistent manner.

As part of the development and use of a rubric, it is important that the rubric's criteria be thoroughly reviewed with students and they are provided the opportunity to observe examples of desired performance. Once a rubric is used to evaluate student performance, the completed rubric should be shared with students so they can identify areas of deficiency. This feedback will help enable students to appropriately modify their performance.

The journal club evaluation rubric can be used when teaching literature evaluation skills throughout all levels of education and training. Students early in their education will probably need to extensively refer to and rely upon the supplemental questions to help them identify key considerations when analyzing a study. However, as students progress with practice and experience and their literature evaluation skills are reinforced in actual clinical situations, their need to consult the supplemental questions should diminish.

Despite the considerable time and effort invested, the evaluation rubric has proven to be a valuable and ultimately timesaving tool for evaluating student performance when presenting a published study review and critique. More importantly, the rubric has provided students with clear expectations and a guide for desired performance.

Appendix 1. Study Analysis and Critique – Supplement

Appendix 2. Final evaluation rubric for journal club presentations

• Constructive Peer-to-Peer Participation
• ESP Program Home
• Library Resources
• WRITING RESOURCES
• Getting Help

Research Project: Critique Rubric (instructor use)

Critiquing a classmate's work is worth 13.5% of your grade over the course of the semester - (3) separate 2% critiques along the way and then your formal critique of their final Research Project submission, which is worth 7.5%

Critiques Along the Way

In order to gain full credit for the 3 critiquing assignments along the way, you should:

• provide honest, meaningful, constructive input into your classmate's work
• demonstrate that you've given their idea and work careful consideration and thought
• present your work in a well-edited dialog on the ANGEL discussion forum as well as through editing with Track Changes in Microsoft Word

Formal Critique

Your formal critique of your classmate's finished Research Project is worth 7.5% of your total grade for the course. Below are the criteria on which your critique will be evaluated by the instructor. Notice that there are not point values associated with specific areas - the assignment is worth a total of 75 points. This lack of more structured point assignment is because each student's critique is going to be unique based on the project you are critiquing and the quality of that product.

It is important, however, that students address each of the following areas in their Critique.

Write your Critique bearing in mind that it will be shared with your classmate, and we will be discussing our thoughts about this process during the lesson that week.

The formal Critique must be completed and submitted as assigned.

• Units and Programs

Make a Gift

• Student Resources
• Faculty Resources
• Teaching Assistants
• Undergraduate Assistants

Jester Center Room A115
 201 E 21st St.
 Austin, Texas 78712
 512-471-4421

Holistic Scale for Grading Article Summaries

From John Bean, Engaging Ideas

A summary should be directed toward imagined readers who have not read the article being summarized. The purpose of the summary is to give these persons a clear overview of the article’s main points. The criteria for a summary are

• accuracy of content
• comprehensiveness and balance
• clarity, readability, and grammatical correctness

6-point Summary

A six-point summary meets all the criteria. The writer understands the article thoroughly. The main points in the article appear in the summary with each point proportionately developed (the writer does not spend excessive time on one main point while neglecting others). The summary is as comprehensive as possible in the space allowed, and reads smoothly, with appropriate transitions between ideas. Sentences should be clear, without vagueness or ambiguity, and free of grammatical and mechanical errors.

5-point Summary

A five-point summary is still very good, but weaker than a six summary in one area. It may have excellent accuracy and balance, but show occasional problems in sentence structure. Or it may be clearly written but be somewhat unbalanced or less comprehensive than a 6 summary, or show a minor misunderstanding of the article.

4-point Summary

A four-point summary is good but not excellent. Typically, a four summary reveals a generally accurate understanding of the article, but will be noticeably weaker in the quality of writing than a five or six. Or it may be well written but over only part of the article being summarized.

3-point Summary

A three-point summary must have strength in at least one area of competence, and it should still be good enough to convince the grader that the writer has understood the article fairly well. However, a three summary typically is not written well enough to convey an understanding of the article to someone who has not already read it. Typically, the sentence structure in a three summary is not sophisticated enough to convey the coordinate and subordinate relationships in the article.

2-point Summary

A two-point summary is weak in all areas of competence, either because it is so poorly written that the reader cannot understand the content or because the content is inaccurate or seriously disorganized. However, a two summary convinces the reader that the writer has read the article and is struggling to understand it.

1-point Summary

A one-point summary fails to meet any of the areas of competence.

Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education

• Open Access
• First Online: 07 January 2020

Cite this chapter

You have full access to this open access chapter

• Kiruthika Ragupathi 3 &
• Adrian Lee 3  

45k Accesses

25 Citations

3 Altmetric

This chapter will examine the role of rubrics in higher education. At its best, a rubric is a carefully wrought expression of the professional judgement of a teacher, and identifies the learning goals and aspirations of performance. We will explain and illustrate how rubrics can be used as both teaching and grading tools. We will examine how, in clarifying the learning goals, rubrics help teachers not only provide feedback, but also improve course design. We will pay specific attention to the formative role rubrics play in mediating improvement in student performance. We will explore how assessment involves qualitative judgments and suggest that a good rubric can help in monitoring fairness and consistency in grading. The key features of a quality rubric for assessing student work and their effective use will be discussed. We will consider the approaches that may limit the effectiveness of rubrics and provide ways in which teachers can create valid and reliable rubrics. Finally, we will discuss how rubrics can impact and improve teaching practice.

You have full access to this open access chapter,  Download chapter PDF

Similar content being viewed by others

The Use and Design of Rubrics to Support Assessment for Learning

Challenging the Role of Rubrics: Perspectives from a Private University in Lebanon

Rubrics in the classroom: do teachers really follow them, what is a rubric.

Student-centered learning demands progressive means of assessment that enable students to view learning as a process to develop and use strategies to meet or exceed assessment expectations. Such continuous improvement is possible only when students receive continuous, timely, objective, and constructive feedback. However, most assessment tasks provide little or no information to improve or promote student learning, but instead simply provide test scores or grades that merely quantify performance. Students need to have information about the quality of their work while they work on their assessment tasks and need to comprehend what constitutes good performance. They need to understand what excellent work is and what poor work is and be able to know what they can do to improve. An increasing emphasis on formative assessment has fueled a push toward the use of rubrics in higher education as they focus on the criteria for quality of student work. Footnote 1 The use of rubrics and scoring guides give students a better understanding of what is being assessed, on what criteria grades are based, and what standards are expected.

Teachers read into the term, rubric, a ‘variety of meanings’ Footnote 2 and a ‘series of questions.’ Footnote 3 At its best, a rubric is a carefully wrought expression of the professional judgment of a teacher. In this judgment, the rubric identifies the learning goals and aspirations for performance. A rubric is an assessment tool that explicitly lists the criteria for student work and articulates the levels of quality for each criterion. It is a visual narrative that breaks down the assignment into component parts and provides clear descriptions of the characteristics of the work associated with each component, at varying levels of mastery. In his seminal article on rubrics, Popham was the first to identify the three essential features that a rubric must have: evaluative criteria; quality definitions; and, a scoring strategy. Footnote 4 Evaluative criteria identify the factors that determine the quality of a student’s work. Quality definitions, in turn, provide a detailed description of the skills and knowledge for each level that a student must achieve to reach the suggestive levels of performance to distinguish acceptable from unacceptable responses. The scoring strategy is the use of a rating scale to interpret judgments, and these may be scored either holistically or analytically. Rating scales, if used on their own, have only criteria but no performance level descriptions, and are therefore different from rubrics. Footnote 5 Brookhart provides a clear explanation of how rubrics, checklists, and rating scales include criteria but are different in how the scales are used. Footnote 6 Checklists use a simple yes/no, rating scales use a Likert-scale decision while the rubrics actually describe the performance for each criterion.

Rubrics have different meanings and divergent practices—analytic vs. holistic; generic vs. task-specific; teacher-centered vs. student-centered. Footnote 7 A holistic rubric requires teachers to score the overall process or product without any targeted, specific feedback to students, while an analytic rubric gets teachers to score separate, individual parts of the product or performance first, then sum the individual scores to obtain a total score. Thus, holistic rubrics are primarily used as scoring rubrics, while analytic rubrics are used as instructional rubrics. Another variation to these rubrics is the single-point rubric that only lists the criteria for proficiency but provides space for teachers to identify where students have exceeded expectations, as well as highlight specific areas of concern upon which students need to focus. Footnote 8 Scoring rubrics focus on the product, while instructional rubrics focus on the process. Andrade proposes using instructional rubrics to clarify learning goals, to design the instruction that address these learning goals, to communicate and clarify teacher expectations of these learning goals, to guide the feedback on students’ progress toward the learning goals, and to judge the final products in terms of the degree to which the learning goals were met. Footnote 9 Teacher-centered rubrics are primarily for teachers to quickly and objectively assign accurate grades, while student-centered rubrics specifically focus on student learning and achievement.

There are many reasons for using rubrics. To support their summative use, Broad identifies ‘legitimacy, affordability, and accountability’ as key reasons to use rubrics. Footnote 10 Whereas to support their formative use, Reddy and Andrade identify increasing student achievement, improving instruction and evaluating programs as key reasons. Footnote 11 Recently, the focus of rubrics has shifted gradually away from summative grading to formative purposes. A rubric can provide students with informative feedback on their strengths and weaknesses and prompts them to reflect on their own work. While it can be used as a mechanism to specify and communicate the expectations of an assignment to students, it can also be a secret scoring sheet used only by teachers to assess student work fairly, consistently, and efficiently.

Students generally favor the use of rubrics, whereas teachers tend to resist their use due to their ‘limited conception of the purpose of a rubric.’ Footnote 12 Some teachers do not see the need for a rubric, and many teachers find them to be too specific, too constraining, or too vague. Reddy and Andrade offer hope that ‘teachers might be more receptive if they understand that rubrics can be used to enhance teaching and learning as well as to evaluate.’ Footnote 13 Among the reasons why students favor the use of rubrics, and in particular instructional rubrics, is because they provide informative feedback about students’ strengths and highlight areas for improvement; ultimately, they support learning, good thinking, and the development of understanding and skills. Footnote 14

There is a clear need to shift away from teacher-centered, summative rubrics to student-centered, formative rubrics. In this chapter, we will focus primarily on the use of instructional rubrics to achieve this shift. We will first discuss in more detail why rubrics matter, going beyond their use to achieve fair and consistent grading, to their use as instructional scaffolds. We will then identify and provide a framework for the construction of such rubrics before discussing how these rubrics can be most effectively used and how they impact teaching.

Why Do Rubrics Matter?

In many Asian universities, including Singapore, professors often come from different cultural backgrounds from their students. Additionally, far from there being a singular Asian culture, there are many different cultures in most Asian university classrooms in part because students travel to other Asian nations for their education but also because so many Asian nations are themselves highly multicultural. This diversity of the student and teacher population in Asian Universities—culturally, ethnically, socially, and linguistically—makes transparency ever more important to effective teaching. Winkelmes, Boye, and Tapp argue that transparency in assignment design can overcome inequity in students’ educational experiences. Footnote 15 The characteristics of transparency and fairness embedded within rubrics make it a valuable tool in diverse higher education contexts such as in Asia.

Rubrics for Monitoring Fairness and Consistency

Rubrics offer the possibility of objective, consistent evaluation minimizing difference in grades even when multiple raters are involved in evaluating student work. Footnote 16 In Jonsson and Svingby’s review paper on rubrics, they conclude that ‘reliable scoring of performance assessments can be enhanced by the use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training.’ Footnote 17

Rubrics offer the necessary transparency in providing students with clear, accessible, and understandable benchmarks for developing and judging their work. They clarify teacher’s expectations and performance indicators through explicitly stated criteria, and show students how their work will be evaluated and what is expected of them. Footnote 18 Students agree that the use of rubrics makes the grading process fair as they can easily verify if they have met the criteria or not. Footnote 19 Students further report that they are less anxious and more confident in working on their assignments when expectations are clearly listed in the rubrics. Footnote 20

Rubrics improve students’ self-efficacy by helping students identify the key cognitive skills that they need to develop to excel in the assignment. With these skills identified, students can plan and self-assess their work, and thus rubrics can be important tools in supporting students become self-regulated learners. Footnote 21 However, this transparency in supporting students more effectively deliver what the teacher wants can encourage instrumentalism. Footnote 22 Kohn argues that this instrumentalism is likely to narrow students’ scope and restrict the development of skills beyond what is explicitly stated in the criteria. Footnote 23 A potential solution to this dilemma, as noted by Torrance, is to move from summative assessment of learning through formative assessment for learning to experiential assessment as learning. Footnote 24

Rubrics as Scaffolds for Assessment as Learning

Assessment tasks tend to focus primarily on monitoring and evaluating student learning, but most often provide little or no scaffold to promote learning. Teachers who advocate assessment as learning confirm that rubrics can teach as well as evaluate. Footnote 25 Andrade laments that ‘rubrics used only to assign grades represent not only a missed opportunity to teach but also a regrettable instance of the teacher as-sole-judge-of-quality model that puts our students in a position of mindlessness and powerlessness.’ Footnote 26 Using rubrics, teachers can explicitly list the assessment criteria to enhance the alignment of learning, instruction, and assessment. Footnote 27 In a student-centered approach, the rubric is shared and at times cocreated with students to support student learning. Footnote 28 Students can then use rubrics to plan their assessment task, clarify targets, determine and focus effort where needed, identify issues related to the task, regulate the process in the effort to produce high-quality work. Footnote 29 For example, Broad argues that rubrics ‘may have done more good for writing assessment and the teaching of writing than any other concept or technology’ when used as a scaffold. Footnote 30

Rubrics act as pre-assessment narratives that set clear expectations and visual cues to allow students to plan their response to a task. With standardized criteria of what constitutes good performance clearly stated, students use rubrics as a self-assessment tool to interrogate the assessment and monitor the quality of their own work. Footnote 31 With continued use of rubrics, students quickly start to notice patterns of recurring problems. This self-discovery and critical reflection of their own learning process can lead to self-improvement. In so doing, rubrics become part of a formative, student-centered approach to assessment. Rubrics initiate this approach by communicating and clarifying the teacher’s expectations to the students, and thus these explicit expectations can set in motion a process that can lead to improved student performance.

Feedback is the most effective scaffold to improve student work, and particularly when it is targeted at a specific assessment task, given regularly during students’ performance of the task or immediately after the completion of the task. Footnote 32 Feedback is effective only when it contains concrete information on how the highest level of performance can be achieved rather than simply evaluating the current level of work. Huba and Freed suggest that rather than ‘emphasizing grades in assessment, the focus should be on descriptive feedback for improvement. Feedback that focuses on self-assessment and self-improvement is a form of intrinsic motivation.’ Footnote 33 If meaningful feedback is provided in a timely manner, students are motivated to make positive changes in their current and subsequent work. However, the longer the delay in feedback, the less effective the feedback will be on performance.

Providing timely feedback alone is not sufficient; students need to be adequately prepared to use the detailed feedback. Detailed feedback on the rubric is useful in analyzing where students’ strengths and weaknesses lie and helps students identify the areas that need work so as to set their own plans for improvement. Feedback should never be provided as a means for a one-time quick fix, but rather be considered as a continuous process with repeated instances of feedback and opportunities to change students’ self-perceptions and behavior. Footnote 34

Maximizing improved student performance through the use of rubrics requires teachers to go beyond being merely prescriptive. Although rubrics set the expectations for a task, it is unwarranted to assume that all students will understand what is expected of them, or how they should approach the task. Torrance argues that students are more likely to succeed when assessment tasks have greater clarity on process, criteria and how the tasks are to be graded, coupled with more detailed assistance from teachers on how to achieve a particular grade or result. Footnote 35 In that same vein, Rezaei and Lovorn found that without training regarding effective rubric use, reliability or validity will likely not improve. Footnote 36 Students need help in understanding rubrics and must be taught how to actively use a rubric. Footnote 37 Discussing with students the techniques that can help them understand how to use different grading tools and engaging them in activities that teach them the benefits of grading tools is necessary for students to use rubrics effectively. A further powerful approach is to cocreate the rubric with the students. Andrade describes the cocreation of a rubric in her own teaching practice. Footnote 38 The cocreation process begins with discussing strong and weak examples of student work, and then asking the students to ‘brainstorm criteria for their own work.’ Andrade then uses the resulting list of criteria to draft a rubric, before eliciting comments from the students. Footnote 39 As pointed out by Huba and Freed, this process helps build consensus about the meaning of the criteria to be used in the rubric. They further note that including ‘students’ ideas in the final rubric conveys respect for students as people and builds student ownership for learning.’ Footnote 40

How to Create the Elements of a High-Quality Rubric?

Creating rubrics can be both time consuming and conceptually difficult depending on the type of rubric. Footnote 41 However, the process for developing a rubric, no matter the type, follows a similar set of steps Footnote 42 : (a) define the learning outcomes of the module; (b) describe the assessment tasks that cover these learning outcomes; and, (c) identify the criteria and standards of performance for these assessment tasks. In starting from the learning outcomes of a course, this process is similar to the backward design approach used in course development and recognizes the importance of alignment with the learning outcomes. Footnote 43 It is the criteria and standards of performance that constitute the rubric, but formulating these is not a trivial task.

Figure 3.1 details the procedure for rubric development and how it fits into course design by building on the assessment process of Huba and Freed. Footnote 44

Rubric development as part of course design (adapted from Huba and Freed, 2000)

In this rubric development procedure, the critical step is to identify the criteria and standards for the rubric. It is rarely necessary to build rubrics from scratch, instead referring to rubric samples can be a first step to developing a new rubric. There are many examples of rubrics that can be found on the web. One good collection is the set of rubrics developed under the VALUE (Valid Assessment of Learning in Undergraduate Education) project by the Association of American Colleges and Universities. Footnote 45 However, such sample rubrics should never be used as is, even when being used for a similar assignment. The practice of ‘adopt and adapt’ Footnote 46 can be a useful strategy when faced with the task of developing a rubric. In this approach, a complete working rubric is adopted and then adapted to suit the assessment context. The adaptation process begins by reflecting on the assessment task and the new context for which the rubric is being adapted. This context is twofold: first, the rubric needs to capture what is expected from the students; and second, how the teacher expects to grade the task. In the adaptation, the criteria and standards can be retained, but the performance level descriptions should be adapted to the new context. A rich source for these descriptions can be the feedback provided to students in previous assignments. Most times, teachers end up giving the same or similar feedback to students at various levels of performance, and these consistent evaluative judgments can then be easily translated into the intended proficient standards expected from students.

The rubric development process is never over but always evolving. Footnote 47 Banerjee et al. recommend regular monitoring and modification of rubrics to ensure reliability, validity, and usability. Footnote 48 Monitoring will include evaluating the degree to which ‘the scale was functioning and in which parts.’ Footnote 49 This requires sense-making sessions with peers and students as partners ‘where they are presented with descriptive facts against each evaluation criteria and are involved in a process of determining how well criteria have been achieved.’ Footnote 50 This approach can increase the transparency in evaluation and grading and can form the basis for rubric refinement. Modification of rubrics necessitates a systematic review and revision process. Studies suggest an approach that combines expert intuition, knowledge and experience needs to be employed in the review and revision process. Footnote 51 This involves an understanding of success indicators for the assessment task based on evidence from literature, an empirical analysis of past student performance data, and the study of different task samples from the past assignment submissions.

How to Use Rubrics Effectively?

The use of rubrics is not without criticism; many have argued that rubrics can be too subjective, too vague or too detailed, or that they can restrict students’ understanding of learning and their ability to self-regulate. Footnote 52 Poorly designed rubrics can also ‘misdirect student efforts and mis-measure learning.’ Footnote 53 Students need to understand how to approach and react to different practices of using rubrics-based assessments. Teachers can overcome such challenges and support students through careful planning. In this section, we provide some examples of how to make use of rubrics effectively.

A student-centered learning environment creates opportunities for students to showcase their creativity, push them beyond their comfort zone, and more importantly learn from their mistakes. For example, a well-crafted single-point rubric, as shown in Table  3.1 , can be one such instructional scaffold. It offers clarity of the assessment task along with clear guidance and support, which according to Hockings et al. are necessary to enhance independent learning. Footnote 54 It not only makes the process and expectations of disciplinary knowledge and communication transparent to students, but also opens new possibilities for the teacher to create a significant learning experience that helps students develop higher-level extending and applying skills. Footnote 55

Like holistic and analytic rubrics, a single-point rubric breaks down the assessment tasks into categories and outlines the standards for proficient student performance, but deliberately leaves open-ended the areas for success and shortcomings. Thus, a single-point rubric does not impose boundaries on student performance, instead offers flexibility in setting their own learning goals and promotes student creativity without sacrificing clarity. As Fluckiger notes, it ‘allows time for goal setting and revision, provides a place for noting current status, and sets an expectation of initiative and innovation.’ Footnote 56 It also emphasizes descriptive, personalized feedback specific to individual students and has the power to create a significant learning experience that challenges the focus on grades inherent in analytic rubrics. Thus, a single-point rubric acts as: (a) a process scaffold to support students in tackling complex assignments by breaking them down into smaller components; (b) a critical thinking scaffold to demand sophisticated thinking in students; and, (c) a disciplinary practice scaffold to induct students into the professional discourse and practice of their discipline. Footnote 57

Initiating classroom discussion using single-point rubrics prior to the start of an assignment can prompt reflection and promote scholarly critical thinking. Footnote 58 The intentionally blank open-ended spaces used to record areas of concern and excellence can stimulate discussion on the thought processes, analyses and judgments for each of the components and generate pre-assessment narratives. Reitmeier and Vrchota discuss the use of such exercises for deeper reflection not only to help students identify and achieve baseline knowledge, but also to be aware of how they meet the proficient standards and to provide evidence for how they have gone above and beyond proficient standards. Footnote 59

Turning now to rubrics in general, Sadler argues that the listed criteria may not mean the same thing to all students and may not be specific enough to guide student thinking. Footnote 60 Rubrics may not always be self-explanatory, and so it is good practice to do a mock critique in a class using exemplars, taken for instance from previous years’ work, that have gone through the same marking and feedback process. Footnote 61 As a variation, Blommel and Abate suggest the use of published journal articles as exemplars for such critiques. Footnote 62 Mock critiques help students understand the nature of the assessment task and signal the expected quality required to excel. Lipnevich et al. investigated the efficacy of providing detailed rubrics and written exemplars and found that such practice does indeed lead to substantial improvement in student performance. Footnote 63 Nordrum, Evans, and Gustafsson extended this practice by providing commented exemplars and explanations for the grade the exemplar was assessed to merit. Footnote 64 Mock critiques can further be used as tools for peer assessment and peer feedback. Getting students ready to be engaged in peer assessment is a two-stage process: The first stage involves familiarizing students with the rubric for reflection and critique of their own work and the second stage involves providing detailed feedback to peers. This approach recognizes the powerful learning that takes place when receiving and giving peer feedback, in particular for the development of evaluative judgment. Nicol et al. further suggest that the provider of feedback can benefit more than the receiver, as it is in the provision of feedback that evaluative judgment is exercised. Footnote 65

By integrating rubrics into assignments, students can self-assess their own work using a rubric and attach their marked rubric along with the assignment for grading. Recurring use of rubrics in a course that offers multiple opportunities for students to use them in assignments is an effective method to improve student learning. Such use can shape the formation of criteria in which the work is graded and can support the process of designing and developing co-constructed analytic rubrics, which can be ‘powerfully instructive.’ Footnote 66

Thus, rubrics help in providing timely feedback that is detailed, diagnostic, easier-to-read, personalized, and specific to each student’s work. Teachers can simply circle or check the appropriate descriptions that apply, while also adding in targeted comments only when needed. This makes the grading process both fairer and more efficient. In this way, rubrics are excellent tools that allow for easy tracking of student progress and improvement over time, promoting self-assessment and self-improvement.

How Rubrics Can Impact Your Teaching Practice?

Beyond making assessments fair, transparent and consistent, teachers who advocate the use of rubrics report that they make a significant impact in encouraging and enabling reflective practice, see for example the review by Jonsson and Svingby. Footnote 67 Such advocates argue that rubrics provide insight into the effectiveness of their teaching practice. Huba and Freed suggest several ways in which rubrics can become excellent ‘instructional illuminators’ to enhance instructional quality Footnote 68 :

Foremost is the need to design rubrics that are clearly aligned with learning outcomes. This alignment helps teachers gain greater clarity on both content and outcomes as they are focused on what they what their students to learn rather than on what they intend to teach.

Rubrics steer teachers toward being learning- and learner-centric rather than being task-centric. Referencing overall rubric results in class can be an excellent way of addressing class problems by not singling out individual students. Rubrics showing student development over time can help teachers gain a clearer view of teaching, specifically blind spots, omissions, and strengths.

Revision of rubrics takes place as teachers gather information and capture areas of concern in students’ understanding of criteria, students’ quality of work, and as teachers reflect on the difficulty in assessing and scoring student work. These can be used as teaching moments to teach or talk about the issues that are important and how such issues can be corrected. Not only does such information help in revising rubrics, it also supports the reviewing of teaching strategies and learning activities, or may even lead to the revision of learning outcomes. Rubrics can thus lead to a cycle of continuous improvement.

Teachers can also share their own rubrics and rubrics-articulated feedback with colleagues to initiate dialogue about teaching that can lead to rubric cocreation and development through collaboration. Not only will it promote a careful yet faster review and revision process but may also lead to the development of department-wide and/or institution-wide rubrics that represent common practices of particular interpretations of the discipline. More importantly, the signaling to students is coherent with regard to the quality of work expected of them from their teachers for most of the courses and emphasize the discipline-specific qualities that they need to develop to become experts in that area.

Teachers tend to associate a set of rubrics to one assignment, but it is certainly possible to learn a great deal about one’s teaching methods and student learning across multiple assignments, and across multiple courses. This cross-sectional analysis can be used to identify areas for improvement in one’s teaching. This analysis can also be used as persuasive evidence of teaching improvement in annual review, and in application for promotion and tenure.

A strategy to support this process of reflection and revision for teaching improvement can involve the use of a teacher summary rubric (Table  3.2 ).

Teacher summary rubrics condense how students completed an assignment in terms of accomplishing specific learning goals and understanding their discipline. Summary rubrics can be completed while grading the assignment with an assessment rubric. The patterns that develop under each of the criteria can help teachers identify the strengths and weaknesses of the assignment. Improvements to the assignment can then be developed and overall teaching practice can be modified to support better student learning. This strategy is an efficient way to check the alignment between course objectives and student learning, and to gather meaningful feedback on the overall class performance.

Concluding Remarks

Rubrics support the process of both summative and formative assessments. They are excellent tools for grading and judgment evaluation when used as scoring rubrics but can also be effective tools to explicate the learning and teaching processes when used as instructional rubrics.

Scoring rubrics are primarily grading tools that are effective in providing objective and consistent assessment of student work. They provide teachers a mechanism to score reliably, make valid judgments, and rationalize the grades awarded. They clarify teacher expectations and inform students on how to meet them in an easy-to-follow visual format. They facilitate transparency in instruction by making objectives and criteria explicit to students that are consistent with teaching goals. The feedback that students receive through scoring rubrics can help them improve their performance on subsequent work. Such transparency and feedback increase student self-confidence, self-efficacy, self-awareness, and self-determination. Footnote 69

Instructional rubrics embody the clear shift from teacher-centered, summative assessments to student-centered, formative assessments. They promote student achievement by allowing both students and teachers to use evaluative judgments and assessment results as means to further student learning. They help teachers provide productive, targeted feedback, and prompt students’ active involvement in making sense and engaging with feedback for ongoing improvement. They prompt students to continuously self-evaluate their work against specific criteria through reflection and action on feedback. While teachers are the enablers of facilitating student feedback literacy, students are the architects of utilizing such feedback for reflection and action. Footnote 70 Instructional rubrics, and in particular, single-point rubrics are excellent scaffolding tools that support such student engagement with feedback and favor the process of assessment as learning. Further, they demand students’ cognitive thinking and develop students’ disciplinary expertise and promote the use of assessment for new learning. Thus, holistically designed instructional rubrics scaffold the processes of self-assessment and self-regulated learning, and enable students to achieve specific outcomes of an assessment task and demonstrate what has been learned and achieved. Footnote 71 Finally, rubrics also promote the process of peer assessment and peer feedback by improving their ability to judge and provide feedback to their own and their peers’ work, thus changing students’ perspectives on their own abilities and potential.

Rubrics allow teachers to: (a) summarize student performance; (b) tabulate student accomplishment of learning goals; (c) disaggregate student scores by specific criteria and skills; and, (d) identify patterns of strengths and weaknesses of students’ work and of the assignments themselves. Thus, rubrics provide teachers with a greater understanding of their own teaching practice and encourage teachers to become reflective practitioners.

Susan M. Brookhart, How to Create and Use Rubrics for Formative Assessment and Grading (Alexandria: ASCD, 2013).

Phillip Dawson, “Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice,” Assessment & Evaluation in Higher Education 42, no. 3 (2017): 347–360, https://doi.org/10.1080/02602938.2015.1111294 .

W. James Popham, “What’s Wrong—And What’s Right—With Rubrics,” Educational Leadership 55, no. 2 (1997): 72–75.

Popham, “What’s Wrong—And What’s Right—With Rubrics.”

Brookhart, How to Create and Use Rubrics for Formative Assessment and Grading .

Susan M. Brookhart, “Appropriate Criteria: Key to Effective Rubrics,” Frontiers in Education 3, no. 22 (2018), https://doi.org/10.3389/feduc.2018.00022 .

Anders Jonsson, and Gunilla Svingby, “The Use of Scoring Rubrics: Reliability, Validity and Educational Consequences,” Educational Research Review 2, no. 2 (2007): 130–144; D. Royce Sadler, “Transforming Holistic Assessment and Grading into a Vehicle for Complex Learning,” in Assessment, Learning and Judgement in Higher Education , edited by Gordon Joughin (Dordrecht: Springer, 2009), 1–19.

David Balch, Robert Blanck, and David H. Balch, “Rubrics-Sharing the Rules of the Game,” Journal of Instructional Research 5 (2016): 19–49; Jarene Fluckiger, “Single Point Rubric: A Tool for Responsible Student Self-Assessment,” Delta Kappa Gamma Bulletin 76, no. 4 (2010): 18–25.

Heidi G. Andrade, “Teaching with Rubrics: The Good, the Bad, and the Ugly,” College Teaching 53, no. 1 (2005): 27–30.

Bob Broad, What We Really Value: Beyond Rubrics in Teaching and Assessing Writing (Logan: Utah State University Press, 2003).

Y. Malini Reddy, and Heidi G. Andrade, “A Review of Rubric Use in Higher Education,” Assessment & Evaluation in Higher Education 35, no. 4 (2010): 435–448, https://doi.org/10.1080/02602930902862859 .

Reddy and Andrade, “A Review of Rubric Use in Higher Education.”

Heidi G. Andrade, “Using Rubrics to Promote Thinking and Learning,” Educational Leadership 57, no. 5 (2000): 13–18.

Mary-Anne Winkelmes, Allison Boye, and Suzanne Tapp, eds., Transparent Design in Higher Education Teaching and Leadership: A Guide to Implementing the Transparency Framework Institution - Wide to Improve Learning and Retention (Stirling: Stylus, 2019).

Mary E. Huba and Jann E. Freed, Learner - Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning (Needham Heights: Allyn and Bacon, 2000); Deborah Crusan, Assessment in the Second Language Writing Classroom (Ann Arbor: University of Michigan Press, 2010); Dannelle D. Stevens and Antonia J. Levi, Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback and Promote Student Learning (Sterling: Stylus, 2013); and Brookhart, How to Create and Use Rubrics for Formative Assessment and Grading .

Jonsson and Svingby, “The Use of Scoring Rubrics: Reliability, Validity and Educational Consequences.”

Robin Tierney and Marielle Simon, “What’s Still Wrong with Rubrics: Focusing on the Consistency of Performance Criteria Across Scale Levels,” Practical Assessment, Research & Evaluation 9, no. 2 (2004): 1–7; Anders Jonsson, “Rubrics as a Way of Providing Transparency in Assessment,” Assessment & Evaluation in Higher Education 39, no. 7 (2014): 840–852; and Deborah Crusan, “Dance, Ten; Looks, Three: Why Rubrics Matter,” Assessing Writing 26 (2015): 1–4.

Heidi G. Andrade and Ying Du, “Student Perspectives on Rubric-Referenced Assessment,” Practical Assessment, Research & Evaluation 10, no. 5 (2005): 1–11.

Andrade and Du, “Student Perspectives on Rubric-Referenced Assessment.”

Ernesto Panadero, “Instructional Help for Self-Assessment and Self-Regulation: Evaluation of the Efficacy of Self-Assessment Scripts vs. Rubrics” (Unpublished PhD diss., Universidad Autónoma de Madrid, 2011); Ernesto Panadero and Anders Jonsson, “The Use of Scoring Rubrics for Formative Assessment Purposes Revisited: A Review,” Educational Research Review 9 (2013): 129–144; Anastasia Efklides, “Interactions of Metacognition with Motivation and Affect in Self-Regulated Learning: The MASRL Model,” Educational Psychologist 46, no. 1 (2011): 6–25; and Barry J. Zimmerman, “Self-Regulated Learning and Academic Achievement: An Overview,” Educational Psychologist 25, no. 1 (1990): 3–17.

Harry Torrance, “Assessment as Learning? How the Use of Explicit Learning Objectives, Assessment Criteria and Feedback in Post-secondary Education and Training Can Come to Dominate Learning,” Assessment in Education: Principles, Policy & Practice 14 (2007): 281–294.

Alfie Kohn, “The Trouble with Rubrics.” English Journal 95, no. 4 (2006): 12–16.

Torrance, “Assessment as Learning?”

Judith Arter and Jay McTighe, Scoring Rubrics in the Classroom: Using Performance Criteria for Assessing and Improving Student Performance (Thousand Oaks: Sage, 2001); Panadero and Jonsson, “The Use of Scoring Rubrics for Formative Assessment Purposes Revisited”; Reddy and Andrade, “A Review of Rubric Use in Higher Education.”

Andrade, “Teaching with Rubrics: The Good, the Bad, and the Ugly.”

John Biggs and Catherine Tang, Teaching for Quality Learning at University (Maidenhead: Open University Press, 2007).

Jonsson, “Rubrics as a Way of Providing Transparency in Assessment.”

Reddy and Andrade, “A Review of Rubric Use in Higher Education”; Fred C. Bolton, “Rubrics and Adult Learners: Andragogy and Assessment,” Assessment Update 18, no. 3 (2006): 5–6.

Broad, What We Really Value .

Andrade and Du, “Student Perspectives on Rubric-Referenced Assessment”; Dawson, “Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice”; and Sadler, “Transforming Holistic Assessment and Grading into a Vehicle for Complex Learning.”

Paul Black and Dylan Wiliam, “Assessment and Classroom Learning,” Assessment in Education: Principles, Policy & Practice 5, no. 1 (1998): 7–74.

Huba and Freed, Learner - Centered Assessment on College Campuses , 59.

Black and Wiliam, “Assessment and Classroom Learning.”

Ali Reza Rezaei and Michael Lovorn, “Reliability and Validity of Rubrics for Assessment Through Writing,” Assessing Writing 15, no. 1 (2010): 19–39.

Andrade, “Using Rubrics to Promote Thinking and Learning.”

Huba and Freed, Learner - Centered Assessment on College Campuses , 170.

Balch, Blanck, and Balch, “Rubrics-Sharing the Rules of the Game.

Craig A. Mertler, “Designing Scoring Rubrics for Your Classroom,” Practical Assessment, Research and Evaluation 7, no. 25 (2001); Deborah Allen and Kimberly Tanner, “Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners,” CBE Life Sciences Education 5, no. 3 (2006): 197–203; and Y. M. Reddy, “Effect of Rubrics on Enhancement of Student Learning,” Journals of Education 7, no. 1 (2007): 3–17.

Grant P. Wiggins and Jay McTighe, Understanding by Design (Alexandria: Association for Supervision and Curriculum Development, 2005); Biggs and Tang, Teaching for Quality Learning at University .

Huba and Freed, Learner - Centered Assessment on College Campuses , 10.

Terrel L. Rhodes, ed., Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics (Washington: Association of American Colleges and Universities, 2010).

Crusan, Assessment in the Second Language Writing Classroom , 72.

Balch, Blanck, and Balch, “Rubrics-Sharing the Rules of the Game.”

Jayanti Banerjee, Yan Xun, Mark Chapman, and Heather Elliott, “Keeping Up with the Times: Revising and Refreshing a Rating Scale,” Assessing Writing 26 (2015): 5–19, https://doi.org/10.1016/j.asw.2015.07.001 .

Gerriet Janssen, Valerie Meier, and Jonathan Trace, “Building a Better Rubric: Mixed Methods Rubric Revision,” Assessing Writing 26 (2015): 51–66. https://doi.org/10.1016/j.asw.2015.07.002 .

Pauline Dickinson and Jeffery Adams, “Values in Evaluation—The Use of Rubrics,” Evaluation and Program Planning 65 (2017): 113–116.

Banerjee, Yan, Chapman, and Elliott, “Keeping Up with the Times”; Janssen, Meier, and Trace, “Building a Better Rubric”; and Dickinson and Adams, “Values in Evaluation—The Use of Rubrics.”

Torrance, “Assessment as Learning?”; Lene Nordrum, Katherine Evans, and Magnus Gustafsson, “Comparing Student Learning Experiences of In-text Commentary and Rubric-Articulated Feedback: Strategies for Formative Assessment,” Assessment & Evaluation in Higher Education 38, no. 8 (2013): 919–940; and Brookhart, “Appropriate Criteria: Key to Effective Rubrics.”

Brookhart, “Appropriate Criteria: Key to Effective Rubrics.”

Christine Hockings, Liz Thomas, Jim Ottaway, and Rob Jones, “Independent Learning—What We Do When You’re Not There,” Teaching in Higher Education 23, no. 2 (2018): 145–161, https://doi.org/10.1080/13562517.2017.1332031 .

L. Dee Fink, Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses (San Francisco: Jossey-Bass, 2003).

Fluckiger, “Single Point Rubric.”

Allyson Skene and Sarah Fedko, Instructional Scaffolding (University of Toronto Scarborough: Centre for Teaching and Learning, 2014).

Jon F. Schamber and Sandra L. Mahoney, “Assessing and Improving the Quality of Group Critical Thinking Exhibited in the Final Projects of Collaborative Learning Groups,” The Journal of General Education 55, no. 2 (2006): 103–137, http://dx.doi.org/10.1353/jge.2006.0025 .

Cheryl A. Reitmeier and Denise A. Vrchota, “Self-Assessment of Oral Communication Presentations in Food Science and Nutrition,” Journal of Food Science Education 8, no. 4 (2009): 88–92.

D. Royce Sadler, “The Futility of Attempting to Codify Academic Achievement Standards,” Higher Education 67 (2014): 273–288, https://doi.org/10.1007/s10734-013-9649-1 .

Dawson, “Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice.”

Matthew L. Blommel and Marie A. Abate, “A Rubric to Assess Critical Literature Evaluation Skills,” American Journal of Pharmaceutical Education 71, no. 4 (2007): 63.

Anastasiya A. Lipnevich, Leigh N. McCallen, Katherine P. Miles, and Jeffrey K. Smith, “Mind the Gap! Students’ Use of Exemplars and Detailed Rubrics as Formative Assessment,” Instructional Science 42 (2014): 539–559.

Nordrum, Evans, and Gustafsson, “Comparing Student Learning Experiences of In-text Commentary and Rubric-Articulated Feedback.”

David Nicol, Avril Thomson, and Caroline Breslin, “Rethinking Feedback Practices in Higher Education: A Peer Review Perspective,” Assessment & Evaluation in Higher Education 39, no. 1 (2014): 102–122, https://doi.org/10.1080/02602938.2013.795518 .

Huba and Freed, Learner - Centered Assessment on College Campuses ; Popham, “What’s Wrong—And What’s Right—With Rubrics.”

Winkelmes, Boye, and Tapp, Transparent Design in Higher Education Teaching and Leadership .

David Carless, “Feedback Loops and the Longer-Term: Towards Feedback Spirals,” Assessment & Evaluation in Higher Education 44, no. 5 (2019): 705–714, https://doi.org/10.1080/02602938.2018.1531108 .

David Boud and Rebecca Soler, “Sustainable Assessment Revisited,” Assessment & Evaluation in Higher Education 41, no. 3 (2016): 400–413, https://doi.org/10.1080/02602938.2015.1018133 .

Bibliography

Allen, Deborah, and Kimberly Tanner. “Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners.” CBE Life Sciences Education 5, no. 3 (2006): 197–203.

Google Scholar  

Andrade, Heidi G. “Using Rubrics to Promote Thinking and Learning.” Educational Leadership 57, no. 5 (2000): 13–18.

———. “Teaching with Rubrics: The Good, the Bad, and the Ugly.” College Teaching 53, no. 1 (2005): 27–30.

Andrade, Heidi G., and Ying Du. “Student Perspectives on Rubric-Referenced Assessment.” Practical Assessment, Research & Evaluation 10, no. 5 (2005): 1–11.

Arter, Judith, and Jay McTighe. Scoring Rubrics in the Classroom: Using Performance Criteria for Assessing and Improving Student Performance . Thousand Oaks: Sage, 2001.

Balch, David, Robert Blanck, and David H. Balch. “Rubrics-Sharing the Rules of the Game.” Journal of Instructional Research 5 (2016): 19–49.

Banerjee, Jayanti, Yan Xun, Mark Chapman, and Heather Elliott. “Keeping Up with the Times: Revising and Refreshing a Rating Scale.” Assessing Writing 26 (2015): 5–19. https://doi.org/10.1016/j.asw.2015.07.001 .

Biggs, John, and Catherine Tang. Teaching for Quality Learning at University . Maidenhead: Open University Press, 2007.

Black, Paul, and Dylan Wiliam. “Assessment and Classroom Learning.” Assessment in Education: Principles, Policy & Practice 5, no. 1 (1998): 7–74.

Blommel, Matthew L., and Marie, A. Abate. “A Rubric to Assess Critical Literature Evaluation Skills.” American Journal of Pharmaceutical Education 71, no. 4 (2007): 63.

Bolton, Fred. C. “Rubrics and Adult Learners: Andragogy and Assessment.” Assessment Update 18, no. 3 (2006): 5–6.

Boud, David, and Rebecca Soler. “Sustainable Assessment Revisited.” Assessment & Evaluation in Higher Education 41, no. 3 (2016): 400–413. https://doi.org/10.1080/02602938.2015.1018133 .

Broad, Bob. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing . Logan: Utah State University Press, 2003.

Brookhart, Susan M. How to Create and Use Rubrics for Formative Assessment and Grading . Alexandria: ASCD, 2013.

———. “Appropriate Criteria: Key to Effective Rubrics.” Frontiers in Education 3, no. 22 (2018): 1–12. https://doi.org/10.3389/feduc.2018.00022 .

Carless, David. “Feedback Loops and the Longer-Term: Towards Feedback Spirals.” Assessment & Evaluation in Higher Education 44, no. 5 (2019): 705–714. https://doi.org/10.1080/02602938.2018.1531108 .

Crusan, Deborah. Assessment in the Second Language Writing Classroom . Ann Arbor: University of Michigan Press, 2010.

———. “Dance, Ten; Looks, Three: Why Rubrics Matter.” Assessing Writing 26 (2015): 1–4.

Dawson, Phillip. “Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice.” Assessment & Evaluation in Higher Education 42, no. 3 (2017): 347–360. https://doi.org/10.1080/02602938.2015.1111294 .

Dickinson, Pauline, and Jeffery Adams. “Values in Evaluation—The Use of Rubrics.” Evaluation and Program Planning 65 (2017): 113–116.

Efklides, Anastasia. “Interactions of Metacognition with Motivation and Affect in Self-Regulated Learning: The MASRL Model.” Educational Psychologist 46, no. 1 (2011): 6–25.

Fink, L. Dee. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses . San Francisco: Jossey-Bass, 2003.

Fluckiger, Jarene. “Single Point Rubric: A Tool for Responsible Student Self-Assessment.” Delta Kappa Gamma Bulletin 76, no. 4 (2010): 18–25.

Hockings, Christine, Liz Thomas, Jim Ottaway, and Rob Jones. “Independent Learning—What We Do When You’re Not There.” Teaching in Higher Education 23, no. 2 (2018): 145–161. https://doi.org/10.1080/13562517.2017.1332031 .

Huba, Mary E., and Jann E. Freed. Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning . Needham Heights: Allyn and Bacon, 2000.

Janssen, Gerriet, Valerie Meier, and Jonathan Trace. “Building a Better Rubric: Mixed Methods Rubric Revision.” Assessing Writing 26 (2015): 51–66. https://doi.org/10.1016/j.asw.2015.07.002 .

Jonsson, Anders. “Rubrics as a Way of Providing Transparency in Assessment.” Assessment & Evaluation in Higher Education 39, no. 7 (2014): 840–852.

Jonsson, Anders, and Gunilla Svingby. “The Use of Scoring Rubrics: Reliability, Validity and Educational Consequences.” Educational Research Review 2, no. 2 (2007): 130–144.

Kohn, Alfie. “The Trouble with Rubrics.” English Journal 95, no. 4 (2006): 12–16.

Lipnevich, Anastasiya A., Leigh N. McCallen, Katherine P. Miles, and Jeffrey K. Smith. “Mind the Gap! Students’ Use of Exemplars and Detailed Rubrics as Formative Assessment.” Instructional Science 42 (2014): 539–559.

Mertler, Craig A. “Designing Scoring Rubrics for Your Classroom.” Practical Assessment, Research and Evaluation 7, no. 25 (2001).

Nicol, David, Avril Thomson, and Caroline Breslin. “Rethinking Feedback Practices in Higher Education: A Peer Review Perspective.” Assessment & Evaluation in Higher Education 39, no. 1 (2014): 102–122. https://doi.org/10.1080/02602938.2013.795518 .

Nordrum, Lene, Katherine Evans, and Magnus Gustafsson. “Comparing Student Learning Experiences of In-text Commentary and Rubric-Articulated Feedback: Strategies for Formative Assessment.” Assessment & Evaluation in Higher Education 38, no. 8 (2013): 919–940.

Panadero, Ernesto. “Instructional Help for Self-Assessment and Self-Regulation: Evaluation of the Efficacy of Self-Assessment Scripts vs. Rubrics.” Unpublished PhD diss., Universidad Autónoma de Madrid, 2011.

Panadero, Ernesto, and Anders Jonsson. “The Use of Scoring Rubrics for Formative Assessment Purposes Revisited: A Review.” Educational Research Review 9 (2013): 129–144.

Popham, W. James. “What’s Wrong—And What’s Right—With Rubrics.” Educational Leadership 55, no. 2 (1997): 72–75.

Reddy, Y. M. “Effect of Rubrics on Enhancement of Student Learning.” Journals of Education 7, no. 1 (2007): 3–17.

Reddy, Y. Malini, and Heidi G. Andrade. “A Review of Rubric Use in Higher Education.” Assessment & Evaluation in Higher Education 35, no. 4 (2010): 435–448. https://doi.org/10.1080/02602930902862859 .

Reitmeier, Cheryl A., and Denise A. Vrchota. “Self-Assessment of Oral Communication Presentations in Food Science and Nutrition.” Journal of Food Science Education 8, no. 4 (2009): 88–92.

Rezaei, Ali R., and Michael Lovorn. “Reliability and Validity of Rubrics for Assessment Through Writing.” Assessing Writing 15, no. 1 (2010): 19–39.

Rhodes, Terrel L., ed. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics . Washington: Association of American Colleges and Universities, 2010.

Sadler, D. Royce. “Transforming Holistic Assessment and Grading into a Vehicle for Complex Learning.” In Assessment, Learning and Judgement in Higher Education , edited by Gordon Joughin, 1–19. Dordrecht: Springer, 2009.

———. “The Futility of Attempting to Codify Academic Achievement Standards.” Higher Education 67 (2014): 273–288. https://doi.org/10.1007/s10734-013-9649-1 .

Schamber, Jon F., and Sandra L. Mahoney. “Assessing and Improving the Quality of Group Critical Thinking Exhibited in the Final Projects of Collaborative Learning Groups.” The Journal of General Education 55, no. 2 (2006): 103–137. http://dx.doi.org/10.1353/jge.2006.0025 .

Skene, Allyson, and Sarah Fedko. Instructional Scaffolding . University of Toronto Scarborough: Centre for Teaching and Learning, 2014.

Stevens, Dannelle D., and Antonia J. Levi. Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback and Promote Student Learning . Sterling: Stylus, 2013.

Tierney, Robin, and Marielle Simon. “What’s Still Wrong with Rubrics: Focusing on the Consistency of Performance Criteria Across Scale Levels.” Practical Assessment, Research & Evaluation 9, no. 2 (2004): 1–7.

Torrance, Harry. “Assessment as Learning? How the Use of Explicit Learning Objectives, Assessment Criteria and Feedback in Post-secondary Education and Training Can Come to Dominate Learning.” Assessment in Education: Principles, Policy & Practice 14 (2007): 281–294.

Wiggins, Grant P., and Jay McTighe. Understanding by Design . Alexandria: Association for Supervision and Curriculum Development, 2005.

Winkelmes, Mary-Anne, Allison Boye, and Suzanne Tapp, eds. Transparent Design in Higher Education Teaching and Leadership: A Guide to Implementing the Transparency Framework Institution-Wide to Improve Learning and Retention . Stirling: Stylus, 2019.

Zimmerman, Barry J. “Self-Regulated Learning and Academic Achievement: An Overview.” Educational Psychologist 25, no. 1 (1990): 3–17.

Download references

Author information

Authors and affiliations.

Centre for Development of Teaching and Learning, National University of Singapore, Singapore, Singapore

Kiruthika Ragupathi & Adrian Lee

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kiruthika Ragupathi .

Editor information

Editors and affiliations.

Yale-NUS College, Singapore, Singapore

Catherine Shea Sanger

New York University Abu Dhabi, Abu Dhabi, United Arab Emirates

Nancy W. Gleason

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Ragupathi, K., Lee, A. (2020). Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education. In: Sanger, C., Gleason, N. (eds) Diversity and Inclusion in Global Higher Education. Palgrave Macmillan, Singapore. https://doi.org/10.1007/978-981-15-1628-3_3

Download citation

DOI : https://doi.org/10.1007/978-981-15-1628-3_3

Published : 07 January 2020

Publisher Name : Palgrave Macmillan, Singapore

Print ISBN : 978-981-15-1627-6

Online ISBN : 978-981-15-1628-3

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research