SIGN IN

Reliability Concerns for Classroom Formative Assessment

This week, I started teaching a course called Assessment: Theory and Practice to graduate students in the Leadership program at Saint Mary’s University. More than 70% of the students in the class are K-12 teachers. In a course like this, reliability and validity are of course big topics. In fact, next week’s class (5 hours of class time) will be spent on these two topics. So, I guess it’s the right time to return to these topics on this blog.

My last blog post on this topic was on May 25 on reliability concerns for classroom summative assessments. Today, let’s discuss reliability concerns for classroom formative assessment. Recall, we defined reliability as the degree to which students’ results remain consistent over time or over replications of an assessment procedure. An important point to remember is that reliability is a necessary, but insufficient, condition for valid score-based inferences. That is, you cannot make valid inferences from a student’s test score unless the test is reliable.

So, what are the reliability concerns for classroom formative assessment? Before we answer that question, let’s first define what we mean by “formative assessment.” Let’s use Popham’s definition (he always seems to have nice clear definitions. Professor Popham (2009) says “Formative assessment is not a test. Rather, it is an ongoing process in which teachers use test-elicited evidence to adjust their instruction or students use it to adjust their learning tactics.” Basically, formative assessment is a process by which teachers use assessment results to improve teaching and to help guide student learning. You can also find more of our thoughts on formative assessment here.

Let’s return to Nitko and Brookhart (2011) to summarize the reliability concerns for classroom formative assessment and what you can do to address them.

1. For oral questioning, you should be concerned about the dependability of the interpretation of answer(s) and the accuracy of teacher judgment. To increase reliability, you should:

  • Use sufficient number of questions or observations
  • Allow enough time

I really like this last suggestion. All too often, I find that teachers (me included) tend to answer their own questions too quickly when students don’t readily provide an answer. Give students enough time to think about an answer. If they don’t answer in 10-15 seconds, wait it out. Give them several more minutes. Usually, they eventually provide a very good answer.

2. For observations, you should also be concerned about the dependability of interpretation and accuracy of teacher judgment. To increase reliability, you should:

  • Interpret the observed behavior with the most likely and reasonable explanation.
  • Use a systematic procedure to observe students.

3. For self-assessment, you should be concerned with the rater (self) judgment. To increase reliability, you should:

  • Have a systematic procedure for rating. Often, it is good to use a rubric.
  • Instruct students on the use of the rubric.

So, how reliable are your assessment procedures? How do you increase reliability? How do you measure (compute) the reliability of your assessment procedures? Please leave a comment. I’d love to hear and learn from you.

In the next post in this series, I will consider validity concerns.

References:

Nitko, A. J. & Brookhart, S. M. (2011). Educational Assessment of Students (6th Edition). Boston, MA: Pearson.

Popham, J. (2009). A process – not a test. Educational Leadership, 66(7), 85-85.

Search

Popular Posts

  • Data Driven Instruction with Naiku

    Premier educators such as Dr. Paul Bambrick-Santoyo (Driven By Data 2.0: A Practical Guide to Improve Instruction) and Dr. John Hattie (Visible Learning) promote the use of data in enhancing instruction and student learning. In this linked white paper, Dr. Adisack Nhouyvanisvong discusses these techniques and showcases how to implement them using Naiku. Tweet

  • Achieving Visible Learning with Naiku

    Visible Learning is an excellent resource illustrating the effectiveness of student-centered learning which Naiku wholeheartedly supports. In Visible Learning and Visible Learning for Teachers, Dr. John Hattie (2009, 2012) synthesizes research studies involving hundreds of millions of students to show the effectiveness of different approaches to improve learning. Dr. Hattie found that student-centered learning strategies have the highest […]

  • Use ACT Quick Checks for Progress Monitoring

    Naiku provides over 50 ACT Quick Checks for teachers to use for student progress monitoring in all ACT test subjects. ACT Quick Checks are short, topic-focused, formative assessments; typically 6-12 questions in length. Each Quick Check contains questions from a single topic, such as Math-Functions, so teachers can use to easily monitor progress between benchmark […]

Menu