Volume XXXVII, No. 21 | October 2, 2015

Tests Are Not Enough

Measuring student classroom learning is now among the top concerns identified by the Southern Association of Colleges and Schools (SACS), and pressure is increasing to hold teachers more strictly accountable for it. Although student-evaluation-of-teacher surveys may be institutionally helpful, they have great limitations for those individuals who wish to better understand student perceptions of learning in relation to their course performance. The changing profile of undergraduates and their instructional needs, especially in core courses, makes it imperative to know students’ perceptions of learning.

People enrolling in community colleges often come to us ill-prepared for the rigors of postsecondary education; they are also notoriously bad test and note takers. While these students may actually learn a great deal in a course, they may not have developed the skills necessary to express that learning through tests alone. Yet, test scores remain the chief indicator of student learning among educators. It is my belief that determining student learning based on that one parameter is insufficient. Since learning is such a complex phenomenon, how can the assessment tool be so simple?

That is not to say that test performance is not a very important consideration. However, having numerous other assessment strategies available provides students with more opportunities to demonstrate knowledge growth when test scores are poor (for whatever reason) and to boost their grades. I use the following ten parameters in my science classes to observe trends in individual and classroom learning achievement, which is clearly consistent with the American Association for Higher Education’s principles governing good assessment practices.

1. Identical pre- and post-testing
2. Three regular tests and one take-home final
3. Three written assignments
4. In-class reading of scientific articles and viewing of course-related videos, all of which require on-the-spot written summaries
5. Community service and/or educational initiatives within or outside the classroom
6. Up to four optional assignments
7. Attendance
8. Attitude
9. Effort
10. Completion of an end-of-semester, course-specific questionnaire designed to help gauge the strengths and weaknesses of the instructional approach and textbook, as well as to understand students’ perceptions of their learning.

While there is no end-of-semester survey, students are asked to estimate the percentage of course material they feel they have learned.

Each parameter is assigned a maximum percentage point value, which allows students to earn anywhere from zero to the assigned number of points according to his or her level of engagement in the given activity. The maximum points a student can earn in a single parameter depends upon the nature of his or her involvement. Tests and quizzes and regular assignments having the highest value (up to 15 percent each). Point allocations vary slightly according to the course. In human ecology, for example, greater weight is placed on community environmental involvement. In introductory biology, written assignments have a higher point value, and in labs, it is attitude and effort that count the most. The only parameter that gets the same range of percentage points across courses is attendance, which is on a 1-10 scale.

The grading system is fully described in the syllabi and reinforced throughout the semester in class so nothing is left to speculation. Students are even provided with a “Grading Sheet” on which to record and keep track of their points as they earn them, and an “Absence Scale” to refer to as a constant reminder of the number of points they can earn based on how regularly they attend class. Students are vehemently and repeatedly assured throughout the semester that they will never lose points for not doing something while in the same breath they are warned that they will not earn points either for doing nothing.

A word about my teaching approach is in order here. I use instructional objectives, but not those provided in textbooks. For example, I go through each course textbook and personally prepare and distribute to students sets of objectives that cover each chapter that will be discussed during the semester. There are three sets, and students are obliged to complete all objectives designated in each set and submit them in writing when taking the test that covers the same material. Every assignment objective is discussed in class, and the tests are comprised only of questions dealing with those discussions. There are no trick questions. If students choose to complete all the objectives (assigned and unassigned) in a packet, they receive extra credit points. The objectives are revised with each edition or new textbook adoption.

This assessment strategy is designed to develop the mindset that the ball is in the students’ hands, not the teacher’s, and it aligns well with the “fairness to student ethic” being promoted nationwide in assessment programs like the “Measuring Student Learning” module from San Francisco State University’s Center for the Enhancement of Teaching. Students become more at ease with and even enjoy the learning process when they feel in complete control of their grades.

In order to quantify the value of this approach to students, a few years ago I analyzed close to 600 performance records that I had retained from five consecutive semesters. I compared individual final averages of the lecture students with (1) their pre-and post-test scores, (2) the percentage of course material they estimated on the questionnaire as having learned, and (3) the percentage of points students earned in each of the other parameters. For example, if a student had gotten six out of ten possible points, 60 percent was recorded for that parameter. I presented my results to the Science Education section of the Kentucky Academy of Science annual meeting.

The paper, entitled “Useful Parameters for Assessing Classroom Student Learning Outcomes: A Case Study” (J. KY Acad. Sci. 68(1): 115), reported that aside from test scores and regular assignment completion, the most influential factors in student achievement levels were frequency of attendance, individual involvement activity, and optional assignment completion. In other words, the more students took advantage of these learning strategies, the higher their final grades. Moreover, their pre-test scores, ranging from 40-57 percent, demonstrated that students begin the semester with similar levels of knowledge in the subject area, and their post-test scores (73-85 percent) indicated they had improved their knowledge considerably, suggesting one of two possibilities. Either the majority of students had learned quite a bit, or that many of them had guessed very well! Given the nature of most post-test answer choices—’Correct’ or ‘Incorrect’ versus the pre-test, which includes a ‘Don’t Know’ option—the latter would not be at all surprising. Still, there is no getting around the fact that post-test scores did align very well with the percentage of material that students said they felt they had learned.

However, students’ final grade averages did not show the same degree of alignment as the post-test scores did with their perceived learning percentages. In particular, there was significant disparity among the A and E earners. The mean estimate of what the former thought they had learned was 15 percent below their final grade, while the mean estimate of the latter was 20 percent above their final grade. To me, this perfectly illustrates two old adages: “The more you learn, the less you feel you know.” and “The less you learn, the more you claim to know!”

With respect to lab students, the percentage points earned for attitude (conscientious adherence to lab protocol) and attendance were critical to their final grades. The amount of material they perceived as having learned was notably higher than that among lecture students, which reaffirms the long-held belief that hands-on experience improves interest and, hence, learning. Again, this finding comes as no surprise.

Over the five semesters studied, 71 percent of students completed the course-specific, end-of-semester questionnaires, a figure that our institutional “Student Evaluation of Instructors” initiative at the time could only hope to have achieved. Nearly all students said that the instructional approach—teacher-prepared instructional objectives they are obliged to complete—was helpful to learning and test preparation and favored the attendance policy. Indeed, of the parameters employed here, attendance figured most prominently in the final grades, with students that attended regularly earning the highest grades. This is certainly not a revelation, but many college and university teachers either lack or have very loose attendance policies, which I believe to be a mistake. A fair, well-structured attendance policy compels students to be better organized and prepared. Likewise, class participation is one of the hallmarks of late 20th century instruction, and every student in the 60 percent of those who participated in course-related, individual involvement activities during those five semesters earned an A, B, or C final grade.

The retention rate for my classes predictably hovers around 90 percent or more, and students complete them with a successful grade at 89 percent. If we were to exclude those students who earned an E because they never showed up for class and failed to withdraw, the retention rate would be somewhere around 98 percent. These are highly desirable figures for any instructor of a core college course and suggest to me that having many parameters, including a non-punitive attendance policy, is an important factor.

However limited in scope my study was, it does illustrate that employing several parameters not only provides students with a wider berth in which to express their learning, but the approach also provides instructors with a better strategy for assessing students’ understanding of what we teach.  It also supports the view that a non-punitive attendance policy can work and demonstrates that pre- and post-testing can be another valuable learning assessment tool. Student performance on these tests provides an excellent opportunity for teachers to identify content areas in which more attention might need to be focused. Likewise, by soliciting student opinions at semester’s end about their learning experience via a subjective course-specific survey, teachers can acquire useful feedback that may help them improve future instructional delivery.

John G. Shiber, Professor, Biology

For further information, contact the author at Kentucky Community and Technical Colleges System-Big Sandy District, 1 Bert T. Combs Drive, Prestonsburg, KY 41653. Email: John.shiber@kctcs.edu