Tests measure only how many test questions a student can answer correctly at that moment. How that knowledge got there, how long it will stay there, and how meaningful it is to the student are not being measured, even though most teachers would agree that such matters are truly important. Since there is no reliable way to know what a test score means, it is important to know what it probably means.
When a student fails a test, there is a strong possibility that the student didn’t understand the material. Let’s call that a true negative. But it is also possible that other factors, like test anxiety or even personal issues can create false negatives: the student knows the material, but the test doesn’t show it. When I was in graduate school, I had knee surgery. Several days later I flat-out failed a test in my heat transfer course (yes, there are whole courses about heat transfer), a course in which I had previously aced every test and would for the rest of the course. To this day, I have no idea how that happened, but that is a classic false negative.
As for people who do well on tests, a true positive means a good test score representing true learning. A false positive would mean a high score, but little or no genuine learning. From the self-reporting of my students, false positives (or partial false positives, where a student aces a test but has only learned some of the material) are commonplace, perhaps even the norm. I recognize this is not a valid sociological study, but studies of long term retention rates are, and they confirm the broad level of misleading test scores. It is simply not credible that the students who have exemplary test scores in class after class are forgetting what they “learned” in a matter of weeks or months. That’s not learning, but it sure looks like it - a classic symptom of doing school.
In a sense, trusting that high test scores mean learning has taken place is like asking a class “Are there any questions?”, and assuming that when no one has any questions it means everyone understands the material. They are both false positives.
When a teacher realizes he cannot trust or control the meaning of test scores, it is a humbling experience. No one can ever make good test scores into true positives. All a teacher can do, all he should do, is shape the classroom culture into one where the student prefers learning over cramming and forgetting, prefers honesty over the dishonesty of doing school.
A realistic teacher will recognize that only the student can truly know whether he has been learning or cramming, and that that is as it should be. It is exactly when a teacher uses forceful, intrusive tactics (pop quizzes spring to mind) that he becomes part of the problem and increases the ammount of doing school occurring in his classroom. Since his goal is to reduct the act of doing school as much as possible, such tactics should be used sparingly, if at all.