I have been curious how the state would address calculating value added scores for the sake of evaluations once new assessments were in place…and now I know.
Thinking through things for a moment…OAAs/OGTs were paper-based tests, which the state and new assessment advocates have purported to be less than par. Old assessments asked far more lower-level questions and required little critical thinking on the part of students. Passages in the reading assessments were not dynamic or rich in literary aesthetics; instead, they were often canned pieces written for the sake of testing, not for their value. Our teachers have learned how to teach to these tests, and our students have learned how to take these tests.
New assessments are computer-based requiring students to manipulate simulations, use various digital literacies to think through synthesizing information presented in many formats, and think much more critically and analytically in response to questions. These new assessments are more difficult, require more thinking, and mandate a totally different level of test-taking skills from kids. (Side note: I don’t think this is a bad thing. I like that the tests are more engaging and relevant to kids, and our student surveys show kids were more interested in online assessments than paper-based ones. The amount of testing time is another story…)
For some kids, scores may improve dramatically on these new assessments because they will be more engaged and challenged (versus practice boredom from the old regime); and for other kids, scores may decrease dramatically because they had become pros at testing in the old regime. A student who scored in the 80th percentile could grow to the equivalent 90th percentile in new tests just as a student previously scoring in the 80th percentile could drop to the 60th percentile in new tests.
The state, however, is not taking these potential differences in increases and decreases into consideration when defining how they will calculate value addedness for evaluations.
Raw scores between old and new assessments will be remarkably different, but the state says raw scores don’t matter in calculating value added scores. They say:
A student scoring in the 80thpercentile one year is expected to land in the 80th percentile the following year. Jumps or falls in percentile ranking would indicate students learned more or less than expected during that year and will improve or lower the value-added score.
This, then, assumes that all students will perform as well as or as poorly as all other students. In non-mumbo-jumbo terms, If the 80th percentile on an old assessment was a score of 90%, and scores are expected to plummet 30% on new assessments, then all students scoring at the 80th percentile on the old assessment will need to score around 60% on the new assessment for the percentiles to stay the same. Does that make sense? Students will have to be successful together or fail together to keep the percentiles.
This point is reiterated in the article:
Either way, he said, the state is comparing the curves and percentiles from different years – not just raw scores. Even if tests or scores change across the board, they change for everyone and value-added will only look at where a student, or group of students, falls on the curve. [emphasis added]
The expectation is that all scores will change evenly across the student population, which completely undermines the idea that some students could be more or less successful with changing test formats.
I am continually mystified by the use of value added scores in teacher evaluations. Completely mystified.