March 28, 2010

What is Measurement - Revisited or Howling at the Purple Moon

As education evolves, there is one aspect of it that keeps changing- how one measures growth in academic learning. At the heart of this red hot issue is the simplest of requirements, namely, that measurement requires numbers, numerical scores obtained from specific areas of assessment. Whenever one sees words, or descriptions of growth, rather than number scores on tests, you know instantly there is a problem. The goal when one measures anything is objective information as free from personal opinion and judgment as possible. Let’s look at some recent examples of measurement.

Statistics on special education programs in 2009 were recently published (Annual Disability Statistics Compendium: 2009, https://www.ideadata.org/arc_toc9.asp). It is a count of information that provides objective data. This includes:

1. 6,605, 695 students received special education in 2007. 10.6%, or 699,841, were ages 3-5.
2. 40.6%, or 2,682,524, were ages 6-11. (Once you get the numbers, then you can ask the questions, such as why are there so many more kids in special education at age 6 and age 5? Then you might discuss the issues of why more are not receiving services earlier in order to prevent referral later on.)
3. 43.8%, or 2,892,720, were ages 12-17. (The stability of the measurement from age 6 on suggests that this number is actually the one that should have there from ages 3-5.)
4. 5.0 %, or 330,610, are ages 18-21. (The others have graduated or dropped out.)
5. Numbers of students in the classification categories in 2007 were:
   1. 43.3%, 2, 559, 090, with specific learning disability
   2. 19.2%, 1,136,732 with speech or language impairment
   3. 8.3%, 487,460, with mental retardation
   4. 7.4%, 438,418, with emotional disturbance
   5. 2.2%, 131,227, with multiple disabilities
   6. 1.2%, 71,291, with hearing impairments
   7. 1.0%, 59,958, with orthopedic impairments
   8. 10.6%, 624,805, with other health impairments
   9. 0.4%, 25,530, with visual impairments
   10. 4.3%, 256,809, with autism
   11. 0.02%, 1,256, with deaf-blindness
   12. 0.4%, 23,475, with traumatic brain injury
   13. 1.5%, 88,193, with developmental delay
6. In 2007, Idaho had the smallest percentage of special education students, 8.60%, while New Jersey had the largest percentage, 15.66%.

Once you have the exact facts, with all of the nuances that measurement provides, then you can engage in discussion and debate. But the more problems we have in special education, the farther away from exactitude we travel. Here is the most recent example. As you read, compare what is construed to be measurement with the requirement that measurement must use numbers.

PROGRESS MONITORING

The National Center on Response to Intervention (RTI) has created a “Monitoring Tools Chart for Reading and Math.” Its “Technical Review Committee” (Remember, folks, all of these people are paid with our money...) “established a set of criteria for evaluating the scientific rigor of progress monitoring tools”. It is a Chart to measure Reading and Math to be used for both special education and general education, grades K-8. It is to be conducted at least monthly to (a) estimate rates of improvement (Stop right there. Estimate?), (b) identify students not showing adequate progress, (c) Compare the efficacy of different forms of instruction to design more effective, individualized instruction. Sounds great, doesn’t it? But remember that RTI is in place to prevent referrals to special education and is a general education component of intervention. Special education is a completely different animal, with testing to supply exact scores in whatever was being measured, in order to determine baselines and outcome. General education has standardized scores only from State and local testing of basic skills achievement.

This Progress Monitoring tool is being pushed by several national groups, including The National Dissemination Center for Children with Disabilities (nichcy.org). It describes its value as “a research-based strategy that measures student achievement through the use of targeted instruction and frequent (e.g., weekly, monthly) assessment of academic performance.” (Highlighting added.) It cites various references to support the tool, noting, “Progress monitoring works when teachers use it regularly to reflect on how well instruction is supporting each student’s needs, a standards-based system (that) can be the key to unlocking powerful skills and knowledge for teachers and students and can resulting success for the school, district, and state in an inclusive standards-based assessment and accountability system.”

How does Progress Monitoring work? (studentprogress.org/progresmon.asp#2)

1. Current levels of performance are determined and goals identified. Remember, this is general education and the way performance is measured is up to the teacher and the school. Measurement is also curriculum based and not generalized into other subjects or life experiences.
2. Academic progress is measured regularly.
3. Progress toward meeting the student’s goals is measured by comparing expected and actual rates of learning. (But how do they know without standardized testing and at least a baseline?)

Now let’s look at the Progress Monitoring Charts again (www.rti4success.org). Get acquainted with them in detail because you are definitely going to see them in some form in your school district. General Outcome Measurement is noted in early reading and math, phonics, comprehension, math computation, math concepts and application, quantity discrimination, and reading fluency. So far, so good. Now look at the measurement outcome once the teacher puts in all of the data collected (Chart Legend). The “scientific rigor” in the outcome measurement ends with:

Convincing Evidence
Partially Convincing Evidence or Convincing Indirect Evidence
Unconvincing Evidence
No Evidence Submitted.

Instead of numeric scores you have purple circles and purple and white half moons. (I now pull out my hair and howl in pain.) This group of federally funded hotshots has no shame. They even have the nerve to dress up this pseudoscientific mess with categories of measurement captioned:

Reliability of the Performance Score Reliability of the Slope
Validity of the Performance Level Score Predictive Validity of the Slope of Improvement
Alternate Forms Sensitive to Student Improvement
End of year benchmarks Rates of Improvement specified
Norms disaggregated for Diverse Populations Disaggregated reliability and Validity data

I’ve attached these charts for you to examine because if I just wrote about it you would likely think- Oh! She’s just doing her thing. Another rant! But here it is and you decide for yourself. The one good thing about this catastrophe is that it gives some new vocabulary to general education teachers to learn. But there is an endless list of what is truly awful about the creation and promotion of sloppy, unscientific, fraudulent measurement that it leaves you breathless and heartsick at the future of education and of children who need special education. What is the practical result of such malicious craziness? Here is an example as of today.

The New York Times reported that America’s reading scores have remained stagnant for 17 years. (Stagnant National Reading Scores Lag Behind Math, Sam Dillon, 3/25/10). Now remember, these are numerical scores- no purple circles or half moons. A reading test, mandated by Congress, was given to 338,000 4th -8th graders. They scored 264 on a 500 point scale in reading in 2009, compared with 263 in 2007, when the test was last given. 8th graders scored 221 on the 2009 test, the same average as two years earlier. This is an obscene, disgusting abandonment of every child in America. And to remedy it they give us purple circles. The special education data that began this blog tells the real story. Students with learning disabilities continue to be half of the total number of children in special education. And my bet is that there are at least twice as many students with reading impairment. Does anybody care? Or are they all going to continue to be educated and analyzed by those with endless letters after their names, fancy titles, and with the mantra “You lie, and I’ll swear to it.”