“A person who never made a mistake never tried anything new.”

~ Albert Einstein, born on this day in 1879 – died April 18, 1955

I embrace this statement. Nevertheless, I think Albert’s insight needs a few qualifications or caveats.

Eduhonesty: Science requires reliability and validity. A well-designed experiment will be structured to test one variable, or in rare cases, a few variables, controlling outside influences. Does light help the plant grow? Keeping all other conditions equal, we give one group of plants light and put another group of plants in the dark. We might expand the model to include different kinds of light.

When our test fodder is students, not plants, it’s simply irresponsible to make large numbers of changes at the same time. Changing the schedule, the materials, the interventions, the required classroom activities and the administration at the same time hardly ever creates progress — and when it does, it’s extremely difficult to pinpoint what caused that progress.

Thanks to No Child Left Behind, an element of frantic activity has taken hold in lower-scoring districts. Many districts have shown little improvement. I believe part of the problem has been the act of instituting sweeping changes that cannot subsequently be analyzed because we have too many interacting variables in our equation. Any systematic approach to teasing out what works becomes impossible because we did too many things at once.

Another factor that I’m certain plays into this picture: Not only do we do too many things at once, we frequently do them badly. The first year of any new system is likely to have bugs, sometimes lots of bugs. Staff will implement changes differently too. My intervention is not John’s intervention is not Maria’s intervention even if we went to the same training. Maybe I am spending 10 minutes daily on the new math system, John is spending thirty and Maria is spending the whole class period. Joey may have tossed the binder with the new system into his wastebasket. In cases like this, you can only end up with gobbledygook for data. That does not keep administrations from publishing that data, at least internally, and making decisions based on their nonsense numbers.