Metrics and Moving Forward

Outcomes have assumptions.Well, outcomes can’t really have assumptions, but the people and the institutions that measure outcomes make assumptions about them.An outcome is the effect of a program. We measure it to show, communicate, prove that X service results in Y change. The assumption is that, say, rising test scores in Such-and-Such School is an outcome of the Mentoring Young Students Program. The assumption, in other words, is that the relationship between what’s measured and what’s offered is causal. So, outcomes communicate the (hopefully positive) relationship between the program and the people it serves.Basic stuff here, I know.But I’m interested in the set of assumptions behind the things being measured. For example, if we’re measuring test scores, it’s likely we’re assuming that an increase is good and a decrease is bad. That seems straight forward. That's merely one assumption. Here are some questions that point toward a bunch of other assumptions that are held (knowingly or unknowingly) by those measuring test score outcomes:

1. 1. Are other test scores being recorded? Are they increasing too? Or are they falling?
   2. Has the test changed over time, thereby influencing scores?
   3. Does the test measure for retention or application?
   4. Are critical thinking, abstract thought, cross disciplinary association being considered?
   5. Does the test matter socially, in the work force?
   6. Does the test consider environmental factors, emotional health, appreciation of the material?

What are you using to measure success?What are the outcomes you’re looking for that assure you that you’re making the kind of difference you want?Maybe the metrics that have been used in the past don’t help measuring success today. Do your homework and know why. It’s not enough to know that they don’t work.  (We all know that already!)Know the faulty assumptions behind the metrics, or you will either: a) keep using them and grumble along the way, b) not use them and not know how to replace them, or c) try to replace them and merely repackage them in different language.In all three cases, you have not moved forward (and will continue to be frustrated).