The Delusion of Understanding, The Illusion of Power: Numbers

Dr. Susan Coia-Gailey
Higher Education Consultant
Data-Based Institutional Research

As an institutional leader you need numbers, right? You need numbers/figures to make enlightened decisions, and to develop effective policies. More broadly, you need data to inform strategic plans and tactics to achieve strategic goals. Your institution’s strategic direction and goals should be guided by data. Though we may make broader reference to evidenced-based decision-making, there is always the quest for numbers, i.e., for reports, and a lot of them.

Well, we’ve heard the saying, “A little knowledge is a dangerous thing.” Figures can create the delusion of understanding and the illusion of power, i.e., that we’re knowledgeable and we’re in control. Figures – numbers – can become the end-all. You are not being scientific – analytical or objective – just because there is a number involved.

Moreover, in the board room, and in conference rooms across campus, figures may be used by some colleagues to appear knowledgeable and prepared – to impress. Fire enough figures, and colleagues may glaze over into confused submission to whatever sense the presenter makes of them, should the presenter dare to take that leap; figures are fodder for control. Information overload occurs even with the best of intentions. On the other hand, presenting figures in smaller, more digestible, doses may spark debate at your institution – sometimes heated – over interpretation and, importantly, the course of action they imply. A number may be a statistical fact; interpretation is subjective.

Even more dangerous is credibility attributed to the heavy hitter who presents figures from a presumed plentiful stash of data and who has no close rivals to challenge him. Colleagues may grow blindly confident, accepting his opinions with the assumption that cogent data, analysis, and methodology always back him up. The guru has spoken – enough said.

What is the fallout of this state of affairs?

There might be speculation over, and arguments about, the cause of a lower than desired graduation rate, including those of specific student populations and whether the rates are inevitable.

Efficient and effective distribution of financial aid/discounts is elusive, with enrollment, retention and revenue goals and budgets hanging in the balance (reactive tactics counter inaccurate forecasts).

Your Early Alert system is not so early, kicking in when students are already in free-fall, instead of time of acceptance for admission, or matriculation, in order to act pre-emptively.

After assessment of learning outcomes, there may be deliberation over what data-based course of action will improve outcomes without changing what may already work.

In higher education, learning and graduation are institutional outcomes that are in the limelight, as is cost of education; these outcomes will likely stay on the radar of the current overarching regard for return on investment.

Symptoms of this state of affairs consist of: (a) disconnects between figures and course of action, also known as leaps of logic (or faith); (b) debates over cause and underlying dynamics; (c) arguments about effective courses of action to improve outcomes of interest; (d) stalemates to action; and, (e) requests for more and more figures in vain and misguided attempts to clarify previous figures, which keeps offices quite busy in “go fetch” mode.

Arguments/debates and differences of opinion are not, in themselves, bad. They are often based on people’s various observations. People could simply be observing various facets, or features, of the situation at hand that fall on their radar or simply resonate with them. In fact, it is said, “Science begins with observation.” Don’t stop there. Problems result in the absence of follow-through: Observe, hypothesize, and test your hypothesis. Ask, “How do I know this to be true?” Conduct empirical investigations, confirmatory and exploratory. Understanding and managing important outcomes of interest to you requires an organized and integrated system of data-based institutional research, assessment and reporting. Expertise in data management and data analysis to meet research, assessment and reporting needs is critical, as is institution-wide coordination and participation.

So, how is it that figures/numbers give the delusion of understanding and the illusion of control? Well, every figure has a story. You need statistical analysis to enlighten you about dynamics, which includes insight into the interplay of multiple factors that impact the figure that is an outcome of interest to you. Quantitative information is the “bare bones’ that gives form to the “flesh” of qualitative information. When you can predict, you can control, or at least manage, your outcomes. Remember the saying, “If you think you understand something, try to change it.” The next time you are presented with a number, to what extent has data enlightened you about how to improve it?