Tag Archives: testing

Measuring Up

I’ve become increasingly fascinated by our human obsession with measurement, and the ways in which measuring something affects–and often distorts–our ability to understand it.

There’s polling, of course, for the political among us. Despite the admonitions of the pollsters themselves, we far too often see the “snapshots” they provide– not to mention the selected populations they quiz and the often-ambiguous questions they ask–as descriptive of the whole of America’s electorate and thus predictive of the future.

In education, legislators have embraced subject-matter testing without considering the way it distorts what happens in the classroom. Subjects that will be tested get extra time and attention; subjects that are of equal (or often superior) importance, like civics, get short shrift because they aren’t tested. (And don’t get me started on the mistaken belief that students’ test rresults measure teacher competence…)

Scientists know that the very act of testing something  can change the results. Scholars also remind us that drawing unwarranted conclusions from what we have chosen to test can lead us astray. Which brings me to a Guardian column by Joseph Stiglitz, one of my favorite economists.

The world is facing three existential crises: a climate crisis, an inequality crisis and a crisis in democracy. Will we be able to prosper within our planetary boundaries? Can a modern economy deliver shared prosperity? And can democracies thrive if our economies fail to deliver shared prosperity? These are critical questions, yet the accepted ways by which we measure economic performance give absolutely no hint that we might be facing a problem. Each of these crises has reinforced the fact that we need better tools to assess economic performance and social progress.

Stiglitz proceeds to point out problems with relying on GDP–long the standard measure of economic performance–to measure a country’s economic performance. (GDP is the sum of the value of goods and services produced within a country over a given period.)

As Stiglitz notes, GDP metrics don’t fully reflect impacts of things like Europe’s austerity measures on long-term standards of living.

Nor do our standard GDP measures provide us with the guidance we need to address the inequality crisis. So what if GDP goes up, if most citizens are worse off? In the first three years of the so-called recovery from the financial crisis, about 91% of the gains went to the top 1%. No wonder that many people doubted the claims of politicians who were then saying the economy was well on the way to a robust recovery.

For a long time I have been concerned with this problem – the gap between what our metrics show and what they need to show. During the Clinton administration, when I served as a member and then chairman of the Council of Economic Advisers, I grew increasingly worried about how our main economic measures failed to take into account environmental degradation and resource depletion. If our economy seems to be growing but that growth is not sustainable because we are destroying the environment and using up scarce natural resources, our statistics should warn us. But because GDP didn’t include resource depletion and environmental degradation, we typically get an excessively rosy picture.

In other words, Stiglitz is telling us that there is something fundamentally wrong with how we measure economic performance and social progress.

Getting the measure right – or at least a lot better – is crucially important, especially in our metrics- and performance-oriented society. If we measure the wrong thing, we will do the wrong thing. If our measures tell us everything is fine when it really isn’t, we will be complacent.

A recent article in Time suggests that other nations are coming around to Stiglitz’ view.

New Zealand became the first nation to formally drop gross domestic product (GDP) as its main measure of economic success. The government of Prime Minister Jacinda Ardern said the budget would aim not at maximizing GDP but instead at maximizing well-being.

Apart from schools, hospitals and roads, whose budgets would be allocated in the normal way, resources would be distributed according to their impact on five government priorities: mental health, child well-being, the inequalities of indigenous people, building a nation adapted to the digital age and fashioning a low-emission economy.

Shades of Bhutan’s “Gross National Happiness” index!

Stiglitz says it is now possible to construct much more accurate measures of an economy’s health.. I think it is fair to say that we should adopt those measures–but only after we subject them to a rigorous analysis to assure ourselves that the elements being measured are the ones that should be measured, the ones that will give us a more accurate understanding of ecological and economic (and inevitably social and political) reality.

What we choose to measure will tell us what we really value.

 

 

Testing….1,2,3…Testing

I’ve been watching school reform efforts for several years now, and I’m depressed.

Most of the organizations that have formed to improve our public schools are populated by wonderful, well-meaning people, and most of the men and women who have chosen to teach in those schools are caring, dedicated professionals. So you’d think they would all be talking to each other and working together to identify and eliminate the barriers to better schools.

Instead, they seem to be at war with each other.

Now, I understand that focusing on common goals has been made more difficult by  the “take no prisoners” attitudes of ideologues like the departed-but-certainly-not-missed Tony Bennett, whose arrogance and autocratic tactics created a backlash of resentment among the teachers he regularly and unfairly bashed. (It shouldn’t surprise us when people who’ve been told they are overpaid and underperforming nitwits are unenthusiastic about collaborating with those who leveled the accusations.) But Bennett and his equally tone-deaf boss are gone, and the folks on the front lines–the teachers–need to help the real reformers understand what they need.

I haven’t been a high school teacher for nearly 50 years; neither do I have mastery of the reform literature. I’m just an interested observer who believes that public education is an immensely important public good, so you should take the following observations with the appropriate amount of salt.

Reformers are absolutely right to want teacher accountability. But teachers are absolutely right that high-stakes testing is not accountability.

Testing to figure out what kids know is a time-honored necessity; testing as a way to evaluate teacher performance is deeply problematic. For one thing, poor people move so frequently that turnover in many inner-city schools exceeds 100% during the school year, and the kids being tested at the end of the year aren’t the same kids who were tested at the beginning. Tests in such classrooms are meaningless.

Even in more stable environments, the current testing regime does significant damage–to students, who are being taught that there is always a “right” answer, and to teachers who are forced to focus their efforts on the subjects being tested and neglect other, equally important lessons. Furthermore, years of research demonstrate that more affluent kids test better for lots of reasons unrelated to the quality of classroom performance. If teachers are going to be evaluated and paid based upon test results, a lot of good teachers are going to leave the poorer schools that need them most and head for precincts where the students are better off and easier to teach.  (And yes, I know the theory is that we are testing for improvement, not absolute knowledge, but that theory is too often just that–theoretical.)

Here’s a heretical thought: before we engage in programs to assess accountability, let’s see if we can achieve agreement on what we mean by “education” and “quality instruction.” In other words, let’s be sure we know what instructors are supposed to be accountable for.

Too many of the self-styled “reformers” (not all, but too many) equate education with job training and quality instruction with (easy to test) rote learning.  For that matter, too many teachers agree with those definitions.

The people who genuinely want to improve public education–and there are a lot of them in both reform organizations and classrooms–  start by tackling the hard questions: what do kids really, really need to know in order to function in 21st Century America? What skills are essential? What are the barriers to imparting that information and those skills?  What additional resources do poorer kids need?  How much money does it take to provide a  good education, and how much does ignorance cost us?

Here’s how you can separate out the genuine education reformers from the ideologues and shills: real reformers understand the importance of public education’s civic mission. Because they understand the constitutive function of the public schools–because they understand that education is more than just another consumer good–they want to fix public education by working with teachers and parents and policymakers to make our public school systems work.

The genuine reformers aren’t the ones insisting that we  privatize or abandon those schools.