Stacking the Deck

Selective Use of Data to Support a Desired Narrative

Nov 06, 2020

I know this image doesn’t seem related to Navigating News, but I promise, it is. I want to start this week’s post with a fun optical illusion:

If you haven’t seen it before, here’s how it works: depending on how you look at this picture, you will either see an old woman with a pointy chin, hook nose and head scarf… or a young woman, looking over her right shoulder, wearing a choker and a hat with a feather in it. If you look at it long enough, you should eventually be able to see both.

The reason I like this image is because you and I could both be looking at it and see two totally different things… and neither one of us would be wrong. We would both be correct. It just depends on your perspective, right?

What does this have to do with Navigating News? I’m glad you asked! A lot of data gets shared via news media these days. In fact, between COVID and Presidential election polling, I’d say we’ve suffered from data overload this year. One of the biggest problems arising from so much data sharing is the tendency for reporters and regular people alike to selectively share and view data that supports their own opinions or a desired narrative, rather than a full set of data that tells a full story.

The example I want to use is REALLY simple, from a 2018 article that ran online in US News and World Report. Here was the headline:

Blacks, Hispanics, Native Americans and Alaska Natives graduate at lower rates than Asian and white students

*Unfortunately the link to the original article – which was really brief (about a paragraph at most) - has since been replaced with a 2020 article on the same topic. But that’s OK; for the purposes of this exercise, even just the headline itself will work.

The first thing I did after I read the story was look up the source information online to see if it was true. Sure enough, I found this chart, on the National Center for Education Statistics website, from 2018 (the year the article was published):

As you can see, the chart (which was NOT included in the story) confirms that the headline is definitely true. A factual statement. Whites and Asians did graduate at higher rates than Blacks, Hispanics, Native Americans and Alaska Natives. But the bigger question here is – did that headline and the subsequent short piece really tell you the whole story?

In my experience, the key to understanding data of any kind is viewing it in context, or with some kind of historical perspective. It’s sort of like if I said to you – “I went to the grocery this week and I spent $100 dollars!” Without knowing how much money I usually spend, you wouldn’t know if spending $100 was a good thing or a bad thing. It’s the same idea with these graduation rates. We need context to understand them better.

So, just for fun, I went through all of the prior years’ information on the very same subject; I made a small excel chart with the results of what I found (the top line is the 2017-18 school year data referred to in the US News headline:

*(links to the exact charts I used to source this are at the end of this blog. Also - the yellow line marks the first year the NCES seems to have started collecting this data annually)

When you see the 2017-18 data compared to all the previous years’ data, it suddenly starts to look different, doesn’t it? You suddenly have some additional perspective. In fact, after seeing all of the data in context, I’m not really sure the headline the writer chose for this story …

Blacks, Hispanics, Native Americans and Alaska Natives graduate at lower rates than Asian and white students

… really tells the whole story at all. As I looked at that data, I saw several different stories being told (and I’m sure you, coming from your own, different perspectives, can find many more):

Black students’ graduation rates have climbed upward every single year since 2009 and have made the largest overall gain—13%—of any of the student groups
The next largest gain is with the Hispanic group, at 10%.
The White group stayed exactly the same in both 2017/18 and in 2015/16. Their most recent growth was 1%, in 2016/17.
In 2018, the Alaskan / American Indian group made its first gain since 2014/15.
The Asian group actually took a major dip in 2013 and, though it has been rising ever since, it still hasn’t met its original percentage of 93%.

Given all of that, isn’t it just as feasible that the headline could have included some additional information? Like maybe:

something about the 13% gain in graduation rates for Black students,
or something about how, as other groups have continued to climb, the White groups have stagnated,
or even something about how the Asian group, despite currently being ahead of everyone else, has not yet managed to catch up to its own best number (93%).

I even find it interesting that the headline as written gloms the Asian and White groups together at all. Yes, I know they’re the top two groups, but even without the historical perspective, you can see in 2017-18 that the Asian group had three percent more graduates than the White group. So why lump them together like they are the same?

Here’s another interesting thing to think about: When you look at the 2017/18 data in context with all previous years, you see that, over even just four years’ time, the Black, Hispanic, Native American and Alaskan Native groups have managed to make significant gains in graduation rates. Why wouldn’t the article’s headline include a reference to that? It seems like pretty big news… maybe I’d even call it a buried lede. Except instead of being buried, it’s just… omitted.

I could go on, but hopefully you get the point.

By selectively choosing to focus on just one aspect of this pretty complex data, this reporter published a piece that both fits and feeds the common - and true, but way oversimplified - narrative that Black, Hispanic and American and Alaskan Indian groups are always behind whites in academic achievement. Meantime, the headline completely ignores the rest of a much bigger, and evolving, story - that those groups are all, slowly, closing the high school graduation gap. It’s sort of like showing that optical illusion of the ladies to someone and telling them it’s only a picture of an old woman. That’s not really true. It’s only part of the story.

My takeaway? Any time you read something including data, you have to try as best you can to get as much contextual information about it as possible. If that means looking online for the original source information, so be it. In the end, this has nothing to do with one person being right and another person being wrong. It’s got everything to do with what you see and what I see - the old lady versus the young one - and how a simple list of numbers, shared without context or different perspectives, can very easily be manipulated to make a desired point.

Question? Comment? Feel free to comment below or email me at navigatingnews1@gmail.com

Data for chart:

2017-2018

https://nces.ed.gov/programs/coe/indicator_coi.asp

2016-2017

https://nces.ed.gov/programs/coe/pdf/Indicator_COI/coe_coi_2019_05.pdf

2015-2016

https://nces.ed.gov/programs/coe/pdf/Indicator_COI/coe_coi_2018_05.pdf

2014-2015

https://nces.ed.gov/programs/coe/pdf/Indicator_COI/coe_coi_2017_05.pdf

2013-2014

https://nces.ed.gov/programs/coe/pdf/Indicator_COI/coe_coi_2016_05.pdf