Activity: Misleading and Ambiguous Statistics

Quantitative Reasoning Activity

Misleading and Ambiguous Statistics – Sampling and Graphing

Learning Objective

Students will be able to discern how data sampling and graphical representation can be used in a misleading or ambiguous manner.  Students will be able to prescribe solutions to some common sampling and graphing problems.

Quantitative Literacy VALUE Rubric – Interpretation, Assumptions, Communication

 

Background

Sometimes we want to answer a question about a large group of people such as “what is the average height of an American adult?”  The only way to accurately and truly answer that question is to measure the height of every American adult, which is clearly impractical if not impossible.  Instead, what researchers will do to answer questions like that is obtain a representative sample – a subgroup of the population that represents the whole population – and perform the analysis on the sample.

Once data is accurately obtained, the researcher must then present his or her findings in an understandable format.  Graphs are extremely powerful tools for displaying lots of information or quickly conveying a message about data.

Unfortunately, improper sampling methods or graphical representation can lead to mistaken conclusions.  In this activity, students will get practice in looking at both with a discerning eye.

 

 

Activity

Part 1 – Misleading Sampling

When looking at any analysis or data, you want to make sure that the sample is representative of the population at large.  When this is not the case in a systematic way, it is called sampling bias and can result in distorted conclusions.

Truman

  1. The above photo is from November 3, 1948, the day after the presidential election in which incumbent Harry Truman defeated challenger Thomas Dewey. Truman is holding a copy of the Chicago Daily Tribune which incorrectly printed that Dewey had won the election.  The mistake of the Chicago Tribune editors is often attributed to a telephone survey that was not representative of the population.  Why might the results of a telephone survey not be representative of the entire population?
  1. You want conduct a study on the average allowance received by all teenagers in your neighborhood. Because the mall tends to have a lot of foot traffic, you decide to conduct your survey there.  Will your sample be a representative sample?  Why or why not?
  1. You read about a study which seeks to measure teenage use of illegal drugs. The study used a survey of high school students.  Why might this be an issue?

 

 

Part 2 – Misleading Graphs

All of the graphs presented in this section are based on the following data.

Year Home prices ($)
2008 122,901
2009 126,120
2010 123,192
2011 122,230
2012 122,990
2013 127,382
2014 124,209
2015 110,214
  1. Calculate the percentage decrease in home prices from 2014 to 2015. How does this compare with historical fluctuations?
  1. What is wrong with the below graph?

Graph 1

  1. How is this graph below an improvement over the previous graph? How might this graph still be misleading?

Graph 2

  1. How is this graph below an improvement over the previous two graphs? How might this graph still be misleading?

Graph 3

  1. How would you present this data in an accurate, meaningful way?


 Suggested Solutions/Discussion

Part 1 – Misleading Sampling

  1. A telephone survey may not be representative of the population because those who answer the telephone are not necessarily representative of the entire population. Back in 1948, telephone use was not yet widespread, and those who owned telephones tended to be wealthier than the general population.  If conducted today, a telephone survey would most likely over-represent people who are home during the day and able to answer their phones.  If only landlines are used, a telephone survey today would be biased towards older respondents.
  2. Your sample will probably be biased in that the subset of teenagers who spend time at the mall probably receives more allowance than the average teenager. Teenagers who receive no allowance are likely underrepresented.
  3. Such a sample would not include those who are homeschooled or those who have dropped out. Both of those subsets of the population are likely to have different behaviors than high school students.

Part 2 – Misleading Graphs

  1. (110,214 – 124,209) / 124,209 = -11.3%. This is much larger than the change in any other year (the next largest change is a 4% increase in 2013).
  2. The vertical axis is not labeled so we have no idea of the relative magnitudes of home prices in the two years.
  3. The vertical axis is now labeled. However, because the axis starts at 105,000, it looks at first glance like home prices were 3 to 4 times higher in 2014 as in 2015.  Also, this graph does not include historical data, and so we have no frame of reference as to whether this drop is historically significant or just a part of a frequently fluctuating housing market.
  4. Now we have historical data going back 8 years , which gives us a better idea of historical trends. However, because the range in the axis is so large, it looks like there has been no significant change at all in any of the years, when in fact there was a very large drop in 2015.
  5. This is just one of many possible solutions:Graph 4

 

Download Content: 

PowerPoint: Misleading and Ambiguous Statistics

Word Document: Misleading and Ambiguous Statistics