Lecture 4 · Week 4 · Lind, Marchal & Wathen ch. 4 · about 45 min

Displaying and Exploring Data

How to display and explore one variable with dot plots, stem-and-leaf displays, percentiles and box plots, how to measure skewness, and how to picture the relationship between two variables.

By the end you can

Construct and interpret dot plots and stem-and-leaf displays.
Locate quartiles, deciles and percentiles with the location formula.
Build and read box plots and flag outliers with the 1.5 times IQR rule.
Compute Pearson's coefficient of skewness and identify distribution shapes.
Read scatter diagrams and contingency tables that describe two variables.

Chapter 3 gave you the mean, median, variance and standard deviation. This chapter adds tools that describe where values sit (position), what the data look like when you draw them (shape), and how two variables move together. Everything here is still descriptive statistics: no probability, no inference.

Dot plots

A dot plot draws one dot for every observation above a horizontal number line. If two observations are equal (or too close to tell apart), the dots are piled on top of each other.

Because every dot is a real observation, you do not lose the identity of any value. That is the key difference from a histogram, where values are binned into classes and the individual observations disappear. Dot plots are best for small data sets and for comparing two or more groups on the same scale; histograms are better for large data sets.

Example. Tionesta Ford Lincoln and Sheffield Motors serviced cars on 24 working days each:

Tionesta: 23, 33, 27, 28, 39, 26, 30, 32, 35, 32, 29, 25, 28, 33, 32, 27, 35, 32, 36, 30, 36, 31, 35, 37
Sheffield: 31, 35, 44, 36, 34, 37, 30, 37, 40, 31, 32, 44, 43, 31, 43, 36, 26, 38, 42, 33, 36, 34, 37, 30

Software summary of the two samples:

Statistic	Tionesta	Sheffield
Mean	31.29	35.83
Median	32	36
Standard deviation	4.11	4.96
Minimum	23	26
Maximum	39	44

Reading the plots: Sheffield services more cars on a typical day (its mean is 4.54 cars higher) and its data are more spread out (larger standard deviation). The Sheffield distribution has a low value of 26 that sits 4 cars below the next lowest day. Tionesta serviced exactly 32 cars on 4 different days. The values cluster around 32 for Tionesta and around 36 for Sheffield.

	Tionesta	Sheffield
Mean	31.29	35.83
Median	32.00	36.00
Range	16.00	18.00
St. dev. s	4.11	4.96

Sheffield services more cars on a typical day (its dots sit further right) and its counts are more spread out. Look for the piles: Tionesta reached 32 cars on four days, and Sheffield has one unusually slow day at 26. Hover a dot to read it.

Stem-and-leaf displays

A stem-and-leaf display is another way to show a whole data set, but it keeps the actual digits. Split each value into a stem (the leading digit or digits) and a leaf (the trailing digit). Stems go down the left side, leaves line up to the right, and it is customary to sort the leaves from smallest to largest.

Compare this with a frequency distribution. A frequency distribution gives you class counts quickly, but it has two disadvantages: you lose the exact identity of each value, and you cannot see how values are spread inside a class. A stem-and-leaf display fixes both.

Example. Table 4-1 lists the number of people attending each of 45 performances at the Theater of the Republic:

96  93  88  117 127 95  113 96  108 94  148 156
139 142 94  107 125 155 155 103 112 127 117 120
112 135 132 111 125 104 106 139 134 119 97  89
118 136 125 143 120 103 113 124 138

The smallest attendance is 88, so stems start at 8; the largest is 156, so stems end at 15. After sorting the leaves:

Stem | Leaf
8    | 8 9
9    | 3 4 4 5 6 6 7
10   | 3 3 4 6 7 8
11   | 1 2 2 3 3 7 7 8 9
12   | 0 0 4 5 5 5 7 7
13   | 2 4 5 6 8 9 9
14   | 2 3 8
15   | 5 5 6

What you can read off it immediately: the minimum is 88 and the maximum is 156; only two performances drew fewer than 90 people; three drew 150 or more, and their exact values were 155, 155 and 156. Attendance concentrates between 100 and 130: for example, 120 people attended two performances, 124 attended one, and 125 attended three.

Data (edit me: commas or spaces)

Mean x̄: 118.889
Median: 118
Mode: 125
Range: 68
Sample variance s² (÷ n−1): 354.283
Sample st. dev. s: 18.822
Population σ (÷ N): 18.612
Q1 · Q3: 103.5 · 134.5
IQR: 31
Pearson skewness: 0.142

Each row is a stem; each digit after the bar is one observation. Turn the page sideways and it is a histogram, but every value is still readable.

Measures of position: quartiles, deciles and percentiles

The standard deviation measures spread; measures of position locate a value inside the ordered data. Quartiles divide the ordered data into 4 equal parts, deciles into 10, and percentiles into 100.

The first quartile $Q_1$ is the value below which about 25% of the observations fall; the third quartile $Q_3$ has about 75% below it. The median is the second quartile $Q_2$ .
If your GPA is in the 8th decile, about 80% of students are below you (decile 8 = the 80th percentile).
If you score in the 92nd percentile, about 92% scored less than you and about 8% scored more.

Location of a percentileFormula 4-1Formula sheet →

L_P = (n + 1)\frac{P}{100}

$L_P$: location (position number) of the Pth percentile in the ordered array
$n$: number of observations
$P$: the percentile wanted (25 for Q_1, 50 for the median, 75 for Q_3, 20 for the 2nd decile)

Use it when you need to find the position of a quartile, decile or percentile in data sorted from smallest to largest.

Steps for using it:

Sort the data from smallest to largest.
Compute $L_P = (n + 1)\dfrac{P}{100}$ .
If $L_P$ is a whole number, the percentile is the value at that position.
If $L_P$ is not whole (say 5.25), go to the position below and move that fraction of the way to the next value.

When the location is not a whole number. Suppose $n = 20$ and you want $Q_1$ : $L_{25} = (20 + 1)\dfrac{25}{100} = 5.25$ . Go to the 5th value and move 0.25 of the distance to the 6th value. A quartile, like the median, need not be one of the actual data values.

For a concrete case, take the six values 91, 75, 61, 101, 43, 104. Ordered: 43, 61, 75, 91, 101, 104. Then $L_{25} = (6 + 1)\dfrac{25}{100} = 1.75$ . The first quartile is 0.75 of the way from the 1st value (43) to the 2nd (61): distance $61 - 43 = 18$ , so $Q_1 = 43 + 0.75(18) = 43 + 13.5 = 56.5$ .

The same interpolation works for any percentile. To find the 23rd percentile of 80 observations, $L_{23} = (80 + 1)\dfrac{23}{100} = 18.63$ : take the 18th value, add 0.63 times the gap between the 18th and 19th values.

Data (edit me: commas or spaces)

Percentile P25

Mean x̄: 1947.933
Median: 2038
Mode: none
Range: 946
Sample variance s² (÷ n−1): 89274.924
Sample st. dev. s: 298.789
Population σ (÷ N): 288.658
Q1 · Q3: 1721 · 2205
IQR: 484
Pearson skewness: -0.904

Location formula: L = (n + 1)P/100 = (15 + 1) × 25/100 = 4. A whole number: read that position directly.

Box plots and outliers

A box plot (box-and-whisker plot) pictures five statistics of a data set at once. Draw a box from $Q_1$ to $Q_3$ , put a line inside it at the median, and extend whiskers from the box out to the smallest and largest values (or to the smallest and largest values that are not outliers).

Interquartile rangeFormula sheet →

IQR = Q_3 - Q_1

$IQR$: interquartile range
$Q_1$: first quartile
$Q_3$: third quartile

Use it when you want the spread of the middle 50% of the data, the width of the box.

Outlier limitsFormula sheet →

L = Q_1 - 1.5\,IQR, \qquad U = Q_3 + 1.5\,IQR

$L$: lower limit below which a value is an outlier
$U$: upper limit above which a value is an outlier
$IQR$: interquartile range

Use it when you must decide whether a value is an outlier: any observation outside these limits is flagged (an asterisk in a box plot).

Example (Alexander’s Pizza). For 20 deliveries:

Statistic	Value
Minimum	13 min
$Q_1$	15 min
Median	18 min
$Q_3$	22 min
Maximum	30 min

The box runs from 15 to 22, so the IQR is $22 - 15 = 7$ minutes: half of all deliveries fall between 15 and 22 minutes. The distribution is positively skewed, and there are two clues. First, the right whisker (22 to 30, a span of 8 minutes) is much longer than the left whisker (13 to 15, only 2 minutes): the top 25% of the data is more spread out than the bottom 25%. Second, the median is not centred in the box: it is 3 minutes from $Q_1$ but 4 minutes from $Q_3$ .

Example (Applewood buyer ages). Summary values: $Q_1 = 40$ years, median = 46 years, $Q_3 = 52.75$ years. Then $IQR = 52.75 - 40 = 12.75$ years, and the outlier limits are:

Upper: $52.75 + 1.5(12.75) = 52.75 + 19.125 = 71.875$ years.
Lower: $40 - 1.5(12.75) = 40 - 19.125 = 20.875$ years.

Three purchasers were 72 years or older (in fact 72, 72 and 73), so they appear as asterisks: the plot flags them as outliers. No one was younger than 21, so there is no low outlier. Apart from those high outliers the distribution is fairly symmetric: the two whiskers are about the same length, and the median sits close to the middle of the box (40 to 46 is 6 years, 46 to 52.75 is 6.75 years).

Skewness

The shape of a distribution is described by one of four patterns:

Shape	What it looks like	Mean versus median	Typical example
Symmetric	Left half mirrors the right half	Mean equals median	Ages of buyers
Positively skewed	Single peak, long tail to the right	Mean greater than median	Salaries, incomes
Negatively skewed	Single peak, long tail to the left	Mean less than median	Test scores when most students score high
Bimodal	Two (or more) peaks	Peaks suggest two populations	Outside diameters made by two machines

Positively skewed data are the most common in business. Think of salaries in a company of about 100 people: the president and a few executives earn far more than everyone else, so the right tail is long and the mean is dragged above the median.

Pearson’s coefficient of skewness uses exactly this mean-versus-median gap.

Pearson's coefficient of skewnessFormula 4-2Formula sheet →

sk = \frac{3(\bar{x} - \text{Median})}{s}

$sk$: coefficient of skewness
$\bar{x}$: sample mean
$Median$: sample median
$s$: sample standard deviation

Use it when you want a quick numerical measure of the direction and degree of skewness from the mean, median and standard deviation.

Interpretation: $sk$ runs from $-3$ to $3$ . A value near $0$ (mean equals median) means symmetry; a value near $-3$ (such as $-2.57$ ) means considerable negative skew; a value such as $1.63$ means moderate positive skew.

Statistical software (Minitab, Excel) instead computes skewness from cubed standardised deviations:

Software coefficient of skewnessFormula 4-3Formula sheet →

sk = \frac{n}{(n - 1)(n - 2)} \sum \left( \frac{x - \bar{x}}{s} \right)^3

$sk$: coefficient of skewness
$n$: number of observations
$x$: each observation
$\bar{x}$: sample mean
$s$: sample standard deviation

Use it when you are reading software output or asked for the software method; it is built from the cubed deviations of each value from the mean in standard-deviation units.

Why it works: each term $(x - \bar{x})/s$ standardises a value (how many standard deviations from the mean). Cubing keeps the direction (values above the mean stay positive). If the data are symmetric the positive and negative cubes cancel to about zero; a few very large values push the sum strongly positive; a few very small values push it negative.

For the two building blocks, recall from Chapter 3:

Sample meanFormula sheet →

\bar{x} = \frac{\sum x}{n}

$\bar{x}$: sample mean
$\sum x$: sum of all observations
$n$: number of observations

Use it when you need the arithmetic average of a sample.

Sample standard deviationFormula sheet →

s = \sqrt{\frac{\sum (x - \bar{x})^2}{n - 1}}

$s$: sample standard deviation
$x$: each observation
$\bar{x}$: sample mean
$n$: number of observations

Use it when you need the typical distance of observations from the sample mean.

Describing the relationship between two variables

Up to now every display described a single variable (univariate data). Bivariate data record two variables for each observation: hours studied and exam score; advertising spending and monthly billings; a father’s height and his son’s height; the odometer reading of a used car and its auction price.

A scatter diagram plots one variable on the horizontal (X) axis and the other on the vertical (Y) axis, one point per observation. Put the variable that you think depends on the other on the Y axis: if height of the father is thought to influence the height of the son, put father’s height on X and son’s height on Y.

How to read the pattern:

Direct (positive) relationship: the points run from lower left to upper right. Example: as a city bus gets older, its annual maintenance cost rises.
Inverse (negative) relationship: the points run from upper left to lower right. Example: as a used car’s odometer reading increases, its auction price falls.
Little or no relationship: the points form a shapeless cloud. Example: the height of shift supervisors has little to do with their yearly salary.
Strength: a tight band of points signals a strong relationship; a loose scatter signals a weak one. The Applewood data on profit and buyer age show only a weak positive relationship.

Contingency tables

A scatter diagram needs both variables to be at least interval scale. When one or both variables are nominal or ordinal, we cross-tabulate the counts instead.

Examples of suitable pairs: gender and class year of students; acceptable or unacceptable product and the production shift; party affiliation and whether the voter has children in the district.

Example (Applewood profit and dealership). Profit is a ratio variable, so it is first converted into an ordinal variable by splitting sales at the median profit of 1,882.50 dollars: above the median versus below it. The four dealerships form the other variable:

Above or below median profit	Kane	Olean	Sheffield	Tionesta	Total
Above median	25	20	19	26	90
Below median	27	20	26	17	90
Total	52	40	45	43	180

What the table says: as expected from the definition of the median, exactly 90 of the 180 sales lie on each side. Comparing dealerships with column percentages (each part divided by its column total):

Kane: $25 / 52 = 48\%$ of sales were above the median profit.
Olean: $20 / 40 = 50\%$ .
Sheffield: $19 / 45 = 42\%$ .
Tionesta: $26 / 43 = 60\%$ .

Tionesta stands out: 60% of its sales beat the median profit, well above Sheffield’s 42%. A second example: of 100 male customers, 32 ordered dessert (32%); of 100 female customers, only 15 did (15%). Men in that sample were about twice as likely to order dessert.

Exam practice

Summary and review

Dot plot: one dot per observation on a number line, equal values piled up. Keeps every value’s identity; best for small data sets and comparing groups.
Stem-and-leaf display: splits each value into a stem (leading digits) and a leaf (final digit); sorted leaves. Keeps exact values while showing shape, minimum, maximum and clusters.
Location formula: $L_P = (n + 1)\dfrac{P}{100}$ gives the position of the Pth percentile in the ordered data. If it is whole, read that position; if fractional, interpolate between the two neighbouring values.
Quartiles, deciles, percentiles: $Q_1 = L_{25}$ , median $= L_{50}$ , $Q_3 = L_{75}$ ; the kth decile is $L_{10k}$ ; about 25%, 50%, 75% of the data lie below the quartiles.
IQR: $IQR = Q_3 - Q_1$ , the middle 50% of the data. Box plot: box from $Q_1$ to $Q_3$ , line at the median, whiskers to the extremes. Long right whisker or low median position means positive skew; the mirror image means negative skew.
Outliers: values outside $Q_1 - 1.5\,IQR$ or $Q_3 + 1.5\,IQR$ ; drawn as asterisks on a box plot.
Skewness: symmetric means mean equals median; positive (right) skew means mean greater than median; negative (left) skew means mean less than median; bimodal means two peaks. Pearson: $sk = \dfrac{3(\bar{x} - \text{Median})}{s}$ , from -3 to 3, 0 meaning symmetric. Software: $sk = \dfrac{n}{(n-1)(n-2)}\sum \left(\dfrac{x - \bar{x}}{s}\right)^3$ .
Sample notation trap: $s$ divides by $n - 1$ , never $n$ ; population symbols are $\mu, \sigma, N$ against the sample’s $\bar{x}, s, n$ .
Scatter diagram: bivariate picture, X horizontal and Y vertical (dependent variable on Y). Lower-left to upper-right is direct/positive; upper-left to lower-right is inverse/negative; a cloud means little relationship. Axis scaling can exaggerate or hide strength.
Contingency table: cross-tabulates two nominal or ordinal variables; compare columns with percentages, as with profit above/below the median by dealership.