Chapter 9: Correlation and Regression

Correlation

Correlation

Correlation: The relationship between two variables.

Variable Types:

• Independent Variable:
  • Variable that is directly manipulated
  • Plotted on the $x$ plane
• Dependent Variable:
  • Variable that is not directly manipulated; its value depends on other variables
  • Plotted on the $y$ plane

Types of Correlation:

• Positive Linear Relationship:
  • When $x$ increases, $y$ increases
• Negative Linear Relationship:
  • When $x$ increases, $y$ decreases
• No Relationship:
  • There is no correlation between $x$ and $y$

Correlative Data Descriptive Statistics

Sample correlative data can be organized is a table:

• Example:

  Month	Miles Walked; $x$	Pounds Lost; $y$
  January	50	4
  February	65	4
  March	59	4
  April	60	3
  May	45	6
  June	52	2
  July	56	6
  August	35	7
  September	65	4

Scatter Plot

Scatter Plot:

• Graph of ordered pairs of the $x$ independent and $y$ dependent variable values
• Example:
  • In the above table, the ordered pairs are {(50, 4), (65, 4)…}.

  • Scatter plot:

    

Scatter Plots can visually demonstrate the relationship between variables:

• Negative Linear Relationship:

  

• Positive Linear Relationship:

  

• No Relationship:

  

Correlation Coefficient

Correlation Coefficient

Correlation Coefficient:

• Measure the strength of the relationship between variables
• In statistics, population correlation is generally inferred from sample correlation.
  • $\rho$: Population correlation
  • $r$: Sample correlation

Pearson product moment correlation coefficient (PPMC):

• Type of correlation coefficient
• Ranges from -1 to 1
  • -1: strong negative linear relationship
  • 0: no linear relationship
  • 1: strong positive linear relationship

Formula for Calculating the Correlation Coefficient

The formula for calculating the correlation coefficient is:

Setting up a table with the relevant data and sums makes it easier to substitute terms in the equation:

Substituting these sums in the above equation gives us:

The calculated value for $r$ is -0.589, suggesting a moderate negative correlation.

Testing a Correlation Hypothesis

Correlation Hypothesis

H0 and H1 Hypotheses:

• H0 is the null hypothesis, which is passive and does not make a claim.
• H1 is the alternative hypothesis, which actively makes a claim.

Correlation Coefficient Hypotheses:

• In a correlation coefficient hypothesis, the active claim is that there exists a relationship between the two variables.
• Thus, H0 is that there is no relationship; H1 claims there is:

Testing a Correlation Hypothesis

A correlation hypothesis can be testing using the $t$ value test:

• First, the $t$ value is obtained.
• Next, the corresponding area is retrieved from a $t$ distribution table, and is compared with $\alpha$:
  • If the area is beyond $\alpha$, H0 is rejected
  • If the area is not beyond $\alpha$, H0 is not rejected

Correlation Coefficient $t$ Value Formula:

• For a correlation coefficient:

  \[d.f. = n-2\]

• The formula for finding the correlation coefficient is:

  \[t=r\sqrt{\frac{d.f.}{1-r^2}}\]

• Alternatively, a dedicated PPMC table can be used.

  • The PPMC table takes $\alpha$ and $d.f.$ (or $n$) and returns a value that can be directly compared to $r$.
    • If $r$ is beyond the value, H0 is rejected, and a linear relationship is accepted.
    • If $r$ is not beyond the value, H0 is not rejected.

Example

• Question:
  • For the above example data, is $r$ significant for $\alpha=0.05$?
• Answer:
  • $r$ was previously calculated to be -0.589.
  • For $d.f.=7$ and $\alpha=0.05$, the PPMC table returns a value of: $\pm 0.666$.
  • As $r$ is not beyond -0.666, $r$ is not significant, and H0 cannot be rejected.

Regression

Linear Regression

While correlation establishes the strength and direction of the relationship between two variables, regression calculates the actual formula by which one variable can be used to predict the other.

Notation:

• $y’$ is used to represent the predicted value of $y$ based on regression analysis.

Regression Line:

• Line that best fits the scatter plot data

Residual:

• Difference between the predicted $y’$ and the actual $y$.
• The regression line is the line which has the least residuals.

Method of Lease Squares:

• Method of determining the regression line by using the residuals’ squared values
  • Example: Would rather a line with 3 points that are all 1 removed than a line with 1 point that is 2 removed

Coefficient of Determination

Coefficient of Determination:

• Percentage of $y$ variability can be attributed to $x$ variability through linear regression.
• Formula: $r^2$

Calculating the Regression Line

The formula for the regression line is given in slope-intercept form:

where:

• $y’$ is the predicted $y$ value;
• $a$ is the $y’$ intercept; and
• $b$ is the slope of the line.

The formula for calculating $a$ is:

The formula for calculating $b$ is:

Example

• Question:
  • For the above data, calculate $y$ for $x=51$.
• Answer:

  • Substituting the above values in the formula for $a$ gives us:

    \[a= \frac { (40*27121)-(487*2091)}{(9*27121)-(487)^2} \approx 9.613\]

  • Substituting the above values in the formula for $b$ gives us:

    \[b = \frac { (9*2091)-(487*40)} {(9*27121)-(487)^2} \approx -0.096\]

  • Resulting in a linear regression line of:

    \[y'=9.613-0.096x\]

  • Calculating for $x=51$ result in:

    \[y' = 9.613-(0.096*51)=4.717\]

Critical Value for PPMC Chart