Supplemental Notes for 39002: Statistical Methods in Geography (Sheridan)
Chapter 7: More on Regression - Page 1
It’s easy to begin explaining the concept of regression via simple linear
(bivariate) regression, as we did last chapter. However, a more common
type of regression we see is called multiple regression, which refers to the
fact that we check how two or more independent variables affect some
dependent variable.
The concept of independent and dependent variables is the same, there’s
just more than one independent one. The concept of the predictive
equation is also the same, though it’s a bit longer:
...
ˆ
332211
xbxbxbay
for as many independent variables (the x’s) as you have. Last time, we had
two variables, and so graphically we fitted a line between them. If we have
three variables (two independent ones), it’s like fitting a flat plane on a 3-D
grid (Figure 7.1 in your book gives you a good idea about what this looks
like.) Beyond this it can’t be visualized, but the principle is the same.
1. Multicollinearity
So let’s say we have a dependent variable, and a hundred independent
variables. Can we enter all hundred? Sure, but is it useful? Most likely
not. The problem is an issue called multicollinearity, which means that the
other independent variables themselves are highly correlated.
Say we have summertime ice cream sales as our dependent variable, and
2 pm temperature and 3 pm temperature as our independent ones. If it’s
hot at 2 pm it’s going to be also hot at 3 pm most likely, so what additional
information does this provide? Not much. Worse however, is that when
you include two very similar variables, the coefficients in the regression
equation are often difficult to interpret, sometimes counterintuitive, and will
fluctuate significantly with one additional or fewer observation.
2. What do the coefficients mean?
In bivariate regression, it’s easier to interpret the coefficient, as there is only
one independent variable. But with multiple coefficients, you have to be
careful. Consider the example from the book. Considered separately,
House price = -57,809 + 36.2 (Lot size)
