Advanced Regression Techniques

There are several ways in which the simplest linear regression method may be modified, depending on the data available and what you want to do with it.

How good is your data? Correcting for 'dodgy' datapoints

One of these techniques gives greater weight to the most reliable (or accurate) data points; in other words, you can force the statistical package to allow these points to have a greater influence over the line that is fitted. Unfortunately, it is up to you to decide whether this is necessary.

Multiple linear regression

Multiple linear regression is an obvious extension to the simple regression shown, and is the inclusion of two or more explanatory factors. For example, the dataset data2.txt contains 3 columns: x1, x2 and y, where y is presumed to be explained by both x1 and x2. Note that there must be some interaction between x1 and x2 for them both to determine y. Regression with two explanatory factors is effectively the fitting of a line to the 'cloud' of points you would get when the points are plotted in 3-dimensional space (an axis for each of 2 explanatory factors and the response variable). This is harder to picture than with one explanatory variable, especially when a line is fitted to more than 2 explanatory variables!

An example of this is shown in Figure 3 below.


	Figure 3. A 3-D scatter plot of y against x1 and x2 (data from data2.txt). Red dots mark the location of points, while black lines to the x1/x2 plane help to show their relative positions.

The regression of y on x1 and x2 in Minitab is shown here. Notice the similarities with the previous output. The regression equation has been expanded to:

y = 9.31 + 0.787 x1 - 0.044 x2,

and we now have an equation that will describe what y will be when given the values of x1 and x2. However, in the output you can see that the p value for x2 is >0.05: this high value suggests that x1 better explains y than x2, though a step-wise regression could be used to confirm this. Step-wise regression is a technique where explanatory variables are included or excluded in the regression such that their relative contribution to the overall explanation of the response variable may be determined. The sequence by which variables are included and/or dropped from the regression is determined by the user and is crucial, especially with larger numbers of explanatory variables.

Descriptive Stats
Diversity Indices
Comparisons
Correlations
Regression

Ted Gaten Department of Biology gat@le.ac.uk

Entry approved by the Head of Department. Last Updated: May 2000

On-line statistics

Linear Regression 2

Advanced Regression Techniques

Descriptive Stats Diversity Indices Comparisons Correlations Regression

Descriptive Stats
Diversity Indices
Comparisons
Correlations
Regression