15.4 Linear and Polynomial Regression
Linear and polynomial regression calculate the best-fit line for one or more XY datasets. In addition, multiple linear regression can be used to study the relationship between several predictor variables and a response variable.
Parameters are estimated using a weighted least-square method. The weight can be given to dependent variable in fitting to reduce the influence of the high leverage points. After fitting, the model is evaluated using hypothesis tests and plots of residuals.
How To Perform the Fitting
To perform linear regression on data with X/Y Error
- The Errors can exist for both dependent and independent values, Errors of dependent variable Y can be treated as weight in all Fitting Tools above by setting the Y Error column as Y Error in Input Data and designate the method of Error as Weight in Fit Control.
- However, if you want to treat errors of independent variable X as weight, the Fit Linear with X Error tool should be used, in which you can set both X and Y Errors in Input Data, which can be treated as weight. See this page for more informations.
To perform linear/polynomial fit with parameters fixed
- Fitting parameters can be fixed in tools above, For example, you can set the Intercept value to 0 by checking on the Fix Intercept in Fit Control dialog and set the Fix Intercept at = 0, which force the fitted line go through the origin point (0,0). Besides, the Model Degree of Freedom will be reduced due to the parameter fixed.
To perform linear/polynomial fit on a graph according to current axis settings
- You can perform linear/polynomial fit on a graph according to current axis settings by checking on the Apparent Fit in Fit Control.
- Apparent Fit will first transform your raw data into a new data space as specified by the graph axis type, and then fit the curve of the new data.
- Here is an example of Finding the slope on the log-log plot by Apparent Fit
To define the x data type for the fitted curve plot
- You can defined the X axis scales, fitted curves interval and range in Fitted Curve Plot: X data Types
Advice and Tips
To force fit curve through specified point
- Two methods are provided to make fitted curve go through certain points for Linear and Polynomial Regression:
- To force the fitted curve go through Origin (0,0), you can just fix the intercept to 0 for a linear or polynomial model.
- To force the fitted curve go through a specific point in raw data, you can set a higher weight for the point.
- For further information, please view this page.
To perform multiple linear regression with boundary or constraint
- You can define a multiple linear regression function and set Constrain for it in the Nonlinear Curve Fit tool.
- Please refer to this page for details.
To identify the outliers in fitting process
- When we get the fitted curve, there may be a large difference between a few points and the fitted curve by the model, these points should be identified as Outliers.
- In Linear Fit, the outliers can be shown in results table by checking on the Outliers checkbox in the Linear Fit dialog. Or, you can use the Q-Q plot to identify an outlier, which is introduced here. You can also exculde the outliers from an analysis routine.
To Find X from Y or Y from X
- The Find X/Y table allows you to obtain either a dependent variable value given an independent variable value, or an independent variable value given a dependent variable value, from the fit you performed on the data.
Interpreting the Results
See Interpreting Regression Results Chapter for following information
Algorithms (Linear Regression)
Algorithms (Fit Linear with X Error)
Algorithm (Multiple Linear Regression)
Algorithms (Polynomial Regression)
Advanced: Linear fit for nonlinear model
You can get an analytical solution of an equation if the equation has multiple terms with linear parameters.
For example:the polynomial equation：
or some more general formula
Instead of using the Nonlinear Curve Fit tool to get an iterative solution, you can treat the linear terms as different independent variables (such as , etc.) and use the Multiple Linear Regression or Ploynomial Fit to perform regression with multiple linear terms and obtain an analytical solution. Here is a case study of How to transfer a reciprocal function into a equation with linear parameters.
However, due to different contributions to residual sum of squares (RSS) for points in two models, the estimate results for parameters is different between nonlinear fit directly and transformed fit with linear way. Please view this page for details.