Simple Linier Regression

Simple Linear Regression Department of Statistics, ITS Surabaya Slide- Prepared by: Sutikno Department of Statistics Faculty of Mathematics and Natural Sciences Sepuluh Nopember Institute of Technology (ITS) Surabaya

Learning Objectives ,[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Correlation vs. Regression ,[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Introduction to Regression Analysis ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Simple Linear Regression Model ,[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Types of Relationships Department of Statistics, ITS Surabaya Slide- Y X Y X Y Y X X Linear relationships Curvilinear relationships

Types of Relationships Department of Statistics, ITS Surabaya Slide- Y X Y X Y Y X X Strong relationships Weak relationships (continued)

Types of Relationships Department of Statistics, ITS Surabaya Slide- Y X Y X No relationship (continued)

Simple Linear Regression Model Department of Statistics, ITS Surabaya Slide- Linear component Population Y intercept Population Slope Coefficient Random Error term Dependent Variable Independent Variable Random Error component

Simple Linear Regression Model Department of Statistics, ITS Surabaya Slide- (continued) Random Error for this X i value Y X Observed Value of Y for X i Predicted Value of Y for X i X i Slope = β 1 Intercept = β 0 ε i

Simple Linear Regression Equation (Prediction Line) Department of Statistics, ITS Surabaya Slide- The simple linear regression equation provides an estimate of the population regression line Estimate of the regression intercept Estimate of the regression slope Estimated (or predicted) Y value for observation i Value of X for observation i The individual random error terms e i have a mean of zero

Least Squares Method ,[object Object],Department of Statistics, ITS Surabaya Slide-

Finding the Least Squares Equation ,[object Object],Department of Statistics, ITS Surabaya Slide- Formulas are shown in the text for those who are interested

[object Object],[object Object],Interpretation of the Slope and the Intercept Department of Statistics, ITS Surabaya Slide-

Simple Linear Regression Example ,[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Sample Data for House Price Model Department of Statistics, ITS Surabaya Slide- House Price in $1000s (Y) Square Feet (X) 245 1400 312 1600 279 1700 308 1875 199 1100 219 1550 405 2350 324 2450 319 1425 255 1700

Graphical Presentation ,[object Object],Department of Statistics, ITS Surabaya Slide-

Regression Using Excel ,[object Object],Department of Statistics, ITS Surabaya Slide-

Excel Output Department of Statistics, ITS Surabaya Slide- The regression equation is: Regression Statistics Multiple R 0.76211 R Square 0.58082 Adjusted R Square 0.52842 Standard Error 41.33032 Observations 10 ANOVA df SS MS F Significance F Regression 1 18934.9348 18934.9348 11.0848 0.01039 Residual 8 13665.5652 1708.1957 Total 9 32600.5000 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Graphical Presentation ,[object Object],Department of Statistics, ITS Surabaya Slide- Slope = 0.10977 Intercept = 98.248

Interpretation of the Intercept, b 0 ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Interpretation of the Slope Coefficient, b 1 ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Predictions using Regression Analysis Department of Statistics, ITS Surabaya Slide- Predict the price for a house with 2000 square feet: The predicted price for a house with 2000 square feet is 317.85($1,000s) = $317,850

Interpolation vs. Extrapolation ,[object Object],Department of Statistics, ITS Surabaya Slide- Relevant range for interpolation Do not try to extrapolate beyond the range of observed X’s

Measures of Variation ,[object Object],Department of Statistics, ITS Surabaya Slide- Total Sum of Squares Regression Sum of Squares Error Sum of Squares where: = Average value of the dependent variable Y i = Observed values of the dependent variable i = Predicted value of Y for the given X i value

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Measures of Variation Department of Statistics, ITS Surabaya Slide- (continued)

Measures of Variation Department of Statistics, ITS Surabaya Slide- (continued) X i Y X Y i SST =  (Y i - Y ) 2 SSE =  (Y i - Y i ) 2  SSR =  ( Y i - Y ) 2  _ _ _ Y  Y Y _ Y 

[object Object],[object Object],Coefficient of Determination, r 2 Department of Statistics, ITS Surabaya Slide- note:

Examples of Approximate r 2 Values Department of Statistics, ITS Surabaya Slide- r 2 = 1 Y X Y X r 2 = 1 r 2 = 1 Perfect linear relationship between X and Y: 100% of the variation in Y is explained by variation in X

Examples of Approximate r 2 Values Department of Statistics, ITS Surabaya Slide- Y X Y X 0 < r 2 < 1 Weaker linear relationships between X and Y: Some but not all of the variation in Y is explained by variation in X

Examples of Approximate r 2 Values Department of Statistics, ITS Surabaya Slide- r 2 = 0 No linear relationship between X and Y: The value of Y does not depend on X. (None of the variation in Y is explained by variation in X) Y X r 2 = 0

Excel Output Department of Statistics, ITS Surabaya Slide- 58.08% of the variation in house prices is explained by variation in square feet Regression Statistics Multiple R 0.76211 R Square 0.58082 Adjusted R Square 0.52842 Standard Error 41.33032 Observations 10 ANOVA df SS MS F Significance F Regression 1 18934.9348 18934.9348 11.0848 0.01039 Residual 8 13665.5652 1708.1957 Total 9 32600.5000 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Standard Error of Estimate ,[object Object],Department of Statistics, ITS Surabaya Slide- Where SSE = error sum of squares n = sample size

Excel Output Department of Statistics, ITS Surabaya Slide- Regression Statistics Multiple R 0.76211 R Square 0.58082 Adjusted R Square 0.52842 Standard Error 41.33032 Observations 10 ANOVA df SS MS F Significance F Regression 1 18934.9348 18934.9348 11.0848 0.01039 Residual 8 13665.5652 1708.1957 Total 9 32600.5000 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Comparing Standard Errors Department of Statistics, ITS Surabaya Slide- Y Y X X S YX is a measure of the variation of observed Y values from the regression line The magnitude of S YX should always be judged relative to the size of the Y values in the sample data i.e., S YX = $41.33K is moderately small relative to house prices in the $200 - $300K range

Assumptions of Regression ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Residual Analysis ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Residual Analysis for Linearity Department of Statistics, ITS Surabaya Slide- Not Linear Linear  x residuals x Y x Y x residuals

Department of Statistics, ITS Surabaya Slide- Residual Analysis for Independence Not Independent Independent X X residuals residuals X residuals 

Residual Analysis for Normality Department of Statistics, ITS Surabaya Slide- Percent Residual ,[object Object],-3 -2 -1 0 1 2 3 0 100

Residual Analysis for Equal Variance Department of Statistics, ITS Surabaya Slide- Non-constant variance  Constant variance x x Y x x Y residuals residuals

Excel Residual Output Department of Statistics, ITS Surabaya Slide- Does not appear to violate any regression assumptions RESIDUAL OUTPUT Predicted House Price Residuals 1 251.92316 -6.923162 2 273.87671 38.12329 3 284.85348 -5.853484 4 304.06284 3.937162 5 218.99284 -19.99284 6 268.38832 -49.38832 7 356.20251 48.79749 8 367.17929 -43.17929 9 254.6674 64.33264 10 284.85348 -29.85348

[object Object],[object Object],Measuring Autocorrelation: The Durbin-Watson Statistic Department of Statistics, ITS Surabaya Slide-

Autocorrelation ,[object Object],Department of Statistics, ITS Surabaya Slide- ,[object Object],[object Object]

The Durbin-Watson Statistic ,[object Object],Department of Statistics, ITS Surabaya Slide- ,[object Object],[object Object],[object Object],H 0 : residuals are not correlated H 1 : positive autocorrelation is present

Testing for Positive Autocorrelation Department of Statistics, ITS Surabaya Slide- ,[object Object],[object Object],Decision rule: reject H 0 if D < d L H 0 : positive autocorrelation does not exist H 1 : positive autocorrelation is present 0 d U 2 d L Reject H 0 Do not reject H 0 ,[object Object],[object Object],Inconclusive

[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- Testing for Positive Autocorrelation (continued)

[object Object],Testing for Positive Autocorrelation Department of Statistics, ITS Surabaya Slide- (continued) Excel/PHStat output: Durbin-Watson Calculations Sum of Squared Difference of Residuals 3296.18 Sum of Squared Residuals 3279.98 Durbin-Watson Statistic 1.00494

[object Object],[object Object],[object Object],[object Object],Testing for Positive Autocorrelation Department of Statistics, ITS Surabaya Slide- (continued) Decision: reject H 0 since D = 1.00494 < d L 0 d U =1.45 2 d L =1.29 Reject H 0 Do not reject H 0 Inconclusive

Inferences About the Slope ,[object Object],Department of Statistics, ITS Surabaya Slide- where: = Estimate of the standard error of the least squares slope = Standard error of the estimate

Comparing Standard Errors of the Slope Department of Statistics, ITS Surabaya Slide- Y X Y X is a measure of the variation in the slope of regression lines from different possible samples

Inference about the Slope: t Test ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- where: b 1 = regression slope coefficient β 1 = hypothesized slope S b = standard error of the slope 1

Inference about the Slope: t Test Department of Statistics, ITS Surabaya Slide- Simple Linear Regression Equation: The slope of this model is 0.1098 Does square footage of the house affect its sales price? (continued) House Price in $1000s (y) Square Feet (x) 245 1400 312 1600 279 1700 308 1875 199 1100 219 1550 405 2350 324 2450 319 1425 255 1700

Inferences about the Slope: t Test Example ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- From Excel output: t b 1 Coefficients Standard Error t Stat P-value Intercept 98.24833 58.03348 1.69296 0.12892 Square Feet 0.10977 0.03297 3.32938 0.01039

Inferences about the Slope: t Test Example ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- Test Statistic: t = 3.329 There is sufficient evidence that square footage affects house price From Excel output: Reject H 0 t b 1 Decision: Conclusion: Reject H 0 Reject H 0  /2=.025 -t α /2 Do not reject H 0 0 t α /2  /2=.025 -2.3060 2.3060 3.329 d.f. = 10-2 = 8 (continued) Coefficients Standard Error t Stat P-value Intercept 98.24833 58.03348 1.69296 0.12892 Square Feet 0.10977 0.03297 3.32938 0.01039

Inferences about the Slope: t Test Example ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- P-value = 0.01039 There is sufficient evidence that square footage affects house price From Excel output: Reject H 0 P-value Decision: P-value < α so Conclusion: (continued) This is a two-tail test, so the p-value is P(t > 3.329)+P(t < -3.329) = 0.01039 (for 8 d.f.) Coefficients Standard Error t Stat P-value Intercept 98.24833 58.03348 1.69296 0.12892 Square Feet 0.10977 0.03297 3.32938 0.01039

F Test for Significance ,[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- where F follows an F distribution with k numerator and (n – k - 1) denominator degrees of freedom (k = the number of independent variables in the regression model)

Excel Output Department of Statistics, ITS Surabaya Slide- With 1 and 8 degrees of freedom P-value for the F Test Regression Statistics Multiple R 0.76211 R Square 0.58082 Adjusted R Square 0.52842 Standard Error 41.33032 Observations 10 ANOVA df SS MS F Significance F Regression 1 18934.9348 18934.9348 11.0848 0.01039 Residual 8 13665.5652 1708.1957 Total 9 32600.5000 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

[object Object],[object Object],[object Object],[object Object],F Test for Significance Department of Statistics, ITS Surabaya Slide- Test Statistic: Decision: Conclusion: Reject H 0 at  = 0.05 There is sufficient evidence that house size affects selling price 0  = .05 F .05 = 5.32 Reject H 0 Do not reject H 0 Critical Value: F  = 5.32 (continued) F

Confidence Interval Estimate for the Slope Department of Statistics, ITS Surabaya Slide- Confidence Interval Estimate of the Slope: Excel Printout for House Prices: At 95% level of confidence, the confidence interval for the slope is (0.0337, 0.1858) d.f. = n - 2 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Confidence Interval Estimate for the Slope Department of Statistics, ITS Surabaya Slide- Since the units of the house price variable is $1000s, we are 95% confident that the average impact on sales price is between $33.70 and $185.80 per square foot of house size This 95% confidence interval does not include 0 . Conclusion: There is a significant relationship between house price and square feet at the .05 level of significance (continued) Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

t Test for a Correlation Coefficient ,[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Example: House Prices Department of Statistics, ITS Surabaya Slide- Is there evidence of a linear relationship between square feet and house price at the .05 level of significance? H 0 : ρ = 0 (No correlation) H 1 : ρ ≠ 0 (correlation exists)  =.05 , df = 10 - 2 = 8

Example: Test Solution Department of Statistics, ITS Surabaya Slide- Conclusion: There is evidence of a linear association at the 5% level of significance Decision: Reject H 0 Reject H 0 Reject H 0  /2=.025 -t α /2 Do not reject H 0 0 t α /2  /2=.025 -2.3060 2.3060 3.329 d.f. = 10-2 = 8

Estimating Mean Values and Predicting Individual Values Department of Statistics, ITS Surabaya Slide- Y X X i Y = b 0 +b 1 X i  Confidence Interval for the mean of Y, given X i Prediction Interval for an individual Y , given X i Goal: Form intervals around Y to express uncertainty about the value of Y for a given X i Y 

Confidence Interval for the Average Y, Given X Department of Statistics, ITS Surabaya Slide- Confidence interval estimate for the mean value of Y given a particular X i Size of interval varies according to distance away from mean, X

Prediction Interval for an Individual Y, Given X Department of Statistics, ITS Surabaya Slide- Confidence interval estimate for an Individual value of Y given a particular X i This extra term adds to the interval width to reflect the added uncertainty for an individual case

Estimation of Mean Values: Example Department of Statistics, ITS Surabaya Slide- Find the 95% confidence interval for the mean price of 2,000 square-foot houses Predicted Price Y i = 317.85 ($1,000s)  Confidence Interval Estimate for μ Y|X=X The confidence interval endpoints are 280.66 and 354.90, or from $280,660 to $354,900 i

Estimation of Individual Values: Example Department of Statistics, ITS Surabaya Slide- Find the 95% prediction interval for an individual house with 2,000 square feet Predicted Price Y i = 317.85 ($1,000s)  Prediction Interval Estimate for Y X=X The prediction interval endpoints are 215.50 and 420.07, or from $215,500 to $420,070 i

Finding Confidence and Prediction Intervals in Excel ,[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

[object Object],Finding Confidence and Prediction Intervals in Excel Department of Statistics, ITS Surabaya Slide- (continued) Confidence Interval Estimate for μ Y|X=Xi Prediction Interval Estimate for Y X=Xi Y 

Pitfalls of Regression Analysis ,[object Object],[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Strategies for Avoiding the Pitfalls of Regression ,[object Object],[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide-

Strategies for Avoiding the Pitfalls of Regression ,[object Object],[object Object],[object Object],Department of Statistics, ITS Surabaya Slide- (continued)

Simple Linier Regression

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Simple Linier Regression

Semelhante a Simple Linier Regression (20)

Mais de dessybudiyanti

Mais de dessybudiyanti (20)

Último

Último (20)

Simple Linier Regression