SlideShare a Scribd company logo
1 of 6
Expected Values and Variances of OLS Estimators:

We defined the population model                   , and we claimed that the
key assumption for simple regression analysis to be useful is that the
expected value of u given any value of x is zero.

We discussed the algebraic properties of OLS estimation. We now return to
the population model and study the statistical properties of OLS.

In other words, we now view              as estimators for the parameters
ļæ½ļæ½ļæ½         that appear in the population model.

Assumptions for unbiased OLS:

SLR 1.

Liner in parameters:



SLR 2.

We use random sample size for the population model

                                             i= 1,2,3ā€¦ā€¦ā€¦.n.

where, is the error or disturbance for observation i(for example, person i,
firm i, city i, etc.).

Thus     , contains the unobservables for observation affects the      .

The      should not be confused with the   that was defined earlier.

Discussion between the errors and residuals will be covered later.

SLR 3.

Zero Conditional Mean:



For a random sample this assumption implies:
SLR 4.

The sample variation in the independent variable:




This means if = wage and = education then SLR. 4 fail only if every one in
the sample has the same amount of education. This is hardly true!

USING SLR1.- SLR4,



for any values of          In other words            are unbiased estimates for


Variance in OLS estimators:

Once we know that        are unbiased estimates for               we must also
know how far do we expect to be from on an average.

Among other things this allows us to choose the best estimator among all, or at
least a broad class of unbiased estimators.

SLR 5.



This is the homoskedasity Assumption.

This assumption plays no role in showing              are unbiased estimators
ofļæ½ļæ½ļæ½

  is often called the error variance or disturbance variance.
** Note

Under Assumption SLR.1 through SLR.5




Where these are conditional on the sample values



Note: All the quantities of entering in the preceding equations except   can
be estimated from the data.
But variance can be estimated using the following formula (IF YOU WANT
REFER TO APPENDIX 3.A)




Estimating the Error in Variance:

So far we know that:

                               And

                           .

Difference between errors (or disturbance)and residuals is crucial for
constructing .

Population model in terms of randomly observed sample can be written as:



and     is the ERROR for observation

 in terms of fitted value can be expressed as:



and     is the RESIDUAL for observation

Thus:

We saw previously that for OLS to be unbiased                      .

But                            .

The difference between them does not have a zero expected value.

Now returning to    :
Thus the unbiased ā€œestimatorā€ for       is:

But since we do not observer          and observe only the OLS residual of
                  .

This is the true estimator, because it gives a complete rule for any sample data
on on

One slight drawback to this estimator is that it turns out to be biased
(although for large the bias is small).

The         is biased only because it does not account for two restrictions that
OLS satisfies:




Since there are only n-2 degrees of freedom in OLS residuals (as opposed to
degrees of freedom in errors)

If we apply the restrictions in are replace     with    the above restrictions
would no longer hold.

The unbiased estimator of            we will use makes degrees-of-freedom
adjustment:




*This estimator is also denoted as

Properties of OLS Estimators:

If assumption 1 through 4 hold then the estimators              determined by
OLS are known as Best Liner Unbiased Estimates (BLUE).



What does BLUE stand for?
"Estimator" -   is an estimator of true value of .

"Linear"-   is linear in parameter.

"Unbiased" ā€“ On average, the actual value of the        will represent the
true values.

"Best" ā€“ Means of OLS estimator has minimum variance among the class of
linear unbiased estimators. The Gauss ā€“ Markov theorem provides of that of
OLS estimator is best.

**Note

More Related Content

What's hot

Multicollinearity1
Multicollinearity1Multicollinearity1
Multicollinearity1
Muhammad Ali
Ā 
Multivariate reg analysis
Multivariate reg analysisMultivariate reg analysis
Multivariate reg analysis
Irfan Hussain
Ā 
Business Quantitative Lecture 3
Business Quantitative Lecture 3Business Quantitative Lecture 3
Business Quantitative Lecture 3
saark
Ā 
Econometrics chapter 8
Econometrics chapter 8Econometrics chapter 8
Econometrics chapter 8
Sehrish Chaudary
Ā 
Dependance Technique, Regression & Correlation
Dependance Technique, Regression & Correlation Dependance Technique, Regression & Correlation
Dependance Technique, Regression & Correlation
Qasim Raza
Ā 
Linear regression
Linear regressionLinear regression
Linear regression
DepEd
Ā 
Business Quantitative - Lecture 2
Business Quantitative - Lecture 2Business Quantitative - Lecture 2
Business Quantitative - Lecture 2
saark
Ā 

What's hot (19)

Multicolinearity
MulticolinearityMulticolinearity
Multicolinearity
Ā 
Multicollinearity
MulticollinearityMulticollinearity
Multicollinearity
Ā 
Multicollinearity1
Multicollinearity1Multicollinearity1
Multicollinearity1
Ā 
Multicollinearity PPT
Multicollinearity PPTMulticollinearity PPT
Multicollinearity PPT
Ā 
Autocorrelation
AutocorrelationAutocorrelation
Autocorrelation
Ā 
Multivariate reg analysis
Multivariate reg analysisMultivariate reg analysis
Multivariate reg analysis
Ā 
Multivariate analysis - Multiple regression analysis
Multivariate analysis -  Multiple regression analysisMultivariate analysis -  Multiple regression analysis
Multivariate analysis - Multiple regression analysis
Ā 
Auto Correlation Presentation
Auto Correlation PresentationAuto Correlation Presentation
Auto Correlation Presentation
Ā 
Business Quantitative Lecture 3
Business Quantitative Lecture 3Business Quantitative Lecture 3
Business Quantitative Lecture 3
Ā 
Econometrics chapter 8
Econometrics chapter 8Econometrics chapter 8
Econometrics chapter 8
Ā 
Regression presentation
Regression presentationRegression presentation
Regression presentation
Ā 
Dependance Technique, Regression & Correlation
Dependance Technique, Regression & Correlation Dependance Technique, Regression & Correlation
Dependance Technique, Regression & Correlation
Ā 
Introduction to regression analysis 2
Introduction to regression analysis 2Introduction to regression analysis 2
Introduction to regression analysis 2
Ā 
Regression analysis made easy
Regression analysis made easyRegression analysis made easy
Regression analysis made easy
Ā 
Regression
RegressionRegression
Regression
Ā 
Data Science - Part IV - Regression Analysis & ANOVA
Data Science - Part IV - Regression Analysis & ANOVAData Science - Part IV - Regression Analysis & ANOVA
Data Science - Part IV - Regression Analysis & ANOVA
Ā 
Linear regression
Linear regressionLinear regression
Linear regression
Ā 
Business Quantitative - Lecture 2
Business Quantitative - Lecture 2Business Quantitative - Lecture 2
Business Quantitative - Lecture 2
Ā 
Dependence Techniques
Dependence Techniques Dependence Techniques
Dependence Techniques
Ā 

Similar to 2.3 the simple regression model

Similar to 2.3 the simple regression model (20)

Data Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic NetsData Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Ā 
CH3.pdf
CH3.pdfCH3.pdf
CH3.pdf
Ā 
Eviews forecasting
Eviews forecastingEviews forecasting
Eviews forecasting
Ā 
ders 8 Quantile-Regression.ppt
ders 8 Quantile-Regression.pptders 8 Quantile-Regression.ppt
ders 8 Quantile-Regression.ppt
Ā 
MF Presentation.pptx
MF Presentation.pptxMF Presentation.pptx
MF Presentation.pptx
Ā 
Bus 173_6.pptx
Bus 173_6.pptxBus 173_6.pptx
Bus 173_6.pptx
Ā 
Interpreting Regression Results - Machine Learning
Interpreting Regression Results - Machine LearningInterpreting Regression Results - Machine Learning
Interpreting Regression Results - Machine Learning
Ā 
Unit 05 - Consolidated.pptx
Unit 05 - Consolidated.pptxUnit 05 - Consolidated.pptx
Unit 05 - Consolidated.pptx
Ā 
Multiple regression
Multiple regressionMultiple regression
Multiple regression
Ā 
Regression and Classification Analysis
Regression and Classification AnalysisRegression and Classification Analysis
Regression and Classification Analysis
Ā 
Dem 7263 fall 2015 spatially autoregressive models 1
Dem 7263 fall 2015   spatially autoregressive models 1Dem 7263 fall 2015   spatially autoregressive models 1
Dem 7263 fall 2015 spatially autoregressive models 1
Ā 
chapter_7.pptx
chapter_7.pptxchapter_7.pptx
chapter_7.pptx
Ā 
Stats ca report_18180485
Stats ca report_18180485Stats ca report_18180485
Stats ca report_18180485
Ā 
pdf (9).pdf
pdf (9).pdfpdf (9).pdf
pdf (9).pdf
Ā 
Ch5_OLSasymptotic.pdf
Ch5_OLSasymptotic.pdfCh5_OLSasymptotic.pdf
Ch5_OLSasymptotic.pdf
Ā 
Multivariate Regression Analysis.pptx
Multivariate Regression Analysis.pptxMultivariate Regression Analysis.pptx
Multivariate Regression Analysis.pptx
Ā 
Binary OR Binomial logistic regression
Binary OR Binomial logistic regression Binary OR Binomial logistic regression
Binary OR Binomial logistic regression
Ā 
Chapter4_Multi_Reg_Estim.pdf.pdf
Chapter4_Multi_Reg_Estim.pdf.pdfChapter4_Multi_Reg_Estim.pdf.pdf
Chapter4_Multi_Reg_Estim.pdf.pdf
Ā 
linear model multiple predictors.pdf
linear model multiple predictors.pdflinear model multiple predictors.pdf
linear model multiple predictors.pdf
Ā 
Introduction to Limited Dependent variable
Introduction to Limited Dependent variableIntroduction to Limited Dependent variable
Introduction to Limited Dependent variable
Ā 

More from Regmi Milan

Prespective On Chinese Financial System and policy-reforms-
Prespective On Chinese Financial System and policy-reforms-Prespective On Chinese Financial System and policy-reforms-
Prespective On Chinese Financial System and policy-reforms-
Regmi Milan
Ā 
E-Commerce -Note -2
E-Commerce -Note -2E-Commerce -Note -2
E-Commerce -Note -2
Regmi Milan
Ā 
E-Commerce-Note-1_MR
E-Commerce-Note-1_MRE-Commerce-Note-1_MR
E-Commerce-Note-1_MR
Regmi Milan
Ā 
GATT & WTO : History and Prospective of Nepal.
GATT & WTO : History and  Prospective of Nepal.GATT & WTO : History and  Prospective of Nepal.
GATT & WTO : History and Prospective of Nepal.
Regmi Milan
Ā 
Nepal japan project_2013
Nepal japan project_2013Nepal japan project_2013
Nepal japan project_2013
Regmi Milan
Ā 
Chitwan overview
Chitwan overviewChitwan overview
Chitwan overview
Regmi Milan
Ā 
Lecture on public finance ( abridged version)
Lecture on public finance ( abridged version)Lecture on public finance ( abridged version)
Lecture on public finance ( abridged version)
Regmi Milan
Ā 

More from Regmi Milan (20)

Work place violence
Work place violenceWork place violence
Work place violence
Ā 
(C) Regmi_Public Private Partnership
(C) Regmi_Public Private Partnership(C) Regmi_Public Private Partnership
(C) Regmi_Public Private Partnership
Ā 
Prespective On Chinese Financial System and policy-reforms-
Prespective On Chinese Financial System and policy-reforms-Prespective On Chinese Financial System and policy-reforms-
Prespective On Chinese Financial System and policy-reforms-
Ā 
E-Commerce-Chapter-4_MR
E-Commerce-Chapter-4_MRE-Commerce-Chapter-4_MR
E-Commerce-Chapter-4_MR
Ā 
Project M&E (unit 1-4)
Project M&E (unit 1-4)Project M&E (unit 1-4)
Project M&E (unit 1-4)
Ā 
E-Commerce -Note -2
E-Commerce -Note -2E-Commerce -Note -2
E-Commerce -Note -2
Ā 
E-Commerce-Note-1_MR
E-Commerce-Note-1_MRE-Commerce-Note-1_MR
E-Commerce-Note-1_MR
Ā 
Micro Hydro Schemes : Case of Ghandruk VDC, Nepal
Micro Hydro Schemes : Case of Ghandruk VDC, Nepal Micro Hydro Schemes : Case of Ghandruk VDC, Nepal
Micro Hydro Schemes : Case of Ghandruk VDC, Nepal
Ā 
Ghandruk
GhandrukGhandruk
Ghandruk
Ā 
Insights in Economic Development Of Nepal & Early United States Assistance to...
Insights in Economic Development Of Nepal & Early United States Assistance to...Insights in Economic Development Of Nepal & Early United States Assistance to...
Insights in Economic Development Of Nepal & Early United States Assistance to...
Ā 
GATT & WTO : History and Prospective of Nepal.
GATT & WTO : History and  Prospective of Nepal.GATT & WTO : History and  Prospective of Nepal.
GATT & WTO : History and Prospective of Nepal.
Ā 
HDI
HDIHDI
HDI
Ā 
Nepal japan project_2013
Nepal japan project_2013Nepal japan project_2013
Nepal japan project_2013
Ā 
Chitwan overview
Chitwan overviewChitwan overview
Chitwan overview
Ā 
Optical fibers
Optical fibersOptical fibers
Optical fibers
Ā 
Pokhara- Field Presentation On Thematic Areas
Pokhara-  Field Presentation On Thematic AreasPokhara-  Field Presentation On Thematic Areas
Pokhara- Field Presentation On Thematic Areas
Ā 
Principle of abiity to pay
Principle of  abiity to payPrinciple of  abiity to pay
Principle of abiity to pay
Ā 
Lecture on public finance ( abridged version)
Lecture on public finance ( abridged version)Lecture on public finance ( abridged version)
Lecture on public finance ( abridged version)
Ā 
Final study report for publication december 17, 2009
Final study report for publication december 17, 2009Final study report for publication december 17, 2009
Final study report for publication december 17, 2009
Ā 
Annex 2 national micro finance policy
Annex 2 national micro finance policyAnnex 2 national micro finance policy
Annex 2 national micro finance policy
Ā 

Recently uploaded

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(ā˜Žļø+971_581248768%)**%*]'#abortion pills for sale in dubai@
Ā 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
Ā 

Recently uploaded (20)

MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
Ā 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
Ā 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Ā 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Ā 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
Ā 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Ā 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
Ā 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
Ā 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Ā 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
Ā 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Ā 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Ā 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Ā 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
Ā 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Ā 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Ā 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Ā 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
Ā 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
Ā 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Ā 

2.3 the simple regression model

  • 1. Expected Values and Variances of OLS Estimators: We defined the population model , and we claimed that the key assumption for simple regression analysis to be useful is that the expected value of u given any value of x is zero. We discussed the algebraic properties of OLS estimation. We now return to the population model and study the statistical properties of OLS. In other words, we now view as estimators for the parameters ļæ½ļæ½ļæ½ that appear in the population model. Assumptions for unbiased OLS: SLR 1. Liner in parameters: SLR 2. We use random sample size for the population model i= 1,2,3ā€¦ā€¦ā€¦.n. where, is the error or disturbance for observation i(for example, person i, firm i, city i, etc.). Thus , contains the unobservables for observation affects the . The should not be confused with the that was defined earlier. Discussion between the errors and residuals will be covered later. SLR 3. Zero Conditional Mean: For a random sample this assumption implies:
  • 2. SLR 4. The sample variation in the independent variable: This means if = wage and = education then SLR. 4 fail only if every one in the sample has the same amount of education. This is hardly true! USING SLR1.- SLR4, for any values of In other words are unbiased estimates for Variance in OLS estimators: Once we know that are unbiased estimates for we must also know how far do we expect to be from on an average. Among other things this allows us to choose the best estimator among all, or at least a broad class of unbiased estimators. SLR 5. This is the homoskedasity Assumption. This assumption plays no role in showing are unbiased estimators ofļæ½ļæ½ļæ½ is often called the error variance or disturbance variance.
  • 3. ** Note Under Assumption SLR.1 through SLR.5 Where these are conditional on the sample values Note: All the quantities of entering in the preceding equations except can be estimated from the data.
  • 4. But variance can be estimated using the following formula (IF YOU WANT REFER TO APPENDIX 3.A) Estimating the Error in Variance: So far we know that: And . Difference between errors (or disturbance)and residuals is crucial for constructing . Population model in terms of randomly observed sample can be written as: and is the ERROR for observation in terms of fitted value can be expressed as: and is the RESIDUAL for observation Thus: We saw previously that for OLS to be unbiased . But . The difference between them does not have a zero expected value. Now returning to :
  • 5. Thus the unbiased ā€œestimatorā€ for is: But since we do not observer and observe only the OLS residual of . This is the true estimator, because it gives a complete rule for any sample data on on One slight drawback to this estimator is that it turns out to be biased (although for large the bias is small). The is biased only because it does not account for two restrictions that OLS satisfies: Since there are only n-2 degrees of freedom in OLS residuals (as opposed to degrees of freedom in errors) If we apply the restrictions in are replace with the above restrictions would no longer hold. The unbiased estimator of we will use makes degrees-of-freedom adjustment: *This estimator is also denoted as Properties of OLS Estimators: If assumption 1 through 4 hold then the estimators determined by OLS are known as Best Liner Unbiased Estimates (BLUE). What does BLUE stand for?
  • 6. "Estimator" - is an estimator of true value of . "Linear"- is linear in parameter. "Unbiased" ā€“ On average, the actual value of the will represent the true values. "Best" ā€“ Means of OLS estimator has minimum variance among the class of linear unbiased estimators. The Gauss ā€“ Markov theorem provides of that of OLS estimator is best. **Note