Simple Linear Regression
Simple Linear Regression
Introduction
Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous variables. Today I will introduce you some basic concept and procedures , talk about the limit of simple linear regression model.
Model Specification
The X-variable is called the explanatory or pre-dictor variable, while the Y-variable is called the response variable or the dependent variable.
True Model: Yi = E(Y|X=x) + ei = b0 +b1xi+ei
where ei is the random error in Yi and is such that E(e | X) = 0, i = 1,…,n. It has idd model assumptions.
ei = Yi - (b0 +b1xi) = Yi - unknown regression line at xi.
- Identically: ei (i = 1,…, n ) have a common variance
- Independently: ei (i = 1,…,n ) are independent of each other
- Distributed: ei (i = 1,…,n ) are normally distributed with a mean of 0 and variance s2
Estimated Model: Yi _hat = b0_hat +b1_hatxi
Where (i = 1,…, n ).
Since b0 and b1 are unknown all we can do is estimate these errors by replacing b0_hat
and b1 _hat by their respective least squares estimates b0 and b1 giving the residuals
eˆ =Y −(b0_hat +b1_hat x)=Yi − estimated regression line at x.
Hypothesis Testing
P-value = P(|T|>|t|)
Reject H0 if p-value < a
R Code
(form STA302 assignment 1)
The explanatory variable is height (in cm) and response variable is forearm length (in cm).
We find that the p-value is 0.6250 which is greater than the benchmark significance level 0.05. In this case, we do not reject H0 in 5% significant level.
Limit
If we want to use this model, we have to satisfy the model assumption. Also, the simple linear regression model is only for estimating the relation between two continuous variables. We can’t use it for multiple variables or categorical variables.
Reference
https://online.stat.psu.edu/stat501/lesson/1
Sheather, S. J. (2009). A modern approach to regression with R.
https://towardsdatascience.com/how-are-logistic-regression-ordinary-least-squares-regression-related-1deab32d79f5
Very good explanation！
ReplyDeleteVery Helpful!
ReplyDeleteUseful resources!
ReplyDeleteGlad to hear it is helpful!
ReplyDeleteIt's useful for my statistical learning! Thanks.
ReplyDeletewow This is what I am looking for! Excellent explain for the simple linear regression. SAVE MY LIFE
ReplyDeleteIt’s a very detailed introduction about simple liner regression especially for those who just learned statistical. Good job!
ReplyDeleteThanks! We can learn them from STA302 and STA303. They are good courses.
DeleteI like your attached picture of the simple linear regression model equation. All variables are well defined, and it is very clear and easy to understand the terminology.
ReplyDeleteVery helpful! It summarizes the key ideas and explanation of simple linear regression. This blog could help me to understand the topic further.
ReplyDeletelike your post! thanks for your explanation!
ReplyDeleteNice blog! I like your style, which is very clear for reading and understanding!
ReplyDeleteI saw many comment asking about the difference between simple and multiple linear regression. Here is what I got:
ReplyDeleteOrdinary linear squares regression compares the response of a dependent variable given a change in some explanatory variables. However, it is rare that a dependent variable is explained by only one variable. In this case, an analyst uses multiple regression, which attempts to explain a dependent variable using more than one independent variable.
Multiple regressions are based on the assumption that there is a linear relationship between both the dependent and independent variables. It also assumes no major correlation between the independent variables.