In Econometrics, we use the tool of Regression Analysis to understand the economic relationships through quantitative estimation. This quantitative estimation is done by Regression which is one of the most frequent and important tool used to understand economic theories. Hence, it is easy to describe a relationship in a theoretical form but it would be difficult to write it in the form of an equation and estimate the theory by a given data. Also, not only do we find the relationship of the variables but the direction and the amount of change in the variables used in the theory. Thus, To predict both the factors of 1) Amount of change 2) Direction of change as well as the significance of the relationship between the variables, we use Regression Analysis.

Regression Analysis Definition

Regression Analysis is a Statistical technique that actually explains the change in dependent variable due to movement in other independent variables. 

It is a technique of predicting the unknown variable through the known variables.

Dependent variable , Independent variable and Causality in Regression Analysis:

A dependent variable is the variable which is dependent over the actions of the independent variable. Any changes made in the independent variable will lead to a change in the dependent variable. This is a cause and effect relationship between the variables.

Important point :  Other names of Dependent and independent variable.

Dependent variable = Explanatory variable = Regressand 

Independent variable = Cause variable = Regressor

To understand the relationship of dependent and independent variable in regression analysis : Let us explain it by an example:

Q=ƒ(P,Ps,Yd)

Where Q is the quantity demanded and a dependent variable which is effected by the independent variable of P= Price , Ps=Price of Substitutes and Yd= Income

This equations explains that Quantity demanded is a function of all these variables. Any change in these variables will change the Quantity Demanded (keeping other things constant ).

The reason of using the word of keeping other things constant is because we will not be able to understand the effect of a single independent variable over the independent variable if all the variables and their effects are checked simultaneously. That is why , we estimate the variable in a such a way that it keeps other things constant while analyzing the effect of a single variable over the dependent variable.

Notice that much of the econometric analysis is concerned with cause and effect relations. Yet you must remember that regression analysis is purely not about proving the case and effect relationship. it is rather understanding the strength and direction of the effect with the condition that the variable really do have a significant relationship or not. 

Simple Regression Model

Simple regression model is a single equation linear model which can be explained in the following way :

Y=β* + β¹X

The above equations states that Y is the dependent variable and is a single equation linear function of the independent variable X.

Why do we call it linear ?

The model is linear because of the following reasons:

  1. If we plot the equations it will be a straight line.
  2. It is a simple one variable linear equation because it includes only one coefficient of variable β1 . It will be multivariate regression model if its includes more than one coefficient of variable.
  3. It will not be a curve but a straight line and a straight line shows  constant slope or rate of change at each point.
  4. The slope coefficient shows that Y will change when X increase by one unit.

Simple linear Regression Graph

simple regression analysis

Source : Studenmund, Econometrics Book ch1, Simple regression analysis

Remember that the βs determine the coordinates of straight line and βo is constant or intercept term which shows the value of Y when X is Zero.

Also note that in regression analysis the emphasis is more on the slope coefficients like β1. In the above straight line regression, the slope is constant throughout the function. and a constant slope means that a one unit change in X will lead to the same amount of change in Y.

Whenever you run a regression in any software, the gist of the whole equation is that it should be linear. And if it is not linear than you have to make it linear before you run regression analysis.

Linear Equation Vs Non-linear equation

A linear equation is :

Y= β0X

where :

  • Y= Dependent variable
  • β0= The intercept
  • β1=The slope coefficient which shows the response or change in Y to a one unit increase in X variable.

A non-linear equation can be somewhat like this :

Y= β0 +βX2

The above equation is a non linear , rather a quadratic equation over which we can run regression in any software available. To make more easier for us to run regression analysis, we attempt to make the equation linear first. To do that , we create a new variable which is equal to the square of X.

Z=X2

Now the equation becomes :

Y= β0 +βZ

The equation can now be estimated by regression analysis as it is in linear form.