Htykuut

I am using Scikit-learn library to do a linear regression. Everything is simple and straightforward.
With 6 lines of code, I can do the job. However, I want to know exactly what is happening behind.

Since I am a beginner in ML, maybe my question is wrong, but I am wondering what algorithm does Scikit-learn is using to minimize the mean squared error in its linear_regression method.

From the documentation:

From the implementation point of view, this is just plain Ordinary Least Squares (scipy.linalg.lstsq) wrapped as a predictor object.

You can give a look at the source code too here, where it calls linalg.lstsq.

An extra note about what's happening behind:

If the linear formula is a * x + b, you can access the coefficients (a) and the bias (b) with the attributes coef_ and intercept_ of the trained model.

A toy example generating the identity diagonal from 3 dots to show the coef_ and intercept_ attributes:

from sklearn.linear_model import LinearRegression



X = np.array([[1], [2], [3]])

y = np.array([1, 2, 3])



lg = LinearRegression()

lg.fit(X, y)

lg.coef_ # 1

lg.intercept_ # ~ 0

Scikit-Learn's LinearRegression uses the closed form solution, i.e the OLS solution. It specifically uses the Ordinary Least Squares solver from scipy.