Implementations of Ridge Regression in Python

ridge.py

def ridge(X, y, l2):
    """Ridge Regression model with intercept term.
    L2 penalty and intercept term included via design matrix augmentation.
    This augmentation allows for the OLS estimator to be used for fitting.
    Params:
        X - NumPy matrix, size (N, p), of numerical predictors
        y - NumPy array, length N, of numerical response
        l2 - L2 penalty tuning parameter (positive scalar) 
    Returns:
        NumPy array, length p + 1, of fitted model coefficients
        
    Note: Solving for the OLS estimator using the matrix inverse does not scale well,
    thus the NumPy function solve, which employs the LAPACK _gesv routine, is used to find the least-squares solution.
    This function solves the equation in the case where A is square and full-rank (linearly independent columns). 
    However, in the case that A is not full-rank, then the function lstsq should be used,
    which utilizes the xGELSD routine and thus finds the singular value decomposition of A.
    """
    m, n = np.shape(X)
    upper_half = np.hstack((np.ones((m, 1)), X))
    lower = np.zeros((n, n))
    np.fill_diagonal(lower, np.sqrt(l2))
    lower_half = np.hstack((np.zeros((n, 1)), lower))
    X = np.vstack((upper_half, lower_half))
    y = np.append(y, np.zeros(n))
    return np.linalg.solve(np.dot(X.T, X), np.dot(X.T, y))