Locally Weighted Regression Algorithm in Python

Subscribe to Youtube and Telegram Channel for Regular Updates

Video Tutorials
Python for Beginners Video tutorial
Big Data Analytics Video Tutorial Placement Video Tutorial

Python Program to Implement the Locally Weighted Regression Algorithm

Exp. No. 10.   Implement the non-parametric Locally Weighted Regression algorithm in order to fit data points. Select the appropriate data set for your experiment and draw graphs.

Locally Weighted Regression Algorithm


  • Regression is a technique from statistics that is used to predict values of the desired target quantity when the target quantity is continuous.
    • In regression, we seek to identify (or estimate) a continuous variable y associated with a given input vector x.
      • y is called the dependent variable.
      • x is called the independent variable.

Loess/Lowess Regression:

Loess regression is a nonparametric technique that uses local weighted regression to fit a smooth curve through points in a scatter plot.

Loess/Lowess Regression

Lowess Algorithm:

Locally weighted regression is a very powerful nonparametric model used in statistical learning.

READ  Bayesian network in Python using pgmpy

Given a dataset X, y, we attempt to find a model parameter β(x) that minimizes residual sum of weighted squared errors.

The weights are given by a kernel function (k or w) which can be chosen arbitrarily


1. Read the Given data Sample to X and the curve (linear or non linear) to Y

2. Set the value for Smoothening parameter or Free parameter say τ

3. Set the bias /Point of interest set x0 which is a subset of X

4. Determine the weight matrix using :

5. Determine the value of model term parameter β using:

6. Prediction = x0*β

Python Program to Implement and Demonstrate Locally Weighted Regression Algorithm

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

def kernel(point, xmat, k):
    m,n = np.shape(xmat)
    weights = np.mat(np1.eye((m)))
    for j in range(m):
        diff = point - X[j]
        weights[j,j] = np.exp(diff*diff.T/(-2.0*k**2))
    return weights

def localWeight(point, xmat, ymat, k):
    wei = kernel(point,xmat,k)
    W = (X.T*(wei*X)).I*(X.T*(wei*ymat.T))
    return W
def localWeightRegression(xmat, ymat, k):
    m,n = np.shape(xmat)
    ypred = np.zeros(m)
    for i in range(m):
        ypred[i] = xmat[i]*localWeight(xmat[i],xmat,ymat,k)
    return ypred
# load data points
data = pd.read_csv('10-dataset.csv')
bill = np.array(data.total_bill)
tip = np.array(data.tip)
#preparing and add 1 in bill
mbill = np.mat(bill)
mtip = np.mat(tip)

m= np.shape(mbill)[1]
one = np.mat(np1.ones(m))
X = np.hstack((one.T,mbill.T))

#set k here
ypred = localWeightRegression(X,mtip,0.5)
SortIndex = X[:,1].argsort(0)
xsort = X[SortIndex][:,0]

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(bill,tip, color='green')
ax.plot(xsort[:,1],ypred[SortIndex], color = 'red', linewidth=5)
plt.xlabel('Total bill')


Locally Weighted Regression Algorithm in Python

Click here to download restaurant bill dataset

READ  E F G and H character pattern programs in Java


This tutorial discusses how to Implement and demonstrate the Locally Weighted Regression Algorithm in Python. If you like the tutorial share it with your friends. Like the Facebook page for regular updates and YouTube channel for video tutorials.

Python Sample Programs for Placement Preparation

Leave a Comment

Your email address will not be published. Required fields are marked *