Locally Weighted Regression Algorithm in Python

VTU Update 9-3-2021 | Reopening of Even Semester 

Step by Step Procedure to apply for a scholarship in State Scholarship Portal – SSP – 2021

Step 1: How to create an account in State Scholarship Portal SSP 2021

Step 2: Upload documents in SSP portal for e-Attestation

Step 3: Log in to your account and apply for the scholarship

List of documents to be uploaded in e-attestation portal

Subscribe to Youtube and Telegram Channel for Regular Updates

Video Tutorials
Python for Beginners Video tutorial
Big Data Analytics Video Tutorial Placement Video Tutorial

Python Program to Implement the Locally Weighted Regression Algorithm

Exp. No. 10.   Implement the non-parametric Locally Weighted Regression algorithm in Python in order to fit data points. Select the appropriate data set for your experiment and draw graphs.

READ  Appropriate Problems for Artificial Neural Networks

Locally Weighted Regression Algorithm


  • Regression is a technique from statistics that are used to predict values of the desired target quantity when the target quantity is continuous.
    • In regression, we seek to identify (or estimate) a continuous variable y associated with a given input vector x.
      • y is called the dependent variable.
      • x is called the independent variable.

Loess/Lowess Regression:

Loess regression is a nonparametric technique that uses local weighted regression to fit a smooth curve through points in a scatter plot.

Loess/Lowess Regression

Lowess Algorithm:

Locally weighted regression is a very powerful nonparametric model used in statistical learning.

Given a dataset X, y, we attempt to find a model parameter β(x) that minimizes residual sum of weighted squared errors.

The weights are given by a kernel function (k or w) which can be chosen arbitrarily

READ  Gradient Descent and Delta Rule


1. Read the Given data Sample to X and the curve (linear or non linear) to Y

2. Set the value for Smoothening parameter or Free parameter say τ

3. Set the bias /Point of interest set x0 which is a subset of X

4. Determine the weight matrix using :

Locally Weighted Regression Algorithm in Python - 2

5. Determine the value of model term parameter β using:

Locally Weighted Regression Algorithm in Python -1

6. Prediction = x0*β

Python Program to Implement and Demonstrate Locally Weighted Regression Algorithm

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

def kernel(point, xmat, k):
    m,n = np.shape(xmat)
    weights = np.mat(np1.eye((m)))
    for j in range(m):
        diff = point - X[j]
        weights[j,j] = np.exp(diff*diff.T/(-2.0*k**2))
    return weights

def localWeight(point, xmat, ymat, k):
    wei = kernel(point,xmat,k)
    W = (X.T*(wei*X)).I*(X.T*(wei*ymat.T))
    return W
def localWeightRegression(xmat, ymat, k):
    m,n = np.shape(xmat)
    ypred = np.zeros(m)
    for i in range(m):
        ypred[i] = xmat[i]*localWeight(xmat[i],xmat,ymat,k)
    return ypred
# load data points
data = pd.read_csv('10-dataset.csv')
bill = np.array(data.total_bill)
tip = np.array(data.tip)
#preparing and add 1 in bill
mbill = np.mat(bill)
mtip = np.mat(tip)

m= np.shape(mbill)[1]
one = np.mat(np1.ones(m))
X = np.hstack((one.T,mbill.T))

#set k here
ypred = localWeightRegression(X,mtip,0.5)
SortIndex = X[:,1].argsort(0)
xsort = X[SortIndex][:,0]

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(bill,tip, color='green')
ax.plot(xsort[:,1],ypred[SortIndex], color = 'red', linewidth=5)
plt.xlabel('Total bill')


Locally Weighted Regression Algorithm in Python

Click here to download restaurant bill dataset

READ  18CPS13/23 C Programming For Problem Solving Question Papers


This tutorial discusses how to Implement and demonstrate the Locally Weighted Regression Algorithm. If you like the tutorial on the Locally Weighted Regression Algorithm please share it with your friends. Like the Facebook page for regular updates and YouTube channel for video tutorials.

Python Sample Programs for Placement Preparation

Leave a Comment

Your email address will not be published. Required fields are marked *