# Locally Weighted Regression Algorithm in Python

Video Tutorials

## Python Program to Implement the Locally Weighted Regression Algorithm

Exp. No. 10.   Implement the non-parametric Locally Weighted Regression algorithm in order to fit data points. Select the appropriate data set for your experiment and draw graphs.

## Locally Weighted Regression Algorithm

### Regression:

• Regression is a technique from statistics that is used to predict values of the desired target quantity when the target quantity is continuous.
• In regression, we seek to identify (or estimate) a continuous variable y associated with a given input vector x.
• y is called the dependent variable.
• x is called the independent variable.

### Loess/Lowess Regression:

Loess regression is a nonparametric technique that uses local weighted regression to fit a smooth curve through points in a scatter plot.

### Lowess Algorithm:

Locally weighted regression is a very powerful nonparametric model used in statistical learning.

READ  Bayesian network in Python using pgmpy

Given a dataset X, y, we attempt to find a model parameter β(x) that minimizes residual sum of weighted squared errors.

The weights are given by a kernel function (k or w) which can be chosen arbitrarily

## Algorithm

1. Read the Given data Sample to X and the curve (linear or non linear) to Y

2. Set the value for Smoothening parameter or Free parameter say τ

3. Set the bias /Point of interest set x0 which is a subset of X

4. Determine the weight matrix using :

5. Determine the value of model term parameter β using:

6. Prediction = x0*β

### Python Program to Implement and Demonstrate Locally Weighted Regression Algorithm

```import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

def kernel(point, xmat, k):
m,n = np.shape(xmat)
weights = np.mat(np1.eye((m)))
for j in range(m):
diff = point - X[j]
weights[j,j] = np.exp(diff*diff.T/(-2.0*k**2))
return weights

def localWeight(point, xmat, ymat, k):
wei = kernel(point,xmat,k)
W = (X.T*(wei*X)).I*(X.T*(wei*ymat.T))
return W

def localWeightRegression(xmat, ymat, k):
m,n = np.shape(xmat)
ypred = np.zeros(m)
for i in range(m):
ypred[i] = xmat[i]*localWeight(xmat[i],xmat,ymat,k)
return ypred

bill = np.array(data.total_bill)
tip = np.array(data.tip)

#preparing and add 1 in bill
mbill = np.mat(bill)
mtip = np.mat(tip)

m= np.shape(mbill)
one = np.mat(np1.ones(m))
X = np.hstack((one.T,mbill.T))

#set k here
ypred = localWeightRegression(X,mtip,0.5)
SortIndex = X[:,1].argsort(0)
xsort = X[SortIndex][:,0]

fig = plt.figure()
ax.scatter(bill,tip, color='green')
ax.plot(xsort[:,1],ypred[SortIndex], color = 'red', linewidth=5)
plt.xlabel('Total bill')
plt.ylabel('Tip')
plt.show();```