Implementation of Decision Tree in Python

Implementation of Decision Tree in Python – Machine Learning

In this tutorial, we will understand the Implementation of the Decision Tree classifier in Python – Machine Learning.

Importing the libraries

To begin the implementation first we will import the necessary libraries like NumPy, matplotlib, and pandas.

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

Importing the dataset

Next, we import or read the dataset. Click here to download the dataset used in this implementation. The breast cancer dataset has the following features: Sample code number, Clump Thickness, Uniformity of Cell Size, Uniformity of Cell Shape, Marginal Adhesion, Single Epithelial Cell Size, Bare Nuclei, Bland Chromatin, Normal Nucleoli, Mitosis, Class.

After reading the dataset, divide the dataset into concepts and targets. Store the concepts into X and targets into y.

dataset = pd.read_csv('Data.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values

Splitting the dataset into the Training set and Test set

Once the dataset is read. Next, divide the dataset into two parts, training and testing using the train test split function from sklearn. The test_size and random_state attributes are set to 0.25 and 0 respectively. You can change these attributes as per your requirements.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)

Feature Scaling

Feature scaling is the process of converting the data into a given range. In this case, the standard scalar technique is used.

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

Training the Decision Tree Classification model on the Training set

Once the dataset is scaled, next, the decision tree classifier algorithm is used to create a model. The hyperparameters such as criterion and random_state are set to entropy and 0 respectively. The remaining hyperparameters are set to default values.

from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(criterion = 'entropy', random_state = 0)
classifier.fit(X_train, y_train)

Decision tree classifier model

DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,
                       max_features=None, max_leaf_nodes=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, presort=False,
                       random_state=0, splitter='best')

Display the results (confusion matrix and accuracy)

Here evaluation metrics such as confusion matrix and accuracy are used to evaluate the performance of the model built using a decision tree classifier.

from sklearn.metrics import confusion_matrix, accuracy_score
y_pred = classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred)
print(cm)
accuracy_score(y_test, y_pred)

Output

[[103, 4]
[ 3, 61]]

See also  Concept Learning in Machine Learning

Accuracy: 0.9590643274853801

Summary:

In this tutorial, we understood, the Implementation of the Decision Tree classifier in Python. If you like the tutorial share it with your friends. Like the Facebook page for regular updates and YouTube channel for video tutorials.

Leave a Comment

Your email address will not be published. Required fields are marked *