Building KNN with Sklearn
K-Nearest Neighbors is an algorithm for supervised learning. Where the data is 'trained' with data points corresponding to their classification. Once a point is to be predicted, it takes into account the 'K' nearest points to it to determine it's classification. The k-nearest neighbors (KNN) algorithm is a non-parametric machine learning algorithm that can be used for both classification and regression tasks. It works by finding the k most similar instances in the training set to a new instance and then predicting the label of the new instance based on the labels of the k nearest neighbors.In scikit-learn, the KNeighborsClassifier class implements the k-nearest neighbors algorithm for classification tasks. The following code shows how to build a KNN classifier with 5 neighbors:
The n_neighbors parameter specifies the number of neighbors to use for the prediction. The default value is 5.
Other parameters that can be tuned for the KNN classifier include:
metric: The distance metric to use for calculating the distance between neighbors. The default metric is the Euclidean distance.weights: The weight function to use for the prediction. The default weight function is uniform, which means that all neighbors are weighted equally.algorithm: The algorithm to use for finding the nearest neighbors. The default algorithm is brute-force search.
You can experiment with different values for these parameters to find the best model for your data.
Here is an example of how to use the KNeighborsClassifier class to classify the Iris dataset:
Dataset
# read in the iris data
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
iris = load_iris()
# create X (features) and y (response)
X = iris.data
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.25)
KNN (K=5)
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(X_train, y_train)
y_pred = knn.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print('Accuracy:', accuracy)
KNN (K=1)
knn = KNeighborsClassifier(n_neighbors=1)
knn.fit(X, y)
y_pred = knn.predict(X)
print(metrics.accuracy_score(y, y_pred))
Now we're going to fit our very first classifier using sci-kit-learn! To do so, we first need to import it. To this end, we import KNeighborsClassifier from sklearn dot neighbors. We then instantiate our KNeighborsClassifier, set the number of neighbors equal to 6, and assign it to the variable knn. Then we can fit this classifier to our training set, the labeled data. To do so, we apply the method fit to the classifier and pass it two arguments: the features as a NumPy array and the labels, or target, as a NumPy array. The scikit-learn API requires firstly that you have the data as a NumPy array or pandas DataFrame. It also requires that the features take on continuous values, such as the price of a house, as opposed to categories, such as 'male' or 'female'. It also requires that there are no missing values in the data. All datasets that we'll work with now satisfy these final two properties. Later in the course, you'll see how to deal with categorical features and missing data. In particular, the scikit-learn API requires that the features are in an array where each column is a feature and each row a different observation or data point. Looking at the shape of iris data, we see that there are 150 observations of four features. Similarly, the target needs to be a single column with the same number of observations as the feature data. We see in this case there are indeed also 150 labels. Also check out what is returned when we fit the classifier: it returns the classifier itself and modifies it to fit it to the data. Now that we have fit our classifier, lets use it to predict on some unlabeled data!
from sklearn.neighbors import KNeighborsClassifier knn = KNeighborsClassifier(n_neighbors=6) knn.fit(iris['data'], iris['target']) KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',metric_params=None, n_jobs=1, n_neighbors=6, p=2,weights='uniform') iris['data'].shape #(150, 4) iris['target'].shape #(150,)
Predicting on unlabeled data
Here we have set of observations, X new. We use the predict method on the classifier and pass it the data. Once again, the API requires that we pass the data as a NumPy array with features in columns and observations in rows; checking the shape of X new, we see that it has three rows and four columns, that is, three observations and four features. Then we would expect calling knn dot predict of X new to return a three-by-one array with a prediction for each observation or row in X new. And indeed it does! It predicts one, which corresponds to 'versicolor' for the first two observations and 0, which corresponds to 'setosa' for the third
prediction = knn.predict(X_new) X_new.shape #(3, 4) print('Prediction {}’.format(prediction)) #Prediction: [1 1 0]
References