Optimizing the performance of the K-Nearest Neighbors algorithm using grid search and feature scaling to improve data classification accuracy

Abstrak

The performance of distance-based classification algorithms such as K-Nearest Neighbors (KNN) is highly dependent on proper feature scaling and optimal parameter selection. Without systematic optimization, KNN may experience decreased accuracy due to feature scale disparities and suboptimal k-values. This study aims to enhance the performance of the KNN algorithm through the integration of Feature Scaling and Grid Search Cross-Validation as a parameter optimization strategy. The research employs the Breast Cancer Wisconsin Dataset, divided into 80% training and 20% testing data. Feature normalization was performed using StandardScaler, while Grid Search was applied to determine the optimal combination of parameters, including the number of neighbors (k), weighting function (weights), and distance metric (metric). The optimized KNN configuration with k = 9, weights = distance, and metric = manhattan achieved an average accuracy of 97.19%, outperforming the baseline accuracy of 93.86%. A paired t-test confirmed that the improvement was statistically significant (p < 0.05). These findings demonstrate that the synergy between feature scaling and parameter tuning can substantially improve both the accuracy and stability of KNN models. The scientific novelty of this study lies in the systematic integration of normalization and parameter optimization through Grid Search, providing an empirical framework that enhances KNN’s robustness across datasets with heterogeneous feature distributions. The proposed approach is recommended for medical data classification and can be adapted to other domains with heterogeneous numerical feature distributions.