REA Press Null REA Press3042-31203042-3120 REA Press https://doi.org/10.22105/ahse.v3i2.62 Research Article Diabetes prediction, Pima Indians dataset, Principal component analysis, Synthetic minority oversampling technique, Extreme gradient boosting, Transformer, Hybrid gradient boosted decision trees A Multistage PCA–SMOTE Preprocessing Pipeline for Diabetes Prediction Using XGBoost and Hybrid TransformersA Multistage PCA–SMOTE Preprocessing Pipeline for Diabetes Prediction Using XGBoost and Hybrid Transformers Rahimi Amir Mohammad Department of Computer Engineering, University of Seyed Jamal E Asadabadi, Kermanshah, Iran. Noorian Hedieh Department of Computer Engineering, Hamedan University of Technology, Hamedan, Iran. 06 2026 15 06 2026 3 2 © 2026 REA Press 2026

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

 A Multistage PCA–SMOTE Preprocessing Pipeline for Diabetes Prediction Using XGBoost and Hybrid Transformers

In this study, the Pima Indians Diabetes dataset (PIMA), comprising 768 clinical records with eight metabolic and hereditary attributes, is used to develop a binary diabetes prediction pipeline based on Principal Component Analysis (PCA) and the Synthetic Minority Oversampling Technique (SMOTE). PCA is applied to reduce multicollinearity and obtain orthogonal features, while SMOTE corrects the strong class imbalance, yielding a more stable and informative representation for learning algorithms. Within this preprocessed space, a wide spectrum of models is optimized by systematic GridSearch (GS) based Hyperparameter (HP) tuning, ranging from Logistic Regression (LR), Support Vector Machine (SVM), and tree ensembles to deep neural networks and Transformer based architectures. The results show that, although Extreme Gradient Boosting (XGBoost) remains a strong traditional baseline, a hybrid Transformer combined with Gradient Boosted Decision Trees (GBDT) achieves the highest Accuracy (ACC), F1 score, and Receiver Operating Characteristic (ROC) Area Under Curve (AUC) on the Pima Indians dataset, demonstrating that rigorous data conditioning together with architecture aware Hyperparameter Optimization (HPO) can substantially enhance the reliability of medical diagnostic models.

Null