In classification problems involving imbalanced data—particularly in the medical domain—the use of resampling methods plays a crucial role in improving the performance of machine learning models. By adjusting class distributions, these methods enhance Precision, Recall, and the ability for early disease detection. In the following, several studies conducted in this area are reviewed.

Burnaev et al. [2] examined three classical resampling methods—oversampling, undersampling, and Synthetic Minority Oversampling Technique (SMOTE)—across more than 1,000 datasets. Their results showed that the success of each technique depends on the appropriate selection of parameters and the classification model, such that in some cases, not applying resampling yielded better performance than applying it improperly. The findings of this foundational study paved the way for more specialized analyses that investigated the more precise effects of specific resampling methods in medical applications. In this context, Celik [3], focusing on the diagnosis of pneumonia from X-ray images, compared the Adaptive Synthetic Sampling (ADASYN) and Random Under-Sampling (RUS) methods and demonstrated that ADASYN, by increasing minority-class samples, improved the Precision of the SVM algorithm to 98.388%, whereas the RUS method resulted in lower performance with an AUC (area under the Random Over-Sampling (ROS) curve) of 97.737%. This performance difference is also consistent with the findings of Afzal et al. [4], who, in the context of software defect prediction, confirmed that model performance depends on the correct choice of the type and level of resampling. The positive impact of oversampling techniques on imbalanced classification problems in medical data is also evident in the study by Gurcan and Soylu [5]. They found that the use of SMOTE and ADASYN led to significant improvements in the F1- score and AUC metrics across most of the applied algorithms, whereas data removal approaches—such as undersampling methods—resulted in a reduction in precision in some cases. These findings were also confirmed by Ghorbani and Ghousi [6]. In the context of predicting students’ academic performance using a decision tree algorithm, they showed that the SMOTE method achieved higher precision of 88.4% compared with undersampling methods (81.2%), indicating the superiority of augmentation-based approaches for imbalanced data. Continuing this line of research, Saputra et al. [7] examined the performance differences between ROS and RUS on the SVM algorithm in sentiment analysis. Similar to previous findings, they demonstrated the significant superiority of oversampling methods: The Precision of the ROS method reached 85% (at k = 10), whereas the RUS method achieved only 75% Precision. This performance gap was attributed to the potential removal of important samples from the majority class when using undersampling. However, the study by Welvaars et al. [8], by incorporating the BSMOTE method into their analyses, showed that more intelligent use of hybrid techniques—such as combining SMOTE with XGBoost—can further enhance performance, achieving precision of up to 97.3% and an AUC of 98.6%. These results are particularly noteworthy in terms of diagnostic quality for imbalanced medical datasets. Carvalho et al. [9], through a comprehensive review of resampling techniques such as SMOTE, ADASYN, Edited Nearest Neighbors (ENN), and Tomek Links, emphasized that selecting an appropriate method should be based on an understanding of the type of imbalance and the algorithm in use. They reported that combining SMOTE with ENN improved the AUC metric in decision tree–based models to as high as 96%.

Over the past several decades, machine learning has become one of the key tools in many scientific and industrial fields. One of the main challenges in using these models is selecting the most suitable one from among a set of algorithms that are evaluated based on various performance criteria. Since different algorithms may compete with one another across various performance criteria—such as precision, training time, and computational cost—the use of MCDM methods has been recognized as an effective approach for comparing models and selecting the best one.

Kou et al. [10] investigated the selection of classification algorithms using five MCDM methods. In this study, the algorithms were ranked using methods such as TOPSIS, ELECTRE III, Grey Analysis, VIKOR, and PROMETHEE II. This study shows that each of these methods may yield different results in algorithm evaluation due to the use of different criteria. Therefore, to reduce the discrepancies among these rankings, they employed Spearman’s rank correlation coefficient and proposed a hybrid approach to improve agreement in algorithm selection. Alqaysi et al. [11] investigated hybrid models for autism diagnosis using medical and socio‑demographic features and combining them with MCDM methods. In this study, three feature selection techniques (ReF, IG, and Chi2) and five machine learning algorithms (Decision Tree, SVM, KNN, Naive Bayes, and AdaBoost) were used to develop 15 hybrid models. The models were evaluated using the FDOSM method. The results indicate that the best models, including the ReF Decision Tree and Chi2 Decision Tree, demonstrate outstanding performance in diagnosing autism severity (mild, moderate, and severe), achieving high precision along with low training and testing times. Song and Peng [12] proposed an MCDM‑based approach for evaluating imbalanced classification models in financial risk prediction. Using the TOPSIS technique, this method combined six evaluation criteria—G‑mean, F‑measure, AUC, FP rate, FN rate, and time—and showed that SMOTE‑based methods, particularly the combination of SMOTE with the C4.5 and MLP algorithms, outperform other approaches. This study emphasizes the importance of using multiple criteria in the evaluation of imbalanced models. Akinsola et al. [13] investigated the selection of the best machine learning algorithms using MCDM. In this study, seven machine learning algorithms were evaluated based on criteria such as Accuracy, Kappa Statistic, True Positive (TP) Rate, and True Negative (TN) Rate, and the FAHP and TOPSIS methods were used to weight the criteria and rank the algorithms. The results indicate that the logistic regression algorithm achieved the best performance and had the highest Kappa statistic. Kumar and Kaur [14] proposed an MCDM‑based framework for selecting the best machine learning techniques in diabetes prediction. In this study, three decision‑making methods—WSM, TOPSIS, and VIKOR—were used to rank the models based on various performance criteria such as precision, recall, and the performance characteristic. Then, a hybrid approach called RPM was used to integrate the rankings and determine the final rank. Experimental results on the Pima dataset showed that the logistic regression model was identified as the best model for diabetes prediction. Moreover, the Bayesian test confirmed that the LR model performs significantly better than the other models. Das et al. [15] evaluated 27 screening formulas and 13 machine learning algorithms for identifying the beta‑thalassemia trait in Indian pregnant women. In this study, in addition to using traditional criteria such as recall, precision, and AUC, two MCDM methods—SECA and TOPSIS—were employed to rank the models and select the best screening approach. The results showed that the ELM and GBC algorithms achieved the best performance according to the evaluation criteria.

In this study, significant research gaps in improving clustering and classification algorithms for imbalanced data were identified and addressed. In high‑dimensional data (which in this study include 21 features), the selection of XGBoost and SVM was evaluated as particularly appropriate due to these model’s ability to handle complex data and identify nonlinear patterns, especially in high‑dimensional settings. One of the innovative aspects of this study is its focus on improving classification performance in imbalanced data. To address this problem, two main strategies, oversampling and undersampling, were employed as key approaches. In addition, four subset methods from each of these strategies were implemented to improve classification results. Particularly in the context of diagnosing diseases such as diabetes, where recall is critical, the application of these techniques for improving the performance of classification models has not yet been comprehensively investigated. This aspect is considered one of the innovative contributions of the present research. To achieve a more accurate evaluation of model performance and to identify the most effective approach for improving classification, MCDM techniques were employed. To weight the different criteria and ensure greater precision in the evaluation, the AHP method was applied based on the judgments of domain experts. Finally, to optimally rank and select the best method, the novel MAIRCA approach was employed, which proved particularly effective in the process of criteria ranking and optimal alternative selection. This novel approach can have broad applications not only for diabetes data but also in other classification domains involving imbalanced datasets.

To overcome data imbalance, the development of specialized strategies and techniques for data analysis and modeling processes is essential. Resampling methods and balancing techniques play a vital role in this context. These techniques are implemented to ensure better representation of minority classes in datasets and to support classification algorithms in producing fairer and more accurate results. Resampling methods encompass various strategies, such as reducing majority‑class samples (undersampling) and increasing minority‑class samples (oversampling). These approaches facilitate dataset balancing and allow classification algorithms to produce more accurate and reliable results.

In evaluating the performance of machine learning models, the selection of appropriate methods for weighting and ranking evaluation criteria plays a vital role in improving decision-making precision. The use of AHP as a valid method for weighting criteria makes it possible to systematically and structurally calculate the relative importance of each criterion. This is particularly important in situations where different criteria have varying impacts on model performance. After determining the weights using AHP, the MAIRCA method is employed to rank the models. This method identifies existing discrepancies by comparing ideal and actual evaluations and ranks the models based on how closely their actual performance matches the ideal condition. The combination of these two methods not only increases evaluation precision, but also makes the decision‑making process more transparent and reliable, as both approaches comprehensively and rigorously analyze the criteria and assess model performance.

The Behavioral Risk Factor Surveillance System (BRFSS) is a health‑related telephone survey conducted annually by the CDC, collecting data from more than 400,000 Americans on risk behaviors and chronic diseases. The dataset used in this study contains 441,455 records and 330 features, extracted from the 2015 version available on the Kaggle website. Based on prior research on diabetes and chronic diseases, 21 features and one target variable were selected to enable a more focused and accurate analysis of the influential factors. This selection was made due to their direct relevance to the factors influencing diabetes and chronic diseases, in order to provide a more precise analysis of the contributing factors. It should be noted that after performing the preprocessing operations, the number of records is reduced to 440,658. Based on the analysis of the correlation matrix and the heatmap shown in Fig. 2, the features “General Health” (GenHlth) and “Physical Health” (PhysHlth) exhibited the strongest positive correlation with the target variable (diabetes diagnosis), whereas the feature “Sex” showed the weakest association. These findings confirm the decisive role of overall health status and mental health in diabetes diagnosis compared to demographic factors.

Finally, after performing the necessary preprocessing steps, the dataset was randomly divided into training (80%) and testing (20%) subsets while preserving the class distribution. These two subsets were subsequently used as inputs to the machine learning algorithms.

In this section, the results obtained from applying the eight resampling methods to the classified data are presented in Table 2. These results were evaluated and compared using five performance criteria, including Accuracy, Precision, Recall, F1‑score, AUC and run time (in seconds). In this regard, the Accuracy metric is used to evaluate the overall performance of the models, while the Precision and Recall metrics are employed to assess the model’s ability to predict the minority class (diabetic people). As shown in Table 3, the baseline models for data classification perform well in predicting the majority class; however, they exhibit weaknesses in predicting the minority class, as evidenced by the low values of the Recall and F1‑score metrics. Based on the values reported in Table 2, the use of resampling methods leads to improvements in the Recall and F1‑score metrics, while the AUC criterion does not exhibit significant changes.

Based on the obtained results, for both the SVM and XGBoost algorithms, it is observed that applying resampling methods generally leads to a decrease in the Accuracy metric, except for the TomekLinks approach, for which the Accuracy remains comparable to the baseline results. The Precision metric decreases across all methods; however, the Recall metric, which is of particular importance in this study, shows a substantial increase for all methods, except for the XGBoost/SVM–TomekLinks approach, for which the changes are more limited. Furthermore, the F1‑score metric shows a significant improvement across all methods, while the AUC metric remains approximately unchanged. Finally, the Run Time in the SVM models is significantly higher, indicating a greater computational time requirement for training these models. In an overall comparison, applying resampling methods leads to improved model performance in predicting the minority class (diabetic individuals) by increasing Recall and F1‑score, which can contribute to enhancing the model’s effectiveness in classifying imbalanced data.

In this section, the final results of the multi‑criteria analysis using the MAIRCA method are presented. Table 4 presents the pairwise comparison matrix used for weighting the criteria, which was completed by an expert in this field. In Table 5, following the calculations performed on the pairwise comparison matrix, the final weight for each criterion has been obtained.

Table 6 presents the final results of implementing the MAIRCA method; in this table, the final values of the criteria function have been calculated for each of the 16 alternatives, and based on these values, the final ranking of the alternatives is provided. This ranking enables a comprehensive comparison and precise prioritization of the alternatives and is used as a basis for decision‑making to improve the performance of imbalanced data classification models.

Based on the obtained results, the final ranking of resampling methods using the MAIRCA technique shows that the XGBoost‑ENN method achieved the highest rank, indicating its superior performance in combining multiple criteria. In contrast, the XGBoost‑NearMiss method holds the lowest rank among the 16 alternatives, indicating its limitations in improving the model’s performance.

In Fig. 3, the comparative performance chart of the six implemented model types is plotted based on the five selected criteria. Each curve in this chart represents the average (values reported in Table 1) corresponding to each category of models for its specific criterion. Among these, the two dashed curves, representing the states before applying the resampling methods, clearly illustrate the differences in results compared to the states after resampling was applied. As observed in the chart, the Accuracy criterion in the state without resampling is significantly higher than the corresponding values in the states with resampling (undersampling and oversampling). This phenomenon occurs due to the high number of TP under imbalanced data conditions, which allows the model to identify a large number of individuals as healthy. However, when resampling methods are applied and the data are balanced, the number of TN increases, which in turn leads to a more balanced Accuracy criterion. On the other hand, the Precision criterion has decreased in all resampling methods, which is due to the increase in the Recall criterion and the inherent trade‑off that exists between these two metrics. The Recall criterion, which holds particular importance in this study, has increased in all resampling approaches compared to the states without resampling. This increase indicates a substantial improvement in the model’s ability to identify diabetic samples (the minority class), and in other words, the number of False Negatives (FN) has decreased. This is recognized as one of the key achievements of the present study, because reducing the number of diabetic individuals who are incorrectly classified as healthy is of great importance. On the other hand, the F1-score criterion has increased on average due to the significant increase in Recall and the relative decrease in Precision, indicating an improved balance between the Recall and Precision criteria. These changes indicate the success of the resampling techniques in improving the model’s performance with respect to the F1-score criterion. Finally, the AUC criterion has remained stable, and no significant changes have been observed in it. On average, the AUC value has remained within a specific range, indicating no noticeable change in the discriminatory power of the models from this perspective, even after applying the resampling techniques. In Table 7, the percentage changes in model performance compared to the models without applying resampling methods are presented. Overall, the resampling methods have led to noticeable improvements in certain criteria compared to the models without resampling. The “Average” column indicates the overall improvement or lack of improvement across all resampling approaches based on the 5 evaluated criteria. As can be observed, the Accuracy criterion has decreased by an average of 10.33% in oversampling approaches and by an average of 17.28% in undersampling approaches; therefore, overall, it has decreased by an average of 13.80%. Similarly, for the Precision criterion, an average decrease of 37.56% is observed across all approaches. With the application of resampling approaches, the XGBoost algorithm shows an average increase of 105.77% in the Recall criterion compared to the non‑resampled state; similarly, the SVM algorithm has also increased by an average of 68.88% in Recall. It should be noted that the F1-score criterion exhibits the largest changes among all evaluated criteria, with a substantial portion of this variation attributable to the SVM algorithm. Finally, based on the obtained results, the AUC criterion has not undergone any noticeable changes and has decreased by only 4.81% on average. It can be stated that applying resampling methods leads to a significant increase in the Recall and F1-score criteria, while, on the other hand, it reduces the Accuracy and Precision metrics.

In addition to the multicriteria ranking obtained with the AHP–MAIRCA framework, an unsupervised clustering analysis was conducted on the sixteen resampling–classifier combinations. The goal of this analysis was twofold. First, we aimed to identify groups of methods with similar overall performance profiles across the evaluation criteria (accuracy, precision, sensitivity, F1‑score, AUC, and Run time), rather than focusing solely on the global ranking from MCDM. Second, by grouping the alternatives, we sought to provide a more interpretable structure for DMs: Instead of choosing from sixteen isolated options, they can reason in terms of a small number of homogeneous clusters that reflect distinct trade‑offs between performance metrics.