Response surface-driven hyperparameter optimization for XGBoost

This work uses the response surface methodology (RSM) to determine the hyperparameters with which the XGBoost should work to improve its specificity in classifying patients with breast cancer. The algorithm was compared against two other machine learning algorithms, and its results were the worst compared to those of the other two algorithms. Therefore, the XGBoost hyperparameters were adjusted using the RSM to measure the effectiveness of the RSM in tuning the machine learning algorithms. A Box–Behnken experimental design was used to adjust the hyperparameters, and the response variable was specificity. The average specificity at the beginning was around 93% in the breast cancer problem. When applying the RSM to adjust the hyperparameters, there is an increase in specificity of more than 5%, reaching more than 98% in this variable. Furthermore, compared to the other algorithms, there are also improvements. Better results in specificity are obtained with RSM compared to hyperparameter tuning using Metaheuristics, Grid Search, and Bayesian Optimization in this study.