PurGE: Towards Responsible Artificial Intelligence Through Sustainable Hyperparameter Optimization

Hyperparameter optimization (HPO) plays a crucial role in enhancing the performance of machine learning and deep learning models, as the choice of hyperparameters significantly impacts their accuracy, efficiency, and generalization. Despite its importance, HPO remains a computationally intensive process, particularly for large-scale models and high-dimensional search spaces. This leads to prolonged training times and increased energy consumption, posing challenges in scalability and sustainability. Consequently, there is a pressing demand for efficient HPO methods that deliver high performance while minimizing resource consumption. This article introduces PurGE, an explainable search-space pruning algorithm that leverages Grammatical Evolution to efficiently explore hyperparameter configurations and dynamically prune suboptimal regions of the search space. By identifying and eliminating low-performing areas early in the optimization process, PurGE significantly reduces the number of required trials, thereby accelerating the hyperparameter optimization process. Comprehensive experiments conducted on five benchmark datasets demonstrate that PurGE achieves test accuracies that are competitive with or superior to state-of-the-art methods, including random search, grid search, and Bayesian optimization. Notably, PurGE delivers an average computational speed-up of 47x, reducing the number of trials by 28% to 35%, and achieving significant energy savings, equivalent to approximately 2,384 lbs of CO₂e per optimization task. This work highlights the potential of PurGE as a step toward sustainable and responsible artificial intelligence, enabling efficient resource utilization without compromising model performance or accuracy.