Resource-Efficient Techniques for Hyperparameter Optimization in Machine Learning

Hyperparameter optimization (HPO) is a key step in developing high-performing machine learning models, but it often requires significant computational resources, making it difficult to apply in constrained environments. This paper focuses on evaluating a grammar-guided, two-stage evolutionary framework for resource-efficient HPO that combines search space pruning with focused refinement. The main contributions of this study are twofold. First, the framework is systematically benchmarked against Covariance Matrix Adaptation Evolution Strategy in addition to Grid Search, Random Search, Tree Parzen Estimator in our previous work, across image and tabular classification tasks. Second, a detailed ablation study is conducted to assess the impact of model complexity, training data volume, and pruning configurations on optimization outcomes. Experiments on five benchmark datasets show that the method achieves performance within 1–2% of the best baseline while reducing computational time by up to 40% and memory usage by approximately 30%. The ablation results demonstrate that strong performance can be achieved even with 5% of the training data and that dependency-aware pruning improves efficiency with minimal trade-off. These results highlight the framework’s potential for applications where compute and memory resources are limited.