The defense will take place on july 6th, 2026 (2pm) at the Claude Chappe amphitheater (Lamarr Building).
Title
Towards energy-efficient machine learning models for resource-constrained platforms
Abstract
Neural networks have demonstrated high-level performance across various fields and have largely driven recent progress in artificial intelligence. However the substantial energy consumption of these models constitutes a major obstacle to deploying and, particularly, training neural networks on resource-constrained devices while also raising concerns about their environmental impact. Enhancing the energy efficiency of neural networks is therefore an urgent need regardless of the availability of hardware resources.
This thesis investigates the energy efficiency of neural networks training. To this end, we first propose a taxonomy of existing approaches. We then focus on the growing technique that extends the architecture size during training and on data selection which trains the model on a subset of examples rather than the entire dataset. We first analyze the trade-off that the two methods offer between energy consumption and prediction performance by comparing them to pruning, a standard approach for optimizing computational costs. Overall, results demonstrate that data selection offers proportional energy savings to the number of discarded examples while training on smaller subsets incurs larger performance drops. Growing preserves accuracy while energy consumption is influenced by when the expansion occurs. This timing of growing indeed determines how long each architecture size is used during training, which directly impacts the costs.
Following the comparison results, we further investigate the impact of the growing timing by comparing the one-shot schedule which extends the neural network to a target size in a single step to the iterative schedule which grows the model in multiple steps. We find that one-shot growing offers a competitive accuracy, despite being rarely considered in prior works. It can also enhance energy gains, compared to the iterative schedule, particularly when costly initialization strategies are employed for grown neurons. We then explore when one-shot growing should occur during training to improve the accuracy-efficiency trade-off and find that this choice is governed by a combination of factors including dataset complexity as well as the target performance.
After investigating the growing timing, we focus on adaptive data selection that allows updating the subset throughout training. We consider loss-based methods, a common selection criteria. Such methods require a forward phase on all examples to compute selection losses which can result in a significant overhead and hinder the benefits of training with less data. We explore using the early exit mechanism to build a smaller sub-model to compute selection losses more efficiently and improve the overall computational gains of loss-based data selection. Results show that such a strategy is more beneficial on larger models. We then explore a combination of loss-based and random selection criteria while decreasing the number of selected training examples as training progresses. We employ loss-based selection during first epochs to focus on challenging examples before switching to the efficient random criterion to maximize energy gains. This strategy combines the strengths of the two approaches and improves the accuracy with a careful choice of the switching epoch.
Jury
– Anne-Laure Ligozat (Full professor, ENSIIE): Reviewer
– Denis Trystram (Full professor, Grenoble INP): Reviewer
– Virginie Fresse (Associate professor HDR, Université Jean Monnet Saint Etienne): Examiner
– Anne-Cécile Orgerie (Senior researcher, CNRS): Examiner
– Jalil Boukhobza (Full professor, ENSTA-Bretagne): Examiner
– Walid Bechkit (Full professor, Université Lumière Lyon 2) : Supervisor
– Frédéric Le Mouël (Full professor, INSA Lyon) : Co-supervisor
– Malcolm Egan (Researcher, Inria): Co-supervisor
