| dc.description.abstract |
Integrating AI-based methods into emerging 5G and 6G networks is essential for improving current network functions, unlocking new features, and accommodating the rapid increase in connected devices. In addition, these methods play a crucial role in supporting advanced applications such as smart cities, autonomous driving, and immersive experiences. A key technology that helps meet the stringent requirements of these applications is multi-access edge computing (MEC), which brings computational and storage resources closer to end users to enable low latency, high reliability, and enhanced privacy. However, MEC networks face significant uncertainty due to dynamic environments, unpredictable user availability and behavior, and limited resources. Motivated by these challenges, this thesis designs and applies bandit algorithms to address key decision-making problems in MEC networks under uncertainty. Additionally, it proposes methods to optimize network design for improved communication and learning performance, with a particular focus on federated learning (FL).
The first contribution addresses the service placement problem in small cell networks, where edge servers, modeled as bandit agents, aim to identify the best service to deploy locally in order to minimize total user delay compared to cloud deployment, under unknown service demand. To this end, we extend the best arm identification (BAI) algorithm for linear bandits in the fixed-confidence setting to a distributed and adaptive multi-agent scenario, where edge servers learn collaboratively. Numerical results show that the optimal service is identified with the desired confidence level and that collaboration accelerates learning in proportion to the number of participating servers. Additionally, we derive upper bounds on per-agent sample complexity and communication cost.
The second contribution focuses on decentralized task offloading and load balancing in dense MEC networks with random channel conditions and varying task sizes. Each user acts as an agent that offloads its task to a server where it can be completed within a specified time limit. Since users compete for shared communication and computational resources, their rewards depend on both their server selection and the actions of other users. We model this interaction as a mean-field multi-armed bandit game, where the environment appears stationary in dense networks. We then propose a load-balancing approach that adjusts users' rewards to influence their decisions and achieve a target load distribution across servers. We theoretically prove convergence to a steady-state load distribution and demonstrate our findings through numerical results.
The third contribution considers an unmanned aerial vehicle (UAV)-enabled network, where UAVs provide sensor coverage and act as FL clients using data collected from energy-harvesting sensors. The goal is to determine optimal UAV coverage that jointly maximizes coverage and minimizes FL delay under uncertainty. After deriving the statistical distribution of the FL delay, the problem is formulated as a multi-objective BAI bandit problem. A general scalarization-based, cost-aware BAI algorithm is proposed to identify the optimal arms that maximize the ratio of expected reward to expected energy cost. We derive an upper bound on the algorithm’s error probability, and numerical results demonstrate its varying effectiveness in identifying optimal arms under different parameter settings. The evaluation also compares the regret and fairness of algorithm variants, highlighting the trade-offs between competing objectives.
The final contribution addresses the problem of minimizing FL convergence time in UAV-enabled networks by jointly optimizing coverage and resource allocation. The proposed solution is divided into two stages. First, a heuristic method is designed to determine UAV placements that adjust coverage to reduce the computation time. Then, given that UAV locations influence communication delays, a fair channel allocation and power control strategy is employed to minimize communication time. These two stages collectively aim to mitigate the straggler effect in FL. Unlike previous work, this approach does not employ bandits, as the sensor distribution is assumed to be known. Simulation results demonstrate the effectiveness of the proposed joint optimization approach.
Overall, this work demonstrates the effectiveness of bandit algorithms in addressing key MEC challenges and motivates the development of new methods inspired by the problem setting. It also shows the critical role of efficient network design in supporting, and in some cases accelerating, the learning process. |
en |
