Reinforcement Learning for Resource Planning in Drone-Based Softwarized Networks

In the last few years, some research papers have proposed usage of UAVs organized in Flying Ad-Hoc Network (FANET) in remote areas with a poor or completely non-existent structured network to support novel application scenarios in which data generated by the end users must be processed on site with ultra-low latency. This way, a FANET can be seen as a provider of services and/or slices for extreme-edge 5G networks for delay-sensitive applications. However, keeping a FANET available and active is an ongoing challenge as the autonomy of UAVs is limited and strongly influenced by power consumption of both the engines and the computing element (CE) where the application functions are executed as virtual machines. In this paper, we present an optimization framework capable of increasing the overall duration of the FANET. To this purpose, we apply Reinforcement Learning (RL) based on Double Deep Q-learning (DDQN) to optimize the percentage of available CPU resources for Virtual Function virtualization, and Integer Linear Programming (ILP) to optimize VF placement inside the active UAVs of the FANET.