Deep Reinforcement Learning for UAV Placement over Mixed FSO/RF-Based Non-Terrestrial Networks

Abstract

Non-terrestrial Network (NTN) architecture, leveraging high-altitude platforms (HAP)-based free-space optical (FSO) backhaul and unmanned aerial vehicles (UAV) for radio frequency (RF) last-mile access, is a promising solution for the future 6G era. Nevertheless, the mobility of the end-users, together with time-varying turbulence/clouds in backhaul links, poses significant challenges in deploying UAVs to maximize the end-to-end network performance. This paper introduces a framework that utilizes deep reinforcement learning (DRL) to optimize UAV placement, considering both dynamic backhaul and access constraints. The results indicate that the trained agent can effectively learn from the environment, confirming its effectiveness in maintaining a relatively high end-to-end throughput performance. © 2024 IEEE.