Comprehensive Performance Analysis of Aerial-Platforms-Enabled Mixed FSO/RF Communication with NOMA Framework

Abstract

Free-space optics (FSO) and radio frequency (RF)-based non-orthogonal multiple access (NOMA) are key enablers for 5G and beyond communication. This paper proposes a novel mixed FSO/RF communication system with the NOMA framework leveraging high-altitude platform stations (HAPS) and unmanned aerial vehicles (UAVs) as relays to enhance reliability and spectral efficiency. The proposed system employs decode-and-forward (DF) relaying, with FSO links modeled using a doubly inverted Gamma-Gamma (IGGG) distribution incorporating atmospheric turbulence, pointing errors, and angle-of-arrival (AoA) fluctuations, while RF links follow a Nakagami-m fading model. A relay selection scheme based on channel conditions is implemented to optimize UAV selection. Further, closed-form expressions for outage probability (OP), ergodic capacity (EC), and throughput are derived and validated through Monte-Carlo simulations. Numerical results show that system performance is dominated by the link reliability of FSO communication at shorter user-to-UAV distances and transitions to RF link dominance at longer distances. For the two-user scenario, the analysis reveals that user 1 experiences interference-limited saturation at a high signal-to-noise ratio (SNR), while user 2 achieves linear capacity growth due to successive interference cancellation (SIC). The proposed system model effectively mitigates channel impairments, offering valuable insights for the next-generation wireless networks. © 2025 IEEE.