A Q-learning approach for the support of reliable transmission in the internet of underwater things

—Underwater networks are characterized by undesirable time variability of the channel conditions due to Doppler effect, serious multipath and variable impulse response. Energy efficiency is a primary concern in these scenarios, because nodes batteries cannot be recharged or replaced. Accordingly, in order to increase network lifetime and effectiveness of underwater channel transmissions, in this paper, we present a reinforcement learning approach for communication in multihop underwater networks. The proposed methodology employs a Markov underwater channel model to characterize the link status that allows the relay device to choose the most efficient next hop node to forward data towards a remote gateway device for connection to the terrestrial internet. Simulation results are presented to show the effectiveness of the proposed methodology in terms of energy consumption and latency performance so allowing to increase network lifetime.