Diagnosing Non-Stationarity and Relative Overgeneralization in Fully Decentralized Multi-Agent Reinforcement Learning for Priority-Aware Mobile Edge Computing Multi-agent reinforcement learning; Non-stationarity; Relative overgeneralization; Mobile edge computing; Decentralized control; Task offloading; Priority-aware scheduling; Quality of service
Main article
Abstract
Fully decentralized multi-agent reinforcement learning is an attractive control paradigm for priority-aware mobile edge computing, because edge nodes must decide how to admit, schedule, and offload heterogeneous tasks under strict latency budgets without relying on a central coordinator or on the exchange of raw state. Yet decentralized learners in a shared edge environment routinely under-perform their centralized counterparts, and the reasons are frequently misattributed to hyper-parameters or to insufficient training. This paper argues that two structural pathologies explain most of the gap: non-stationarity, in which each learner faces a moving target because its peers adapt concurrently; and relative overgeneralization, in which agents are drawn toward a robust but jointly sub-optimal equilibrium whose shadowed reward dominates the narrow optimum. We formalize priority-aware edge offloading as a decentralized partially observable Markov decision process, derive a priority-weighted objective that couples the agents through shared queues and interference, and show analytically why priority coupling deepens both pathologies. Using a controlled edge simulator we isolate each effect, quantify it with dedicated diagnostics (a target-drift index, a shadowed-reward gap, and a miscoordination rate), and measure how the pathologies translate into deadline misses for high-priority traffic. Across the configurations studied, independent Q-learning converges 2.1 times slower and misses 3.4 times as many high-priority deadlines as a centralized-critic reference, and roughly two thirds of that penalty is traceable to relative overgeneralization rather than to non-stationarity alone. We then evaluate leniency, hysteretic updates, distributional value estimation, and opponent modeling as targeted remedies, and discuss which classes of edge workload each remedy suits. The paper supports the position that decentralization failures in edge learning are diagnosable and separable, and that remedies should be matched to the dominant pathology rather than applied indiscriminately.
