User Association in the Presence of Jamming in Wireless Networks Using the Whittle Index

In wireless networks, algorithms for user association, i.e., the task of choosing the base station (BS) that every arriving user should join, significantly impact the network performance. A wireless network with multiple BSs, operating on non-overlapping channels, is considered. The channels of the BSs are susceptible to jamming by attackers. During every time slot, a user arrives with a certain probability. There exists a holding cost in each slot for every user associated with a BS. The goal here is to design a user association scheme, which assigns a BS to each user upon arrival with the objective of minimizing the long-run total average holding cost borne within the network. This objective results in low average delays attained by the users. This association problem is an instance of restless multi-armed bandit problems, and is known to be hard to solve. By making use of the framework presented by Whittle, the hard per-stage constraint that every arriving user must connect to exactly one BS in a time slot is relaxed to a long-term time-averaged constraint. Subsequently, we employ the Lagrangian multiplier strategy to reformulate the problem into an unconstrained form and decompose it into separate Markov Decision Processes at the BSs. Further, the problem is proven to be Whittle indexable and a method for calculating the Whittle indices corresponding to different BSs is presented. We design a user association policy under which, upon arrival of a user in a time slot, it is assigned to the BS having the least Whittle index in that slot. Through extensive simulations, we show that our proposed association policy based on the Whittle index outperforms various user association policies proposed in previous work in terms of different metrics such as average cost, average delay, and Jain's fairness index.