Multi-Armed Bandits for V2X User Association

Project Overview

Focus: Online learning for user association in vehicular networks Context: Ph.D. research on V2X communication under mmWave channel blockage Technologies: Multi-armed bandits (UCB family)

Choosing which roadside unit (RSU) a vehicle should associate with, moment to moment, is a sequential decision problem under uncertainty. Link quality is unknown ahead of time, changes as the vehicle moves, and can be blocked outright by buildings or other vehicles at mmWave frequencies. This project treats that problem as a multi-armed bandit, where each RSU is an arm and the reward is the achievable link quality, and studies how classic bandit algorithms like UCB need to be adapted once the “arms” behave like real wireless channels instead of stationary slot machines.

Why Bandits

Supervised learning needs labeled data on every option; in a live vehicular network you only observe the outcome of the RSU you actually chose. That’s exactly the exploration/exploitation trade-off bandits are built for. The branches below trace a progression from the textbook stochastic bandit setting toward the messier, non-stationary conditions that show up in real V2X scenarios, sparse rewards, sudden hand-off shifts, and gradual path-loss decay.

Where This Is Going

Next steps are extending this from single-agent UCB toward contextual and multi-agent formulations that account for mmWave blockage prediction directly, tying this research line together with the scenario co-simulation work being built to generate more realistic V2X test scenarios.