Combating Latency and Disconnectivity in mmWave Networks: From Theory to Implementation

Project Team

Principal Investigators

Ness Shroff
Morteza Hashemi
Taejoon Kim
Eylem Ekici

Project Vision

The ever-increasing number of wireless devices, which is projected to exceed 12.3 billion by 2022, are catalyzing a coming spectrum crisis in the sub-6 GHz bands. These devices are fueled by applications with ultra-low latency and extremely-high data rate requirements. Thus, spectrum-rich millimeter-wave (mmWave) frequencies, between 30-300 GHz, are being considered as critical components of future mobile cellular systems and emerging WiFi networks with Gbps data rates. However, while it is true that mmWave bands provide very high rate connections to end users, contrary to the general belief, this does not necessarily translate to low latency. In fact, in mmWave-based systems, delay is primarily dominated by availability and not by data rate, where lack of availability manifests itself due to blockage, highly directional communication and the need for additional beam alignment/refinement steps, and inefficient resource allocation and scheduling policies. In this project, our overarching goal will be to develop the theoretical foundations and algorithmic development for low-latency mmWave networking design from the physical layer and MAC layer to network layer and data prediction at the application layer. Thus, we will combat latency in each of its constituent components to engineer a multiplier effect in the overall design.

In this project, our overarching goal is to develop the theoretical foundations and algorithmic development for low-latency mmWave networking from the physical layer and MAC layer to network layer and data prediction at the application layer. To achieve this, the following major goals are considered:

Goal 1–Agile mmWave Connection Setup: This thrust will focus on developing theories and algorithms for the purpose of achieving fast beam alignment and tracking based on advanced channel representation techniques. The emphasis will be placed on the interpretability of the beam refinement process and its time-varying interdependence with CSI estimation

Goal 2–User Management and Predictive Data Delivery: This thrust will focus on mobility- and blockage-aware user scheduling and traffic-aware resource allocation methods. We will also explore capacity and delay gains attained through predictive and concurrent data delivery to multiple users.

Goal 3–Evaluation: Proof-of-Concept Implementation and Testing: To test, validate, and refine methods and algorithms developed in the first two thrusts, we will develop a mmWave Beam Tracer software and implement a testbed with SDRs and other available CoTS components. Components of the testbed and the developed software will be used to investigate a richer set of cases in a low-cost software-in-the-loop setup.

In addition to the Intellectual Merits, this project will have significant societal, economic, and research impacts. 5G-and-beyond technologies have become a centerpiece of technology innovation, as they can provide foundations for economic growth. But as the true value of 5G wireless networks is a combination of mmWave connectivity and availability, their full potential cannot be realized until stringent requirements on data delivery latency are satisfied even under stress conditions (heavily populated cells, blockage, etc.). Our research for low-latency mmWave communications will be instrumental in enabling this development.

Intellectual Merit

To address Goal 1, we divided our research efforts into the following tasks in Year 3 of the project:

First, we explored a BP-type approach for mmWave MIMO channel estimation, resulting in a computationally efficient AMP algorithm. For structured random measurement ensembles, including independent and identically distributed (i.i.d.) Gaussian matrices, the performance of AMP can be characterized by a scalar recursion called state evolution (SE). In this task, we generalize the SE for AMP to a new class of measurement matrices with independent (but not necessarily identically distributed) entries. We also extend the SE to a general class of functions, called controlled functions, which are not limited by the polynomial smoothness; unlike the pseudo-Lipschitz function that has polynomial smoothness, the controlled function may grow exponentially. The lack of structure in the assumed measurement ensembles is addressed by leveraging Lindeberg-Feller. The lack of smoothness of the assumed controlled function is addressed by a proposed conditioning technique. The results broaden and complement previous work by applying the SE analysis for AMP to a wider class of measurement ensembles and a new class of functions.

To achieve Goal 1 and Goal 2, we adopted a cross-layer approach, using the physical-layer characteristics from Tasks 1.1 - 1.2 to facilitate mmWave network planning in a dense urban scenario. We considered two types of blockage: (i) physical blockage caused by obstacles and (ii) network blockage caused by the limited number of UEs that a BS can serve on a resource block (RB) or the excess of network capacity by the aggregated traffic. These factors are influenced by the increasing number of wireless devices and the demand for data-intensive applications. Our model incorporates both types of blockage along with directional beam patterns and random locations of obstacles and UEs. Our network planning approach minimizes the cost of mmWave BS deployment with a blockage guarantee. We present simulation results that validate the UE outage guarantees of our method. Our method distributes the macro-diversity orders in a unique way that differs from other existing work.

Related to Goals 1 and 2 of the project, our concentration was shifted more towards combining these two goals in order to create cross-layer theoretical and algorithmic foundations for low-latency mmWave networking design. In particular, we looked at how peculiar mmWave characteristics at the physical layer (i.e., Goal 1) and network and application layers operations (i.e., Goal 2) can work together to combat latency and create a multiplier effect. In particular, we have completed the following research tasks.

The adaptive bitrate selection (ABR) mechanism, which decides the bitrate for each video chunk, is an important part of video streaming. There has been significant interest in developing Reinforcement-Learning (RL) based ABR algorithms because of their ability to learn efficient bitrate actions based on past data and their demonstrated improvements over wired, 3G and 4G networks. However, the Quality of Experience (QoE), especially video stall time, of state-of-the-art ABR algorithms, including the RL-based approaches falls short of expectations over commercial mmWave 5G networks, due to widely fluctuating throughput. These algorithms find optimal policies for a multi-objective unconstrained problem where the policies inherently depend on the predefined weight parameters of the multiple objectives (e.g., bitrate maximization, stall-time minimization). Our empirical evaluation suggests that such a policy cannot adequately adapt to the high variations of 5G throughput, resulting in long stall times. To address these issues, we formulate the ABR selection problem as a constrained Markov Decision Process where the objective is to maximize the QoE subject to a stall-time constraint. The strength of this formulation is that it helps mitigate the stall time while maintaining high bitrates. We propose COREL, a primal-dual actor-critic RL algorithm, which incorporates an additional critic network to estimate stall time compared to existing RL-based approaches and can tune the optimal dual variable or weight to guide the policy towards minimizing stall time. Our experiment results across various commercial mmWave 5G traces reveal that COREL reduces the average stall time by a factor of 4 and the 95th percentile by a factor of 2.

In the next research task, we considered multi-channel mmWave systems. In practice, we expect that 5G mmWave technology, where transmissions may occur over an unreliable but fast (e.g., mmWave) channel, will be supplemented by a slow reliable (e.g., sub-6GHz) channel. We consider the problem of minimizing the age of information when a source can transmit status updates over these two heterogeneous channels. The unreliable channel is modeled as a time-correlated Gilbert-Elliot channel at a high rate when the channel is in the “ON” state. The reliable channel provides a deterministic but lower data rate. The scheduling strategy determines the channel to be used for transmission in each time slot, aiming to minimize the time-average age of information (AoI). The optimal scheduling problem is formulated as a Markov Decision Process (MDP), which is challenging to solve because super-modularity does not hold in a part of the state space. We address this challenge and show that a multi-dimensional threshold-type scheduling policy is optimal for minimizing the age. By exploiting the structure of the MDP and analyzing the discrete time Markov chains (DTMCs) of the threshold-type policy, we devise a low-complexity bisection algorithm to compute the optimal thresholds. We compare different scheduling policies using numerical simulations. This work has appeared in the IEEE Trans. on Mobile Communications.

Furthermore, we have noted that in many wireless resource allocation problems, Reinforcement Learning (RL) is a technique that is often applied for. In many practical networking applications, it is critically important that the algorithm performs safely, such that instantaneous hard constraints are satisfied at each step, and unsafe states and actions are avoided. However, existing algorithms for “safe” RL are often designed under constraints that either require expected cumulative costs to be bounded or assume all states are safe. Thus, such algorithms could violate instantaneous hard constraints and traverse unsafe states (and actions) in practice. To that end, we develop the first near-optimal safe RL algorithm for episodic Markov Decision Processes with unsafe states and actions under instantaneous hard constraints and the linear mixture model. Both our algorithm design and regret analysis involve several novel ideas, which may be of independent interest to the Machine Learning community. This work was recently presented at ICML 2023.

Decentralized bilevel optimization has received increasing attention recently due to its foundational role in many emerging multi-agent learning paradigms (e.g., multiagent meta-learning and multi-agent reinforcement learning) over peer-to-peer edge networks that are likely to be run over future wireless 5G and 6G systems. However, to work with the limited computation and communication capabilities of edge networks, a major challenge in developing decentralized bilevel optimization techniques is to lower sample and communication complexities. This motivates us to develop a new decentralized bilevel optimization called DIAMOND (decentralized single-timescale stochastic approximation with momentum and gradient-tracking). This work was presented at INFOCOM 2023.

Next, we extended our investigation in edge computing networks and considered the problem of joint communication-computation, and computation offloading in multi-channel systems, such as the mmWave systems that are equipped with sub-6 GHz band as well. We developed online learning solutions using reinforcement learning techniques, such that the mobile user can dynamically select the optimal channel for computation offloading.

Furthermore, we extended our investigation of low-latency mmWave communications and considered the multi-user scheduling problem for mmWave video streaming networks, which are expected to be one of the main use-cases for mmWave technology. In this problem, our main objective is to optimize the long-term quality of experience (QoE) for all users, given two main characteristics of mmWave links: (i) the intermittent nature of mmWave channels due to blockages, and (ii) overhead associated with the beam alignment process. To tackle this problem, we leveraged the contextual multi-armed bandit (MAB) models to develop efficient multi-user scheduling solutions, which are aware of the peculiar mmWave characteristics.

In addition to our theoretical development in Year 3, we have made more progress to establish experimental mmWave testbeds at OSU and KU. In Year 2 of the project, we procured the necessary hardware components to establish the proposed mmWave testbed at OSU, which includes mmWave software-defined radio (SDR) along with all the necessary commercial-off-the-shelf components. In Year 3 of the project, the established testbed has been used to perform various mmWave experimentation and demonstration. The testbed was used to track user equipment via radar imaging, and adjusting the beamform vectors according to the radar image input. The joint radar and communication functions have been streamlined in the code, which has been shared with the public in a Github repository. Secondly, the software architecture of the OSU mmWave testbed has been restructured for better accessibility and modularity. In its current form, it allows for third party beam steering algorithms to be seamlessly integrated into the software architecture without requiring changes to the rest of the software architecture. Moreover, this architecture also simplifies the accommodation of variations of the communication hardware through reusable components in the software architecture. Initial tests with different tracking algorithms have been performed. Tests with new hardware components are planned and requisite components have been ordered.

Summary of Activities in Years 1 and 2 of the project:

The following is a summary of activities completed in Years 1 and 2 in relation to Goal 1.

In Task 1.1, we studied a mmWave beam alignment and tracking problem, based on the application of a machine learning technique, the so-called Kolmogorov model (KM) learning, initially introduced in data analytics. In Task 1.2, we studied the KM learning optimization problem, based on discrete and non-convex programming to develop fast KM learning algorithms. In Task 1.3, we expanded the mmWave link-layer characteristics obtained in Tasks 1.1-1.2 to the network planning problem, for which we developed a link quality-guaranteed mmWave base station (BS) deployment techniques. In Task 1.4, we explored the mmWave channel estimation problem by modeling it as a source coding problem.

In task 1.5, we studied a new approach to the mmWave MIIMO channel estimation problem based on the application of a multiple measurement vectors (MMV) method introduced in compressed sensing. To overcome the challenges associated with the existing MMV methods, we proposed a two-stage beam-sounding method to refine the channel estimation accuracy. The dimension of the angle dictionary in each stage can be reduced, which in turn lowers the computational complexity substantially.

In task 1.6, we investigated the fast Kolmogorov model (KM) learning problem by transforming it into a first-order iterative optimization problem. This task is complementary to Tasks 1.1 and 1.2. The objective is to reduce the complexity while reliably predicting the beam direction to be truly scalable. It is demonstrated that the proposed method can achieve compatible training/prediction performance with significantly reduced computational complexity; roughly two orders of magnitude improvement in terms of the time complexity.

In task 1.7, we formulated the beam alignment problem in the mmWave system as a nonstationary online learning problem with the goal of maximizing the received signal strength under interference constraints. We used a primal-dual method to create a constrained UCB-type kernelized bandit algorithm to reduce interference to other user equipment.

The following is a summary of activities completed in Years 1 and 2 in relation to Goal 2.

First, we investigated the problem of integrating the sub-6 GHz and mmWave interfaces, where the sub-6 GHz interface acts as a fallback data transfer mechanism to combat blockage and intermittent connectivity of the mmWave communications. For this task, PIs Shroff and Hashemi co-supervised a female PhD student at OSU, who graduated in Summer 2022. Related to the same integrated model, we studied the problem of minimizing the age of information when a data source can transmit status updates over two heterogeneous channels, such as the integrated mmWave and sub-6 GHz interfaces. We showed that there exists a multi-dimensional threshold-based scheduling policy that is optimal for minimizing the age. Next, we considered the scheduling and resource allocation problems for mmWave/THz-based midhaul links. In particular, we evaluated the feasibility of higher-band mmWave wireless midhaul links for the transport networks between the Central Units (CU) and Distributed Units (DU) in a disaggregated 5G network architecture with functional splits. To this end, we defined several policies for selecting, associating, and scheduling CU and DU nodes in order to determine the peak data rate that can be supported over each link between a CU and DU.

Broader Impacts

PIs Shroff and Hashemi have worked with a female PhD student on Delay-Optimal Scheduling for Integrated mmWave and RF systems, whose work has been accepted for publication in the IEEE Trans. on Mobile Computing.

The PIs are also actively engaged in Broader Participation efforts at the OSU led AI Institute (AI-EDGE) and have invited their students to participate in a couple of panels given by prominent female faculty to share their experiences and impart their wisdom to the students.

Broader Impacts Activities:

The PIs are actively engaged in Broader Participation efforts at the OSU led AI Institute (AI-EDGE) and have invited their students to participate in a couple of panels given by prominent female faculty to share their experiences and impart their wisdom to the students.

Furthermore, PI Shroff is leading an effort aligned with the AI-EDGE institute where several undergraduate students from Minority Serving Institutions (MSIs) each year are given an opportunity to conduct research at the intersection of AI and Networks. Several projects involving wireless networking and AI aligned with the research in this project across wireless networks and AI are also included in this effort.

Moreover, in summer 2022 the PIs established a wireless summer camp at KU for high school students to teach them the fundamentals of signal processing, wireless communication, and hardware components. The students became familiar with software-defined radios (SDR) and used Pluto SDR to conduct various wireless experiments (FM radio, send/receive a digital image over the air, etc). We held the camp for the second year in summer 2023, and we are planning to enhance the camp materials with mmWave experiments. For instance, observing the impact of the human body on blocking mmWave signals and its comparison with sub-6 GHz signals will be appealing for young high school students.

Publications

J. Pan, A. M. Bedewy, Y. Sun, and N. B. Shroff, "Minimizing Age of Information via Scheduling over Heterogeneous Channels,” ACM MobiHoc’21, Shanghai, China, July 2021.

G. Yao, A. M. Bedewy, and N. B. Shroff, "Battle between Rate and Error in Minimizing Age of Information,” ACM MobiHoc’21, Shanghai, China, July 2021.

G. Yao, A. M. Bedewy, and N. B. Shroff, "Age-Optimal Low-Power Status Update over Time-Correlated Fading Channel,” IEEE ISIT’21, Melbourne, Australia, July 2021.

Q. Duan, T. Kim, H. Ghauch, and E. Wong, "Enhanced Beam Alignment for Millimeter Wave MIMO Systems: A Kolmogorov Model", IEEE Global Communications Conference (Globecom), Taipei, Taiwan, Dec 2020.

M. Dong, T. Kim, J. Wu, and E. Wong, "Millimeter-Wave Base Station Deployment Using the Scenario Sampling Approach," IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 14013-14018, Nov 2020.

G. Xiong, T. Kim, D. J. Love, and E. Perrins, "Optimality Conditions of Performance-Guaranteed Power Minimization in MIMO Networks: A Distributed Algorithm and Its Feasibility," IEEE Transactions on Signal Processing, vol. 69, pp. 119-135, Jan 2021.

M. S. Oh, S. Hosseinalipour, T. Kim, C. G. Brinton, and D. J. Love, "Channel Estimation via Successive Denoising in MIMO OFDM Systems: A Reinforcement Learning Approach," IEEE International Conference on Communications (ICC), Montreal, Canada, June 2021.

J. Kim, S. Hosseinalipour, T. Kim, D. J. Love, and C. G. Brinton, "Multi-IRS-assisted Multi-Cell Uplink MIMO Communications under Imperfect CSI: A Deep Reinforcement Learning Approach," IEEE International Conference on Communications (ICC) Workshop, Montreal, Canada, June 2021.

U. Sajid, M. Chow, J. Zhang, T. Kim and G. Wang, "Parallel Scale-wise Attention Network for Effective Scene Text Recognition," International Joint Conference on Neural Networks (IJCNN), July 2021.

C. Samarathunga, M. Abouelseoud, K. Sakoda, and M. Hashemi, "Multi-hop Routing with Proactive Route Refinement for 60 GHz Millimeter-Wave Networks”, IEEE 93rd Vehicular Technology Conference (VTC2021-Spring) Apr 2021.

C. Samarathunga, M. Abouelseoud, K. Sakoda, and M. Hashemi, "On the Benefits of Multi-hop Communication for Indoor 60 GHz Wireless Networks”. IEEE 18th Annual Consumer Communications & Networking Conference (CCNC) Jan 2021.

Y. Shabara, C. E. Koksal, and E. Ekici, "Source Coding Based Millimeter-Wave Channel Estimation with Deep Learning Based Decoding,” to appear in IEEE Transactions on Communications, April 2021.

G. Yao, M. Hashemi, R. Singh, and N. B. Shroff, "Delay-Optimal Scheduling for Integrated mmWave–Sub-6 GHz Systems with Markovian Blockage Model,” Accepted by IEEE Trans on Mobile Computing.

G. Yao, A. M. Bedewy, and N. B. Shroff, "Age-Optimal Low-Power Status Update over Time-Correlated Fading Channel,” Accepted by IEEE Trans on Mobile Computing.

Y. Deng, X. Zhou, A. Ghosh, A. Gupta, and N. B. Shroff, "Interference Constrained Beam Alignment for Time-Varying Channels via Kernelized Bandits,” to appear in IEEE WiOpt’22, Turin, Italy, Sep. 2022

S. R. Chintareddy, M. Mezzavilla, S. Rangan, and M. Hashemi, "A preliminary assessment of midhaul links at 140 GHz using ray-tracing." In Proceedings of the 5th ACM Workshop on Millimeter-Wave and Terahertz Networks and Sensing Systems, pp. 25-30. 2021.

B. Badnava, T. Kim, K. Cheung, Z. Ali, and M. Hashemi. "Spectrum-Aware Mobile Edge Computing for UAVs Using Reinforcement Learning." In 2021 IEEE/ACM Symposium on Edge Computing (SEC), pp. 376-380. IEEE, 2021.

J. Kim, T. Kim, M. Hashemi, C. G. Brinton, and D. J. Love. "Minimum overhead beamforming and resource allocation in D2D edge networks." IEEE/ACM Transactions on Networking (2021).

Q. Duan, H. Ghauch, and T. Kim, “Dual Optimization for Kolmogorov Model Learning Using Enhanced Gradient Descent”, IEEE Transactions on Signal Processing, vol. 70, pp. 963-977, February 2022.

W. Zhang, M. Dong, and T. Kim, "MMV-Based Sequential AoA and AoD Estimation for Millimeter Wave MIMO Channels,” IEEE Transactions on Communications, pp. 1-15, April 2022.

J. Kim, S. Hosseinalipour, A. C. Marcum, T. Kim, D. J. Love, and C. G. Brinton, "Learning-based Adaptive IRS Control with Limited Feedback Codebooks,” IEEE Transactions on Wireless Communications, pp. 1-16, June 2022.

Q. Duan, T. Kim, H. Ghauch "KM Learning for Millimeter-Wave Beam Alignment and Tracking: Predictability and Interpretability”, IEEE Access, August 2021.

D. Q. Nguyen and T. Kim, "Joint Delay and Phase Precoding Under True-Time Delay Constraint for THz Massive MIMO,” IEEE International Conference on Communications (ICC), May 2022.

R. Simion, T. Kim, and Erik S. Perrins, "Machine Learning With Gaussian Process Regression For Time-Varying Channel Estimation,” IEEE International Conference on Communications (ICC), May 2022.

J. Kim, S. Hosseinalipour, A. C. Marcum, T. Kim, D. Love, and C. G. Brinton, "Deep Reinforcement Learning-Based Adaptive IRS Control with Limited Feedback Codebooks,” IEEE International Conference on Communications (ICC), May 2022.

C. D. Ozkaptan, E. Ekici and O. Altintas, "Adaptive Waveform Design for Communication-Enabled Automotive Radars," in IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 3965-3978, June 2022, doi: 10.1109/TWC.2021.3125924.

C. D. Ozkaptan, E. Ekici, C. -H. Wang and O. Altintas, "Optimal Precoder Design for MIMO-OFDM-based Joint Automotive Radar-Communication Networks," 2021 19th International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt), 2021, pp. 1-8, doi: 10.23919/WiOpt52861.2021.9589830.

C. D. Ozkaptan, H. Zhu, E. Ekici, and O. Altintas, "Software-Defined MIMO OFDM Joint Radar-Communication Platform with Fully Digital mmWave Architecture,“ Proceedings of IEEE Symposium on Joint Communication and Sensing, Online, March 2023.

C. D. Ozkaptan, H. Zhu, E. Ekici, and O. Altintas, “A Fully Digital MIMO Joint Radar-Communication Testbed with Radar-assisted Precoding,” submitted to IEEE Transactions on Wireless Communications, March 2023.

M. Dong, M. Cho, K. Lee, S. Yoon, and T. Kim, “Cost-Optimal Deployment of Millimeter-Wave Base Stations Under Outage Requirement”, IEEE Transactions on Wireless Communications, December 2022.

D. Q. Nguyen and T. Kim, “On the Stability of Approximate Message Passing with Independent Measurement Ensembles,” IEEE International Symposium on Information Theory (ISIT), June 2023.

D. Q. Nguyen and T. Kim, “Joint Hybrid Delay-Phase Precoding Under True-Time Delay Constraints in Wideband Sub-THz Massive MIMO Systems,” submitted to IEEE Transactions on Communications, 2023.

J. Pan, A. M. Bedewy, Y. Sun, and N. B. Shroff, “Age-optimal Scheduling over Hybrid Channels,” IEEE Trans on Mobile Computing, Sep. 2022.

M. Shi, Y. Liang, and N. B. Shroff, “ A Near-Optimal Algorithm for Safe Reinforcement Learning Under Instantaneous Hard Constraints,” ICML, Honolulu, Hawaii, USA, July 2023.

P. Qiu, Y. Li, Z. Liu, P. Khanduri, J. Liu, N. B. Shroff, E. S. Bentley, and K. Turck, “DIAMOND: Taming Sample and Communication Complexities in Decentralized Bilevel Optimization,” IEEE INFOCOM, New York, USA, May 2023.

M. Ghazikor, K. Roach, K. Cheung, M. Hashemi, “Exploring the Interplay of Interference and Queues in Unlicensed Spectrum Bands for UAV Networks”, Asilomar Conference on Signals, Systems, and Computers, 2023.

B. Badnava, K. Roach, K. Cheung, M. Hashemi, N. B. Shroff, “Energy-Efficient Deadline-Aware Edge Computing: Bandit Learning with Partial Observations in Multi-Channel Systems”, submitted to Globecom 2023.

S. Chintareddy, K. Roach, K. Cheung, M. Hashemi, “Collaborative Wideband Spectrum Sensing and Scheduling for Networked UAVs in UTM Systems”, submitted to Globecom 2023.