Managing the Bandwidth Surge
By John Ulm
How to cope with the spikes and long-term growth
“With all due respect, sir,
I believe this will be our
finest hour.”
That quote from the movie Apollo 13 is strangely appropriate for the broadband industry after a global lockdown from the COVID-19 pandemic. The pandemic turned the whole world upside down, with entire populations forced to live, work, learn and play from home. There was 12-18 months worth of bandwidth (BW) growth in a two-week interval.
The broadband industry held its breath to see how our infrastructure would cope. Broadband kept society and the economy running (as best we can) in these hard times. This is arguably its most significant contribution to society in its short life-span.
- COVID-19 pandemic hits — the BW storm surge
Before the BW storm, the downstream peak consumption had been growing ~30% annually over the last few years while the upstream was ~21%. But once the BW storm surge hit, the upstream got crunched much more than the downstream. The NCTA COVID-19 dashboard showed this best. The upstream peak growth jumped 34% in March and relaxed during the summer before a second surge in the fall. By comparison, the downstream peaked at 20% in April and was only at 14% by September.
Normally, day-time usage is much lower than peak evening hours, but that changed dramatically for the upstream. It hits its peak late morning and then stays there until almost midnight.
With network congestion comes impacts on subscribers quality of experience (QoE). Some of the negative QoE effects for subscribers include:
- increased latencies
- video tiling
- increased file download and upload times
Overall, these bad QoE effects have been relatively minimal to date for most service groups (SG). Only a handful of SGs that were pushed to the limit before the BW surge had detrimental QoE. In general, the DOCSIS networks are holding up to this sudden, stressful packet load. Several reasons for this:
- Good network capacity planning added plenty of headroom
- g., enough to absorb the largest SLA bursts
- Excellent CMTS scheduling algorithms
- These are sophisticated AI engines that have evolved over the last two decades
- They are excellent at adapting to congestion, with fair BW distribution
- Most internet applications are elastic and forgiving; TCP and ABR recover from thruput reductions and packet loss
- Repairing leaks during the storm — short-term fixes
Some operators needed some quick fixes for severely congested service groups to relieve the pressure. These needed to be rolled out in a matter of hours or days to relieve the immediate heartburn.
Adding DOCSIS channels with the I-CCAP was first choice if spectrum was available, with the roll-off being a good option for the 3.1 OFDM channels. Turning on OFDMA in the upstream should be a priority as well. Various configuration changes that could improve operation include:
- Enabling TCP ack suppression
- Reduce buffer-bloat latencies with buffer-control TLVs, 3.1 AQM
- Optimizing QAM-modulation or FEC
- Re-locating in-home Wi-Fi
These might only provide 5% – 15% improvements, but everything helps in a congested network. By themselves they may not add up to much, but collectively these might be enough to get an operator through the rough patch until more permanent capacity can be added.
- Cleaning up after the storm —near-term 6-12 month strategies
In the near-term over the coming months, operators can add more substantive improvements to their network capacity. For additional DOCSIS capacity, operators should consider turning on or increasing the 3.1 OFDM/OFDMA channels. To find spectrum for additional DOCSIS channels, operators can use SDV or improved encoding to reduce legacy video spectrum.
On the HFC plant, node segmentations to split SGs will help manage BW. Nodes might go from 1×1 to 1×2 to 2×2/2×4 to 4×4 segmentation. This can be done simply by upgrading the node and not touching anything else in the outside plant. Operators should consider upgrading severely congested sites to 1218/85 MHz or 1218/204 MHz plant.
Meanwhile in the home, more DOCSIS 3.1 modems and Wi-Fi 6 routers should be deployed to customers’ homes.
- Preparing for the next storm —network capacity planning
Network capacity planning is key to withstand another bandwidth surge. The CommScope/Arris Basic Traffic Engineering Formula provides guidelines on how much capacity is required to provide good QoE. The “basic” formula shown in Figure 3 is a simple two-term equation. The first term (Nsub*Tavg) allocates bandwidth capacity to ensure that the aggregate average bandwidth generated by subscribers can be adequately carried by the SG capacity. The first term is the “DC traffic component” that tends to exist as a continuous flow during the peak busy period.
There are obviously fluctuations that occur (i.e., “AC traffic component”). The second term (K*Tmax_max) increases the probability that all subscribers, including those with the highest Service tiers (i.e., Tmax values), will experience good QoE levels.
The second term has an adjustable parameter defined by the K value. Our research shows that a K value in the 1.0-1.2 range provides excellent QoE for typical SG sizes. The operator can use the formula to guide them on how much capacity headroom is needed to provide better QoE to their subscribers.
Networks that have been designed with sufficient capacity have been resilient and have handled the coronavirus BW surge.
It is important that operators not only monitor their network utilizations, but adjust their thresholds accordingly based on the highest tier being offered. For example, a utilization threshold of 80% to take action is not good enough by itself. That might have been adequate when the top tier was 100 Mbps, but then starts to fail when the top tier is 1 Gbps.
As the top tier is increased, the utilization threshold to take action must be correspondingly decreased to maintain consistent QoE.
- Building the storm-proof network — mid- to long-term strategies
In the mid-term over the next several years, operators need to quicken the transition from 3.0 to 3.1. All top tier subscribers should be moved to 3.1 modems. It’s time to start upgrading plants to 1218/85 or 1218/204 MHz. Fiber deep networks continue to be our long term goal as fiber is pushed closer to the home, increasing plant reliability and reducing OPEX. Distributed access architectures including RPD and RMD will take hold, especially on longer fiber runs where they provide the most performance gains. And the IP video migration should be in full swing with technologies like M-ABR supplementing it.
What does the distant future hold? Nobody knows for sure and the crystal ball is cloudier than ever. Bandwidth usage may never snap back to its original pre-coronavirus levels. Bandwidth will obviously continue to grow into the 2020s, although the rate of growth has a lot of debate. The problem experienced during coronavirus is only a sampling of what will happen in the future.
Operators and vendors need to begin working now to upgrade the HFC plant to support the future bandwidth growth.
While DOCSIS 3.1 can carry us a very long way, the industry is working on new technologies today to be ready for the future. It is more important than ever to plan and start to implement our long-term strategies like 10G, especially for the upstream — cable’s Achilles heel. This is what the Cable 10G initiative and DOCSIS 4.0 specification are all about. DOCSIS 4.0 will give us a choice of two options: full-duplex (FDX) technologies for N+0 plants and extended spectrum (ESD) for N+X. Both of these technologies are enabling multi-gigabit upstream tiers to be competitive with 10G PON symmetric offerings.
- Conclusion
The coronavirus BW surge gave us 12-18 months of upstream usage growth overnight. And despite the surge, the DOCSIS network continues to work very well. But this event has shown weaknesses in the system that need filling to make it through the near-, mid- and long-terms.
Looking into the distant future, BW will not snap back after the pandemic. It will continue to grow through the decade. Operators need to begin upgrading their networks for the future. While DOCSIS 3.1 continues to be the workhorse today, development of 4.0 products will enable symmetric multi-gbps services later this decade.
Broadband kept society and the economy running in these hard times. This is arguably its most significant contribution to society in its short life-span. Hats off to everyone who has created and helped make the Internet and broadband available to society.
While collectively we should all be proud of the broadband infrastructure we created, there is still much work to do. The BW surge exposed cracks in the system. This should serve as a wake-up call to make improvements to the infrastructure. It’s time to start upgrading the network for the demands of the 2020s.
—
Source: Data from NCTA member companies and others
—
Figure 1. NCTA COVID-19 Dashboard: National Peak Internet Growth
Figure 2. Coronavirus BW surge impacts across time of day
Figure 3. CommScope/Arris Basic TRAFFIC ENGINEERING FORMULA
John Ulm,
Eng Fellow, Broadband Systems,
CTO – Network Solutions team,
John is an Engineering Fellow for CommScope investigating advanced technologies for broadband systems, John was inducted into the Cable TV Pioneers’ DOCSIS Inaugural class. Research includes HFC, fiber deep and FTTx migration strategies; DAA; DOCSIS 4.0; and network capacity and IPTV modeling. During three decades with YAS Corp, Motorola, and Arris, John was an architect of the industry’s first cable modems and primary author for DOCSIS 1.0/1.1.
Shutterstock
