Long Live the Past, with an Eye to the Future
By Charles Allred
I think we were pleasantly surprised at Cox as we approached two million households passed (HHP) by our distributed access architecture (DAA) and its thousands of shiny new RPD nodes! With so much going on, it is tempting to forge ahead and not reflect on how we got here. This staggering growth did not come without the hard work and dedication of many internal teams and vendor partners; from our leadership teams that make the plan, to the engineers taking concept to design, and our many software developers who bridge the gaps between systems ancient and new. Our Project Managers and always busy construction teams far and wide have been pulling new fiber and building vaults for all those shiny new nodes. It has been quite a journey. We have written a few articles discussing our experiences and a few of our DAA learnings over the last two years. Let us talk about some of the new challenges we have faced as technology and implementation plans continue to move this network forward.
Of the many obstacles we have faced, one recent standout comes to my mind. It is a good example of the complexity and diversity we face in the new DAA. As newer, segmentable RPD nodes became available to us we struggled with how to integrate them into our network. The primary concern wasn’t with the RPDs or the CCAP core, it was how we could configure the various combining scenarios for both testing and production, provision the solutions, and manage and monitor them in our existing tools ecosystem. 2×2 service groups with two downstream and two upstream ports were no problem. 1×2 combining, while slightly more complicated on the core, was also possible. Our automation and provisioning system, having been designed end-to-end to support 1×1 RPDs proved to be a much bigger challenge. The CCAP configuration orchestrator was re-coded to support the new combining modes. The automation system responsible for managing the flow of data from the user’s provisioning tools, inventory tracking system, monitoring tools, and ultimately to the orchestrator was also revamped to support the new data model and API changes. Monitoring tools were also updated to enable bandwidth management and to corollate customer equipment to the multiple upstream and downstream ports of the new RPDs.
The process was largely serial. We did not always realize the implications related to what we thought was a relatively modest change in one system and the effect it would have on the next system until development was well underway. Like dropping a pebble in a pond, the resulting ripples had spread to touch nearly every operational aspect of the company. Only there wasn’t just one pebble or one pond. We had to revisit standard node naming conventions that we had used for years! How would we let our field partners and plant technicians know where to find troublesome modems or noise on a plant leg when our tools couldn’t distinguish which physical segment was affected? This led us down the months-long path of determining the affected systems and software platforms mentioned earlier. This is an example of how DAA fundamentally changes the simplest and long-standing operational assumptions.
These challenges forced us to take a closer look at our entire automation and provisioning path. That included going all the way back to initial node designs and how those entered our inventory and capacity planning systems. Did we have the right templates to account for the new nodes and their differences from the analog nodes? Did we fully consider how our new standards affect legacy systems and the users of those systems? An engineer may occasionally change something in a system or process but if that change renders a familiar system object unfamiliar, have we accomplished what we set out to do? One of our original segmented RPD naming conventions did not consider that the “node” not only referred to the analog laser transmitter/receiver pair within the node enclosure, but also a geographic area. When we gave the segmented RPD node a single name the relationship to geographical area was lost. We were forced to revisit the original naming convention and take multiple geographical node names into account.
These small details underscore the importance of open communication between all our engineering and operational teams. No individual holds all the keys or knows the secrets of every dark corner of our network. To be successful we must rely on each other’s combined tribal knowledge and experiences. Whether from our home offices, kitchen tables, or living room couches, the work continues to get done because our company values have instilled a high level of communication and collaboration.
Charles Allred,
Senior Access Engineer,
Cox Communications, Inc.
charles.allred@cox.com
Charles Allred is a Cable Access Engineer at Cox Communications. He spent nine years at Cox in the Phoenix market where he was involved in their first DOCSIS 3.0 deployments. In 2016 he joined the Atlanta Access Engineering team where his current focus is around modular CCAP and remote PHY deployments.
Shutterstock
