Flexing DAA’s Muscles with Extended CIN
By Alan Skinner
In 2018, Cox began deploying a new type of access network technology, one that would break the mold of the traditional HFC model which had been used for the last three decades. Based on the CableLabs DAA (distributed access architecture) and DOCSIS 3.1 standards, our remote PHY technology would serve as the basis for nearly all node actions used to relieve congestion and increase customer speeds. One of the key tenets of this new architecture is the use of digital IP-managed fiber nodes connected to core resources by way of a CIN (converged interconnect network).
CIN overview
Cox network engineers designed a CIN that could be deployed in each of our facilities in a nearly identical manner, and which could scale elastically to support small, medium, and large sites. The CIN is based on a leaf-spine-leaf design (see Figure 1):
- one leaf consists of a CCAP core and the routers needed to aggregate those core resources
- another leaf consists of RPDs (remote PHY devices) and the routers needed to aggregate them
- the spine exists to route traffic between those leaves
Figure 1: Conventional CIN architecture
Typically all of this equipment is deployed at each of our MTCs or hub sites. However, one of the design requirements of this remote PHY network was true any-to-any reachability of the elements: the nodes could be logically connected to any core resources, not just the CCAPs in that facility.
With these pieces in place, and with the remote PHY technology having been proven sufficiently reliable and mature, it was time to push the envelope and take further advantage of this routed access network. If one of our facilities was tight on space, or on power or HVAC, but another nearby facility had ample room for the larger core equipment, could we extend the CIN and leverage that any-to-any connectivity? We could, and we did.
Enter… extended CIN
In an extended CIN network, the RPDs are connected to the hubsite via optical fiber as usual, but the video and DOCSIS core resources are located elsewhere (see Figure 2). The existing metro IP network is used to connect them, stretching out the CIN across two (or more) sites. This type of topology can significantly reduce the footprint needed in the remote site, alleviating the need to expand the building or construct a new one.
One example of an extended CIN topology looks like this:
Figure 2: Extended CIN architecture
The larger facility on the left is known as the host site, and this is where the bulky, power hungry equipment resides: CCAPs, core aggregation routers, and PTP clocks. The remote facility on the right need only provide the RPD aggregation routers and can leverage an existing hub router. The RPDs in the field can therefore logically connect all the way back to the host site to get their DOCSIS, video, and timing resources.
Topologies
Cox has settled on three extended CIN topologies, and is in active deployment with the first two. The third is least preferable and not yet in deployment.
- Sub-tended hub hosting. The remote site is configured as a spur of a nearby core facility, and traffic can be routed directly between them. This topology is the simplest design and provides the best performance, and is preferred whenever the existing metro design can accommodate.
- RDC-hosting (shown in Figure 2). The RDC, or regional data center, is the “hub” in our existing hub-and-spoke metro network. The RDCs often have ample space, and already terminate connections from the hub sites, so locating CCAPs here to support a remote site is straightforward.
- Hub-hosted. In this topology, one hub site provides the host resources for another hub site, and the RDC provides transit between the two sites. This design adds the most complexity and latency.
Caveats
There are some limitations to keep in mind when operating an extended CIN. DOCSIS is a contention based protocol with scheduled transmit opportunities for the modems. In the case of remote PHY, the RPDs only provide physical layer signal generation. Bandwidth requests still have to flow upstream to the core, and a grant has to make its way back down through the RPD to the modem in order for data to be sent. Extending the distance that this request-grant cycle has to traverse does put an upper bound on how far the remote site can be located from the host site. Conserving space and capital is nice, of course, but not if it comes at the expense of a degraded customer experience. At Cox we are still fine tuning that criteria.
While this extended CIN arrangement can reduce or eliminate the need for facility expansion, it does not come without some drawbacks.
- Additional network hops provide more points of failure and complicates capacity planning.
- Until the DOCSIS scheduler can be relocated into the nodes, there is a latency penalty to be paid for separating the core and the RPD.
- Consolidating video lineups for two or more sites into a single CCAP can be an operational challenge.
Deploying extended CIN does take a bit of extra work. For example, since the CIN spans multiple sites which typically have redundant transport paths between them, it is necessary to carefully engineer the traffic to take the optimal path in both directions. Remote PHY is highly reliant on an accurate PTP source for the RPDs, and PTP requires symmetry between the endpoints to operate properly. If the traffic between the core and RPDs doesn’t take the preferred path, or takes different downstream and upstream paths, the service will suffer — if it works at all.
Because of the restrictions and limitations of extended CIN, and the potential for a suboptimal experience if not executed properly, at Cox we utilize this solution only very strategically. By default, we stick with our standard tried and true design wherever possible. But for small hub sites with limited space and power, and little growth, deploying an extended CIN can be just what the doctor ordered… especially when weighed against the prospect of building a new facility.
Alan Skinner,
Principal Engineer,
Cox Communications, Inc.
Alan Skinner is a 14-year veteran of Cable Access Engineering at Cox Communications. His current efforts center around distributed access architecture and next-gen DOCSIS technologies, but for the three years prior to that he served as technical lead for IPv6 implementation throughout the company. His responsibilities have also included the design and implementation of DSG and related STB technologies. Prior to joining Cox, Alan spent eight years in DOCSIS engineering at CommScope and Cisco.
Feature Image: Shutterstock
