Advancing the Echo-System
By Stewart Schley
Comcast’s Larry Wolcott is helping to guide a revolution in telecommunications. By listening.
Larry Wolcott’s perfect day goes like this: Summer morning. Brilliant Colorado sky. Birds chirping. And Wolcott in a trench, splicing cable.
“That,” he says, “is what I love.”
An affinity for the in-the-trenches workings of telecom is the stuff that feeds Wolcott’s soul and breeds a sort of palpable enthusiasm for the work he does for Comcast and the broader industry. Much of that work has revolved around a CableLabs-led initiative called proactive network maintenance (or PNM). It’s Wolcott’s prized project, his professional raison d’etre. It’s a big reason Wolcott, a Fellow and Distinguished Engineer at Comcast and a founding member of the CableLabs PNM working group, was named SCTE•ISBE’s 2018 member of the year. (Well, that and his role in catapulting the SCTE Rocky Mountain chapter to more than 1,000 members last year.)
Drawing from analytical tools embedded in the CableLabs-authored DOCSIS® specification, Wolcott and his colleagues have spent years perfecting the art of attaining critical intelligence from the network in a bid to curtail outages, improve network performance, reduce truck rolls, produce happier customers and liberate technicians to focus on their craft. All at scale.
It started around 10 years ago when Wolcott and a small group of industry colleagues began to ponder the possibility of using cable’s own network instruments — primarily modems and their CMTS companions — to probe the inner workings of the network.
A little-known feature of DOCSIS is a deliberate and repeated self-awareness exercise. Every 20 seconds, modems connected to cable networks issue a declaration, an abbreviated ping that conveys that they’re up, running and able to communicate. Within this simple expression is the making of a revolution in how networks can be monitored and maintained. By taking note of these cries in the digital wilderness, applying known laws of physics and conducting an astonishing fury of calculations, network-watchers can produce astounding intelligence about the state of affairs anywhere in the network. The process mimics the way radar works by detecting echoes reflecting off elements in a signal’s path. “It’s literally a massive mesh network of sensors,” Wolcott explains.
Granular expression
For Comcast, which reported more than 25 million residential broadband connections as of Dec. 31, 2018, the granularity of the analysis is remarkable. Wolcott, who springs about his office in Denver’s hip LoDo district with the bounce of an excited kid, demonstrates. He picks a network node — a localized fiber optic serving area — and goes to work. A map appearing on his computer screen is decorated with color-coded pinpoints and signal quality graphs, telltale signs of possible impairments or looming trouble points. “See this?” Wolcott says, pointing to a fuzzy scramble of icons arrayed across a table. “That tells us something.”
What Wolcott is seeing is an almost real-time expression of the network state. Behind the graphic indicators is a very real, very analog world of copper that becomes corroded, F-connectors that become loose and noise that works its way into the network. Applying known physics constructs, Wolcott’s software lets him know with exacting certitude where trouble looms. A broken red line on the map tells him where cables are buried or strung, and where, within a foot or two, a technician will find and remedy a problem.
For Wolcott, the joy here is not just identifying a likely impairment but addressing it before a customer ever notices a performance issue. “That,” he says, “is the magic of PNM.”
Long road
None of this came easy. Ten years ago, as Wolcott looked over technical papers written by the likes of Cox Communications technology executive Jeff Finkelstein and CableLabs Fellow Alberto Campos, he realized two things: First, he was staring at the makings of a revolution that could change almost everything about network maintenance. Second, harnessing massive amounts of information so that it could be “operationalized” — a favorite Wolcott expression — was going to take everything he had. After all, millions of modems emitting signals every 20 seconds produces a lot of data.
The good news is Wolcott had a lot to give. The son of a military officer, Wolcott, who was born in Germany, spent his childhood moving from town to town, state to state, from Kentucky to New York to Michigan to Illinois to Wisconsin. “Because of that I really struggled making friends, and having a social life,” Wolcott recalls. “I was the quintessential geek.”
When he was in elementary school, though, salvation arrived. An uncle who worked as an engineer for Motorola recommended to Wolcott’s mother that she buy her son a personal computer.
Wolcott fell in love. The Radio Shack TRS-80 became Wolcott’s reliable companion. Wolcott taught himself to code. He published a commercial-grade video game, “Kung Fu Fighter,” in 1986. Importantly, Wolcott picked up on the nuances of a critical technology intersection: the place where software meets machine.
This appreciation for fundamental building blocks of information technology would endure throughout Wolcott’s career. It’s one reason he loves the cable industry: “I fell in love early with the notion of bare metal in cable…with electrons and photons,” says Wolcott.
The real world is an important theme for Wolcott, who maintains reverence for the hard-scrabble world of boxes and circuits, and the field personnel who attend to them. At risk of “sounding like a suck-up,” Wolcott lists Comcast CTO Tony Werner (ultimately Wolcott’s boss) as his business idol. “Tony and I are both field guys,” Wolcott says.
Career path
Wolcott applied his self-taught software acumen in the business world by co-founding a Wisconsin computer-services company that grew into a $9 million annual business. Later, Wolcott went to work as an information technology specialist for a former client, the bicycle-maker Huffy Corp. As the IT market soared and demand for software expertise skyrocketed, Wolcott realized he could be selective about where he worked. He chose Colorado for the sunshine and the snowboarding, working initially for the fiber optic provider ICG and later handling consulting projects for the big players in Denver’s cable economy, including Comcast predecessors AT&T Broadband and MediaOne. Wolcott was intrigued by an industry structure that encouraged collaboration among multiple companies, as reflected by the technology R&D consortium CableLabs in nearby Louisville, Colo. That attribute, plus encounters with a succession of impressive engineering professionals — Wolcott refers to them as “industry legends” — was all it took.
“It was, and is, a magical industry,” Wolcott says.
At the heart of PNM is reverberation. Modems are trained to produce digital echoes, telltale traces of where they are and what they’re encountering. Within the echo is an expression of knowledge that can be teased out by watching waves and applying hard mathematics. Wolcott likens the approach to surveying ripples on the surface of a pond, produced by a stone that was tossed by a kid. Give Wolcott a look at the surface tension of the water and the magnitude and frequency separation of the waves and he’ll tell you exactly where that rock landed.
PNM is similar. “It’s exactly the same math,” Wolcott points out. A modem’s echo is a statement. String together enough statements, couple them with known laws of physics, run the calculations, and you’ve got the makings of an almost indescribably powerful tool for managing sprawling, complex data networks with scientific accuracy.
It’s the “P” in PNM that’s the important descriptor to Wolcott: the notion of proactive monitoring and early impairment resolution. The symmetry here is that Wolcott, who used to disassemble radios as a kid to explore their inner circuitry, is using qualities of the physical world to make it possible. “I’m a physics guy,” declares Wolcott. “That’s where my heart is.”
Stewart Schley,
Media/Telecom Industry Analyst
Stewart has been writing about business subjects for more than 20 years for publications including Multichannel News, CED Magazine and Kagan World Media. He was the founding editor of Cable World magazine; the author of Fast Forward: Video on Demand and the Future of Television; and the co-author of Planet Broadband with Dr. Rouzbeh Yassini. Stewart is a contributing analyst for One Touch Intelligence.
