Machine Learning and PNM
By Brady Volpe
Yes, it’s real and yes, it’s deployed and working!
The day has arrived, Skynet is here! [1]. Fortunately, no to Skynet and in more good news our machines are in the cable plant and they are learning. They have been deployed into the cable world and they are finding the nasty impairments we knew were there. Machine learning and PNM are making the technician’s job easier on many levels. It’s much easier to find the impairments, it’s significantly easier for technicians to know what the impairments are in advance, and it’s easier for field force operations to manage their technicians in an effective manner.
What is machine learning and PNM?
For years there has been growing excitement about how machine learning (ML) can help improve the benefits of proactive network maintenance (PNM). Countless white papers and SCTE Cable-Tec Expo presentations have covered the topic. Imagine how great it would be to have machine learning combined with PNM to help identify plant impairments. Well, that time is now. Machine learning by definition is a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention. Sounds perfect for PNM and for cable operators, too.
What is the benefit?
Machine learning and PNM have promised to make the use of PNM data super easy. Anyone who has used PNM has likely experienced data overload. Looking through lots of full band capture (FBC) plots or tons of receive modulation error ratio (RxMER) per subcarrier plots in DOCSIS 3.1 can be time consuming. We do this manually because we are trying to find impairments in the outside plant (OSP) or in subscribers’ homes and then fix the root cause. Machine learning does this sorting for us and then it just tells us if the impairment is in the outside plant or in the home. Now we know where to dispatch our technicians and can send the correct technician to the correct location. This saves time and helps reduce aggravation for the subscriber and the technician.
No more sending techs to subscribers’ homes when the problem is obviously in the OSP and vice-versa.
Is it really deployed in production environments?
Yes, this technology is available now and is continuing to get better with every impairment. Strange to say but true.
As I mentioned previously, we have been working for years on this in the lab and now we have finally deployed it in production environments. The initial challenge in deploying to production environments had to do so with a lack of training data. Machines must learn, too, but the collection of data as required by the machine was a challenge.
So, what is training data? Training data is a “nice to have” feature where you can pre-train your machine learning engine so that it is already impairment smart. Basically, the machine learning engine is already knowledgeable of what certain impairments are. So, this should be easy right? We all know what roll-off is for example. It’s not that easy for a machine at first. It needs to learn examples of these impairments. For example, training data would consist of impairments such as roll-off and tilt. The machine learning engine would then be able to immediately identify these impairments when it sees them in a subscriber’s home and notify the cable operator that a subscriber has too much roll-off in their home. Great, we have this data, but the training data must be presented in a certain way so that the machine can actually learn from it correctly. We also need lots of training data. Ideally, millions of examples of each impairment type! Lacking the data with the right parameters, we had to be innovative and create an engine that could detect roll-off without the training data. We had to teach our machines to learn and identify impairments in a different way. We succeeded!
Figure 1. Clustered FBC modems via machine learning and PNM — 100% correlated confidence factor of the presence of a non-periodic wave due to water in coax. Impact to users is very low SC-QAM levels at 650 MHz, most likely impacting data service.
What are we finding?
So, what are we finding with machine learning and PNM? Pretty much exactly what we expected to find. Lots of impairments that we knew were there but were often difficult to find. The most important of which is that many times impairments are in the outside plant and not in the subscriber’s home.
The take-away? We often send technicians to a subscriber’s home when we should be sending techs to the outside plant. This causes two problems:
- it wastes money to roll a truck to a subscriber’s home when the problem is not there
- it annoys the subscriber, especially during a pandemic, to have a tech show up and not be able to fix the problem — when the problem is not there
What are we finding? Machine learning and PNM are extremely good at sifting through massive amounts of data and then telling us where the problem is — in the outside plant or in the home. Machine learning can do in seconds what would take a human days to do. This is amazing. Figure 1 shows just an example of a cluster of modems with an outside plant impairment. These modems have been clustered based on their full band capture data, which shows all RF signals in the downstream. Here we see a non-periodic wave pattern, which we have recently come to understand identifies the presence of water in coaxial cable thanks to the great work of our friends at Comcast.
Figure 2. Cluster of several subscriber modems whose RxMER is too low to support 4096-QAM due to outside plant impairments.
Figure 2 shows a cluster of modems evaluated by machine learning and PNM on RxMER for DOCSIS 3.1. This example is of particular interest because it identifies a group of several subscribers who have an impaired DOCSIS 3.1 OFDM downstream channel. These subscribers will not be able to receive the desired 4096-QAM bit loading across the entire downstream service because the RxMER is lower than 39 dB. If these are gigabit tier subscribers, they will likely not be able to achieve gigabit speeds. Having a tool, such as that shown in Figure 1, is very valuable in helping the technician resolve speed tier issues for subscribers paying for these services.
Next steps
We are just starting on the machine learning and PNM journey. Think back on when we started on the first steps of PNM. There were a lot of lessons learned. Now PNM is mature, and most cable operators are using it with great success and find it to be a valuable tool.
Our next steps with machine learning and PNM will be to mature machine learning to the point where each day you will get several tickets from your PNM server. Each ticket will provide you with everything you need to know about the possible future impairments you may have. The PNM-ML (proactive network maintenance machine learning) application notifies you daily of impairments, where they are located, what the impairment is, and what the likely solution to the impairment is. Your technician is dispatched to the location with the knowledge of what to look for and the inventory already in the truck of what he or she will likely need to repair. Machine learning and PNM will provide this information in advance.
This technology is very timely. At no time in our history have more people been dependent on our infrastructure. Due to this at no time have we been more closely scrutinized and under the possibility of regulation. Keeping our subscribers happy and keeping our service and reliability top notch is one way we can ensure the sanctity of our future. And hopefully prevent a technician from saying those dreaded words “I’ll be back” to a subscriber.
[1] Skynet is a fictional artificial neural network-based conscious group mind and artificial general superintelligence system that serves as the antagonistic force of the Terminator franchise.
Brady S. Volpe
brady.volpe@volpefirm.com
Brady Volpe is Founder of The Volpe Firm, Inc and Nimble This LLC. He has nearly 30 years of broadband cable and telecommunications industry experience specializing in RF, DOCSIS, PNM, and Internet Protocol. Mr. Volpe has been providing a wide range of troubleshooting, design solutions, services and seminars for cable operators and broadband companies specializing in DOCSIS, System Design, PNM, and Troubleshooting. He is a highly respected published speaker, both domestically and internationally. He also hosts a popular industry YouTube and Podcast – “Get Your Tech On“. Mr. Volpe has a MSEE with honors and a BSEE.
Shutterstock
