Home » NFV
Category Archives: NFV
Holiday greetings from sunny and mild Lake Norman, North Carolina (sunrise shown – unaltered photo). There are a lot of follow-ups to cover, and, if reports are true, there may even be a settlement between the Attorneys General and T-Mobile/ Deutsche Telekom/ Sprint/ Softbank prior to their trial start on Monday (hope springs eternal).
Many thanks for the multitudinous comments on last week’s Thanksgiving book review article. We are not turning into the New York Times Book Review (won’t even try) but there’s a lot to discover and learn from the activities of our predecessors. We will have a similar article on Steven Coll’s 1986 classic outlining the events that lead up to the breakup of AT&T on December 29. Preceding that, we will have a “Three Companies to Watch” special TSB on December 22.
A final thanks for the many referrals that we have had over the past month – over 250 new readers have been added. If you know someone who could benefit from this column, have them send a request to firstname.lastname@example.org and we will get them on the list. We are also in the process of revamping the website (end of January) and promise more things in 2020 (including a merchandise fundraiser for the Davidson College Jay Hurt Hub for Entrepreneurship and Innovation).
This week, we will lead with a discussion of a deep topic – rethinking the wireless (and wireline) network operating system. As mentioned earlier, we have several TSB Follow-Ups.
Programming Tomorrow’s Network
Within wireless communications networks, there are multiple pieces of hardware, each running its own operating software. Each needs to operate to a given specification (usually a 3GPP or LTE Release standard), and there are likely additional requirements placed on the suppliers by the local operators.
This model worked reasonably well when voice and text (using the SS7 TCAP standard) constituted the majority of activity. However, the interest in pushing applications (e.g., WhatsApp owned by Facebook) deeper into the network has created a gap between legacy product development and entrepreneurs. On top of this, there is a need to cost-effectively provide access to less developed areas. On top of this, data growth continues to drive up costs, which create pressures on carriers (and, as a result their suppliers) to deliver a better experience and greater profitability.
This has forced two things to occur:
- Greater network sharing (predominately radios and transport) between network operators. CBRS is the beginning of this trend in the USA (see TSB on CBRS here); and
- Separation of hardware (e.g., a shared radio) and operating software (which may be custom to the operator).
Doing all of this in a secure environment is a challenge. Developing new operating systems amidst a global shortage of software development talent (and recognition of venture capital and other investors that this can be a value-producing endeavor) is an additional challenge. Integrating any operating system changes into the stream of concurrent innovations (e.g., 5G Standalone equipment development, increased mobile edge computing deployments, etc.) requires coordination. Creating competitive advantage in addition to achieving cost reduction targets adds to the heap. It’s like replacing Windows yet expecting no change in how current and future versions of Excel and PowerPoint will work.
We outlined the AT&T efforts in this space in a previous TSB (link is here) but think there’s a few “no brainer” areas where application developers and carriers should come together to improve experience.
- Voice calling. This experience is essentially the same across carriers:
- There is a non-real time contact list that is invoked through a clumsy, 1990’s dialer scheme (see nearby picture);
- There is no voicemail ubiquity within the carrier community (there is at the app layer, however, for WhatsApp, Google Voice, and others);
- To the best of my understanding, there’s no way of automatically integrating stored voicemails into CRMs such as Salesforce;
- There is no reminder or follow up function on voicemails (think how Gmail does this with emails);
- There are inconsistent methods of identifying spam calling (and any other incoming call for that matter);
- There’s no way of knowing any details or status about the party I am calling (such as whether they are on the phone or whether they have made a call in the past five/ten/fifty minutes or even the last day – think the notification scheme for apps such as Skype, etc.);
- For incoming calls, there’s minimal context and no ability to instantly locate/trace the incoming caller (mobile edge computing could fix this pronto and you could see that the call showing 704/Charlotte area code is really originating from Omaha);
- There’s no ability to interrupt a current call (e.g., spouse calling), a call feature common in contact centers (whisper tone);
- There’s no common messaging portal incorporating LinkedIn, Facebook, carrier, WhatsApp and other sources;
- Integration between conferencing services such as Zoom or Skype and the mobile device have not materially changed in 20 years. Still a phone number plus an access code and an announced name.
Is it any wonder that Google Voice, Facebook, and WhatsApp are succeeding and that carrier voicemail solutions are flat to declining? Customers are communicating more than ever, but they are just not into that 1990’s dialer.
To change voice, the interaction between a customer’s contact list (directory), the universal contact list (macro directory), storage (voicemail), availability (presence/ proximity) and the network needs to change. This can all be done faster within the network and is a prime example of how operating systems can and should be rewritten.
Voice application (the dialer provider) should be a choice. It should be portable and interoperable. It should be driven by a microphone and intelligence, not by typed search strings into contact list applications. And the private directory should have live updates (if allowed by the directory listing). The integration of applications functionality deeper into the network can do this, and advancements will occur a lot faster than we see today from the carriers.
- Predictive Analytics (and Customer Care). One of the eye-opening experiences I had with my Flash Wireless experience concerned troubleshooting device issues (Flash had a heavy Bring Your Own Device base). As an MVNO, we tried when possible to go the extra mile if the issue was device-related as opposed to a network issue. We formed a checklist which could easily be databased in today’s environment. Some of the important topics included:
- IoS or Android version
- Recent activity (e.g., voice over Wi-Fi connectivity issue vs the network, messaging activity, new apps downloaded, Wi-Fi vs network data access, location)
- Port-in provider (experience expectations)
- Phone age (and purchase source if it came from one from a known vendor)
- Customer lifecycle age (pre-first bill; first 90 days; over 180 days; etc.)
The number of possible iterations quickly grew, especially since we were in a 3-carrier MVNO environment (location in section b. above really mattered for some of our network providers).
A system that continually interacts with the network could do a better job of measuring data and device quality. If a customer had a service need, problem identification could be instant and highly accurate. Success would not be determined by the smartest care expert, but by the network (and the collective experience of all previous users who had ever used the network in that location at that specific day/time). The cost of caring for older devices could be calculated with high confidence.
To make predictive analytics work, measurement software needs to be pushed further into the network core. Economics aside, if the problem can be remedied by a carrier sharing partner, that can be done instantly through the operating software (not through a SIM setting). Anomalies can be detected for individual users and alerts can be delivered. If the problem was with the provisioning process, for example, the device could be re-provisioned right away (in a nearby store or over the air) or overnight.
The network can be the service expert if issues can be detected quickly. With the consolidation of device models (e.g., more iPhone 8, X, XR, XS, and 11 models in service than ever before), there’s plenty of correlations to be determined (e.g., Sprint iPhone XR users living in Somerset, Kentucky that have activated service in the last 90 days). The result of greater analytical capabilities built into the core could result in dramatically lower cost for customer care.
These are two of probably ten or more use cases that demonstrate the value of rewriting equipment operating systems. This will be an evolution, but not one that is done simply to lower costs – there are many product and customer experience benefits that could create competitive advantage.
Qualcomm Snapdragon 865 Chipset specifics revealed. Increasingly, what’s contained in Qualcomm’s chipsets finds its way into the subsequent generations of smartphones. If that is true, we should expect to see more camera focus (new chipset accommodates up to 200 MP cameras), 5G networks (full support), and faster displays (supporting up to 144 Hz). It was also interesting to learn that the Snapdragon 855 would also support Dual SIM/ Dual Standby (more details on that finding from this XDA Developers report here). As we discussed in last week’s TSB, the Apple XR/XS/ XS Max was the first lineup to support Dual SIM/ Dual Standby – Android development efforts in this area have been slower to emerge. The Qualcomm 855 should be the turning point and we should see the capability available on new devices in 2020. According to the XDA Developers article referenced above, they worked with Gemalto to enable eSIM support within the Qualcomm Secure Processing Unit.
One additional note about the Snapdragon 865 is its support for the Android 11 IdentityCredential API. This would allow, among other things, the ability to store your driver’s license in Android and it would be accepted as a proper form of identification. The complete video of Day 2 which has the details on the 865 are here – the discussion of Dual SIM/ Dual Standby starts at minute 31. The Snapdragon 865 spec sheet is also available here.
DOJ Calls Out Carriers on Remote SIM Provisioning (RSP) Collusion
The day before Thanksgiving, the New York Times ran an article describing the settlement between the Justice Department, the GSMA (standards body) and some US wireless carriers (presumably including AT&T and Verizon) over possible collusion surrounding the development of eSIM device locking.
The Times article is sparse on details – Assistant Attorney General Delrahim’s letter (here), however is not. Here’s what was found concerning the current RSP process (actual findings – emphasis added):
First, RSPv2 requires consumer-users to express affirmatively their intent to switch profiles each time the eSIM toggles between profiles or networks, thereby preventing the eSIM from automatically switching (or optimizing) between profiles. Dynamic or automatic switching is a potential competitive threat because it could lead to a service where a device efficiently selects, on behalf of the user, which profile to use in any given situation. For example, the eSIM could switch services if it detects stronger network coverage or a lower cost network, providing consumers with better or less expensive service. The prohibition on automatic switching would tend to prevent at least one existing operator from offering a new innovative service using an eSIM. That is, in order to offer the new service, the operator would have to convince smartphone manufacturers to forego complying with the RSP Specification.
Second, RSPv2 prevents an eSIM from actively using profiles from multiple carriers simultaneously. Multiple active profiles is a potential competitive threat because it would allow a user to divide usage across operators. For instance, the user could actively maintain two profiles on one device if he or she wanted to receive work-related phone calls to one profile and personal phone calls to another profile, all while carrying only one phone. The user could also actively operate profiles optimized for different coverage areas or for international travel. Although there appear to be technical challenges to allowing multiple active profiles at present, the single active profile requirement in RSPv2 serves as a roadblock to additional disruptive innovation that could solve these technical challenges.
The DOJ’s issue was not only with these two outcomes (which are unmistakably anti-competitive), but with the entire approval process used by the GSMA. Everyone agreed to comply with new process, and, with the advent of the new Qualcomm 865 chipset described above, it’s likely that switching between networks will be placed as a consumer choice with the opportunity to mute future notifications (similar to the roaming notifications process followed for over two decades) and also to allow multiple networks to be accessed simultaneously (making data network selection easier for cable MVNOs and others while potentially keeping voice on the MNO network).
Adam Koeppe Takes the Stage in Vegas (While His Boss is on a Separate Stage in Vegas)
Given space constraints in this week’s TSB, I am going to keep this excerpt short (perhaps we will cover in another TSB this month), but, if you want to know what Verizon is doing with respect to network deployment, listen to his Well Fargo talk with Jennifer Fritzsche here or read the transcript here. Adam covers the AWS 5G Edge announcement, fiber deployment strategy, CBRS (and Enterprise LTE solutions pairing CBRS and millimeter wave spectrum bands), Broadband to the home and relationship to cable MVNO, and a few other topics. Less spin is good for Verizon, and Adam is a “balls and strikes” interview.
What Markets Will See the Greatest Improvement to Sprint/ Boost if New T-Mobile Actually Occurs?
We had been thinking about this topic for a while, and accessed publicly available RootMetrics data (2H 2019 measures only to be most current in our assessment) to see where the current gap between T-Mobile and Sprint exists.
To no one’s surprise, Sprint tends to solely occupy fourth place in nearly each of the 125 markets that RootMetrics measures every six months. How would that performance improve once that Sprint/Boost customer (current device) could access the T-Mobile network?
To determine the greatest impact, we looked at the difference between Sprint and T-Mobile’s Overall Score (perhaps in a future TSB we will dive into the components). As of Wednesday, RootMetrics had published the results of 96 out of 125 markets (77%). The results break down as follows (100 pt scale):
Overall Score Difference Number of Markets Percentage of Total
Less than 3.0 points 19 20%
3.1 – 5.0 points 13 13%
5.1 – 10.0 points 48 50%
More than 10.0 points 16 17%
While there may be debate about the impact to customers for the first two levels (device age could play a significant role in a market where both T-Mobile and Sprint are relatively strong), there’s little debate when there’s a spread in excess of 10 points and the market is not one of the previously announced 5G markets. Here’s a sampling of where Sprint has fallen behind:
- Oklahoma. Below are the most recent charts for Tulsa and Oklahoma City. ‘Nuf said.
- Florida. These are legacy MetroPCS markets for T-Mobile and have very dense coverage. Miami, which was once a priority market for Sprint (non-executive Chairman Marcelo Claure has close ties to the area), has fallen off considerably and is no longer a competitive market for Sprint. Other markets with more than a 10 point spread to T-Mobile include Port St. Lucie (12.2) and Sarasota (11.1). Orlando, Kissimmee, Tampa, and Jacksonville have a 5.1 – 10.0 spread. The other markets are waiting to report.
The remaining markets are both big and small metro areas:
- Atlanta (fast growing area, large market, 5G market)
- Baton Rouge, LA (been a weak network for Sprint for many years)
- Charlotte, NC (fast growing area and second home to most of the financial services industry)
- Denton, TX (North Dallas suburbs)
- Kansas City, MO (very odd as it’s Sprint’s current HQ and a 5G market)
- Louisville, KY
- Memphis, TN
- Nashville, TN (fast growing area)
- San Antonio, TX (Sprint PCS dominated this market because of its design; large market)
- Wichita, KS
Interestingly, no Northeast, Northwest, Southwest or California markets with large gaps. We will update this list in January once RootMetrics has completed their 2H 2019 metro studies.
That’s it for this week. Next week we will begin our discussion of 2020 trends unless events dictate otherwise. Until then, if you have friends who would like to be on the email distribution, please have them send an email to email@example.com and we will include them on the list.
Have a great week… and GO CHIEFS!
Greetings from Lake Norman/ Davidson, North Carolina where everybody is working (including the neighbor kid, Caleb, pictured with the Editor and yours truly). This week, we’ll look at two studies that examine data usage, and try to deftly explain AT&T’s efforts to improve their cost structure and competitive cloud position. But first, a follow up from last week’s column.
Follow-up Idea to the Apple Card TSB
I did not anticipate the overwhelming response the Apple Card TSB would receive (Craig Moffett did a mid-week post on the concept and CNBC picked it up here). I have had about a dozen in-depth conversations on the Apple Card TSB this week with several of you and I think we have come up with the ideal suggestion for Apple: 2x Daily Cash towards your monthly phone payment. So, if you get 2% back from using Apple Pay (say $2 on a $100 grocery purchase), that would become $4 towards your monthly phone installment. There could also be an incentive to pay the phone installment first or sooner.
This promotion would reward existing Apple users with a new phone just from using Apple Pay (or buying other products from the Apple family, or just using the physical card), and would be a way for existing Android users to make the switch to Apple for a discount (or to lessen the blow of being underwater from a low trade-in value). The opportunity to “earn” your way to a free iPhone increases Apple Pay (or card) usage, which increases value, etc. And, customers would have the ability to worry less because their phone is never locked to a wireless carrier. Not that the marketing department at Apple needs any help, but if this ends up being the incentive, you heard about it first in TSB.
There’s a Deeper post on the Apple Card that might be very useful for those tracking the topic closely. Thanks for all of the ideas and please keep them coming.
Two “A Ha” Reports This Week Generate Minimal New Insights
Meanwhile, we had two “a ha” reports come out this week. The first was a supposed “scoop” from the Wall Street Journal that customers were being duped by their internet service provider (AT&T, Comcast, Time Warner Cable, others) into paying more for higher throughput speeds and receiving nothing in return. This was not a passing consumer interest story, but had substantial article placement, a video, and even a podcast to support it. (Both the video and the podcast take more time to analyze the complete array of things that could contribute to slower speeds in the home, such as interference or neighborhood congestion, and, inadvertently, make a very strong yet indirect case for paid prioritization).
At TSB, we seek first to understand, but at the end of the day, it’s very difficult for us to have as much excitement as the Wall Street Journal did about possible overpayment for the following reasons:
- The consequences of paying too much for home broadband are not substantial. Analyzing whether customers are over-insured on their home or auto insurance is likely to have 10x more impact than the $10-20/ month being paid for a premium tier. Consumers should spend their time wisely.
- Pursuing an alternative to your current speed (buying a premium tier) usually carries no contract and can be immediately evaluated. Most ISPs do not have a change fee.
- The sample size used in the article is not large enough to make a national conclusion (34 multi-streaming tests with a majority made in New York City is good to know, but not conclusive that Charlotte or Phoenix or Tampa customers will have the same experience).
Bottom line: There’s nothing wrong with 1Gbps speed even if you don’t need it. There’s also nothing wrong with buying the nearby Dodge Challenger SRT Demon (840 hp – don’t we all need one?) if you can afford that. Faster Internet speeds from your ISP could help, and so could a new Access Point or Cable Modem.
The second study that came out this week deserves more attention because of its sample size and implications to the industry. The University of Massachusetts and Northeastern University used a mobile app called Wehe to track the Internet usage patterns of 126,249 mobile users over several months (over 1 million samples). The summary (which is not surprising to most cellular users) is that average speeds for popular sites such as YouTube and Netflix are slower than expected. Here’s a summary chart of the findings for AT&T and Verizon (June 11, 2018 is the day that Net Neutrality rules were made optional) :
What this shows for Verizon is precisely what we discussed in last week’s TSB: 480p customers get consistent YouTube data throughputs at 1.9 Mbps, and 720p customers receive YouTube data throughputs at 4.0 Mbps. This is consistent with their advertised plans (why NBC Sports and Vimeo are not included in the throttle was probably the topic of several Verizon Wireless staff meetings this week). I am sure if they were to upgrade a device to 1080p throughout, they would have a third tier that shows around 8-9 Mbps. For the record, AT&T has disputed the findings and CTIA in 2018 issued a statement disputing the findings.
Bottom line: If you want video resolution that matches the maximum capabilities of your smartphone, you might have to pay more. To last week’s comment, this could be a differentiation point for one of the challenger carriers (e.g., T-Mobile resuming their un-carrier ways and making 720p or 1080p the new “basic” video viewing tier).
AT&T’s Herculean Lift
Condensing AT&T’s Network and Business strategies into one TSB is a challenge because of the breadth of problems they are attempting to solve. Unlike the AT&T of the 1990s (more focused on communications innovations like the Internet and mobile), this generation’s AT&T is much deeper and comprehensive (focusing on information delivery, relevance, timeliness, analysis, and action).
It’s important to remember that AT&T as we know it today is really a 12+ year-old company. BellSouth was formally acquired at the end of 2006 and with that came the formation of “The new at&t”, replacing Cingular Communications. That was followed by a substantial network integration project which ended in 2009. While we think about AT&T as a legacy brand, the common platform that we see today is only a decade old.
Since 2009, it’s been a wild ride for wireless:
- 4G LTE services have been introduced and run their product life cycle (with network traffic growing 40-60X over the decade)
- Smartphones and tablets have become the primary source for information retrieval and entertainment
- Many-to-many communication has become a global standard thanks to social networks
- Commerce and mobility are inextricably linked
- Wireless Machine-2-Machine (screenless) devices have produced an enormous amount of data and dramatically improved utility and productivity
As we discussed in a TSB a few weeks back, mobile growth contributed to fiber infrastructure growth. More towers and small cells spawned additional conduits, trenches, and pole attachments. All of this data needed to be stored and processed, and server farms led to cloud computing sites and eventually hyperscale data centers. The infrastructure and mobile landscapes changed dramatically with LTE network demand.
What didn’t change as quickly were the network operating systems and business processes that determined the efficiency and profitability of AT&T. Integration was the exception rather than the rule, data needed to solve customer problems was available to some departments and not to others and obtaining merger synergies by negotiating better rates with the same menagerie of vendors (often with extended terms) made breakthrough change harder to achieve. Growth was hindered by incompatible product roadmaps and the quantity of equipment required to accommodate data growth was becoming untenable. Something had to give.
In 2013, AT&T began the painstaking process of separating computer processing from separate software development/ integrations for each of its network vendors with the introduction of Domain 2.0 (see whitepaper here). The picture below from the whitepaper summarizes AT&T’s direction and provides a good roadmap for their recent Microsoft, IBM, and Dell announcements:
Routers (Cisco and Juniper being AT&T’s main providers) and optical infrastructure (Ciena and Cisco) were the first focal points of Domain 2.0, and it manifested itself in the creation of a mobile packet core. Key Domain 2.0 vendors as of the end of 2015 were Cisco (via their Tail-F Systems purchase), Ericsson, Nokia (with Alcatel/ Lucent), MetaSwitch (private UK-based company), Affirmed Networks (Evolved Packet Core or EPC), Amdocs, Juniper, Fujitsu, Brocade, and Ciena. There will be others as this new platform makes it much easier to attract new entrants.
Being able to separate operational instructions (core optical equipment and routing functions – the “do this/ do that”) from monitoring, alerting, and change management functions, and then standardizing the structure of the latter across the vendors mentioned above is no small task. All of that had to be done (or at least close enough to completion) to have the Microsoft/ IBM and Dell announcements. Here’s a brief summary of what each does:
The IBM agreement takes the AT&T Business’ internal application infrastructure and moves it to the IBM cloud (this will be a consolidation from multiple providers). As a reminder, IBM’s acquisition of Red Hat allows it to service hybrid cloud configurations. Much of the NFV Infrastructure Cloud will move to IBM. In turn, IBM runs a very large cloud business for third-party customers (top part of the diagram). This moves the network closer to IBM’s (particularly AI or Artificial Intelligence) software.
It makes a lot of sense moving internal application infrastructure to IBM given the strong network relationship the two companies have had for two decades.
The Microsoft agreement takes non-network infrastructure applications and moves them to the cloud. Microsoft is another very large cloud provider but also the developer of Windows, Bing, Office 365, Skype, and Xbox software (Microsoft’s IPTV platform, Mediaroom, was sold to Ericsson in 2013). It gives Microsoft an easy path to both Mobile Edge Computing (MEC) and low latency network APIs. Unlike the IBM agreement (which appears to have a heavy AT&T Business focus), the Microsoft agreement appears to be more open-ended. Xbox wins in a 5G world that’s also tied to AT&T Fiber homes. So does Skype on mobile, desktop, and tablet applications. More video and less latency demand a different way of thinking about network management. Microsoft will help push the product envelope in the home and in the workplace.
The Dell agreement brings the Austin-based company into Airship (an organization governed by the OpenStack Foundation), described as “a collection of loosely coupled, but interoperable, open source tools that declaratively automate cloud provisioning and life-cycle management utilizing containers as the unit of software delivery.” AT&T needs more network edge capacity flexibility and is engaging Dell to help them (via Airship) improve network server delivery. More details on the agreement can be found in this Fierce Telecom article.
Bottom Line: AT&T’s July and early August announcements show their early hand cloud and edge technology partners. They have been at the leading edge of Network Function Virtualization (NFV) and Software Defined Networking (SDN), and are finally bringing together all of their infrastructure platforms into a common architecture. This is the most efficient (really the only) way to enable rapid, profitable local growth. It will be interesting to see how other carriers (specifically Verizon) follow AT&T’s lead.
Next week, per many requests, we will be discussing the role of Citizens Band Radio Services (CBRS) in the communications landscape. Please note that next week’s TSB may not be delivered until Sunday evening due to the Labor Day holiday.
Until then, if you have friends who would like to be on the email distribution, please have them send an email to firstname.lastname@example.org and we will include them on the list.
Have a terrific week!