The publisher just completed its 25th 5G NR benchmark study. For this endeavor they collaborated with Accuver Americas, Rohde & Schwarz, and Spirent Communications to conduct an independent benchmark study of the DISH Wireless 5G Open RAN network in Las Vegas, Nevada.
The publisher conducted this study in collaboration with Accuver Americas (XCAL-M and XCAP), Rohde & Schwarz (TSME6 scanner) and Spirent Communications (Umetrix Voice and Umetrix Data). SRG is responsible for the data collection and all analysis and commentary provided in this report.
Testing took place over a two-day period in late May. The publisher leveraged up to two Motorola Edge+ smartphones on the DISH network and up to two Galaxy S20 smartphones on the T-Mobile network (primarily n71 for comparison purposes). The publisher tested full buffer downlink/uplink/simultaneous transfers with HTTP and UDP while stationary and mobile. The publisher also tested voice services, specifically VoNR on DISH and VoLTE on T-Mobile. The publisher also did latency/jitter stress tests using a low bit rate UDP data transfer to the Umetrix Data server,
Prior to sharing a summary of the results, the publisher noted the high degree of difficulty - entirely new network, 5G SA with carrier aggregation, VoNR, and the use of AWS. However, consumers don't know/don't care about the network architecture. They only want to have a good and consistent user experience.
End user data speeds could be well within expectations, excluding HTTP uplink which meaningfully lagged expectations. However, the seemingly on-again/off-again use of carrier aggregation was problematic, as was RF coverage, which is mentioned in the next bullet. Voice services (VoNR) could deliver very high voice quality with MOS well above 4.0, only to be followed by very poor voice quality, to the point of being unintelligible (below 2.0 or no MOS score achieved).
We documented a significant difference in Band n71 coverage (RSRP) and quality (SINR) between the DISH and T-Mobile networks, and largely due to a significant difference in the number of unique cells (PCI counts). More often than not, a drive test resulted in the phone on the DISH network reverting to the AT&T network.
DISH needs to improve and optimize network coverage, AWS may need to focus on reducing the very long latency tail and VoNR consistency, and the Open RAN vendors should continue working on improving VoNR and scheduling efficiency. The 5G clock is ticking.
Highlights of the Report include the following:
Acknowledgements
The publisher conducted this study in collaboration with Accuver Americas (XCAL-M and XCAP), Rohde & Schwarz (TSME6 scanner) and Spirent Communications (Umetrix Voice and Umetrix Data). SRG is responsible for the data collection and all analysis and commentary provided in this report.
Publisher Methodology
Testing took place over a two-day period in late May. The publisher leveraged up to two Motorola Edge+ smartphones on the DISH network and up to two Galaxy S20 smartphones on the T-Mobile network (primarily n71 for comparison purposes). The publisher tested full buffer downlink/uplink/simultaneous transfers with HTTP and UDP while stationary and mobile. The publisher also tested voice services, specifically VoNR on DISH and VoLTE on T-Mobile. The publisher also did latency/jitter stress tests using a low bit rate UDP data transfer to the Umetrix Data server,
Degree of Difficulty
Prior to sharing a summary of the results, the publisher noted the high degree of difficulty - entirely new network, 5G SA with carrier aggregation, VoNR, and the use of AWS. However, consumers don't know/don't care about the network architecture. They only want to have a good and consistent user experience.
Consistently Inconsistent
End user data speeds could be well within expectations, excluding HTTP uplink which meaningfully lagged expectations. However, the seemingly on-again/off-again use of carrier aggregation was problematic, as was RF coverage, which is mentioned in the next bullet. Voice services (VoNR) could deliver very high voice quality with MOS well above 4.0, only to be followed by very poor voice quality, to the point of being unintelligible (below 2.0 or no MOS score achieved).
RF Coverage was Suboptimal
We documented a significant difference in Band n71 coverage (RSRP) and quality (SINR) between the DISH and T-Mobile networks, and largely due to a significant difference in the number of unique cells (PCI counts). More often than not, a drive test resulted in the phone on the DISH network reverting to the AT&T network.
Opportunities Abound
DISH needs to improve and optimize network coverage, AWS may need to focus on reducing the very long latency tail and VoNR consistency, and the Open RAN vendors should continue working on improving VoNR and scheduling efficiency. The 5G clock is ticking. Table of Contents
1.0 Executive Summary2.0 Key Observations5.0 Test Methodology6.0 Final Thoughts
3.0 RF Related Performance Results and Analysis
4.0 VoNR and VoLTE Results and Analysis
Index of Figures & Tables
Companies Mentioned
- Rakuten