Network Operator Acceptance Testing of LTE Devices
The (new/old) challenges in accelerating LTE device time-to-market.By Konstantinos Stavropoulos, Anite
click to enlarge
Figure 1. A representative set of lab acceptance tests that 2G/3G operators have considered.
With the advent of LTE, operators are again considering network simulation as a powerful and cost-effective means to verify that the highly anticipated LTE devices will meet the needs of the end customer, the mobile subscriber. Indeed, wireless device testing can be challenging, especially for new technologies. Furthermore, it is essential for operators to go beyond standard conformance test cases and assess the device performance with regard to data throughput, home/preferred network cell selection, multiple handovers, or in general scenarios based on live network configuration parameters.
The type of device tests LTE operators are interested in is largely determined by the following correlated factors: experience; technology; strategy. It is not surprising that 3GPP (GSM/UMTS) operators will be using their 2G/3G lab tests as the foundation for LTE device testing. For these operators, LTE is indeed an evolutionary 3GPP technology, which may differ from GSM/UMTS at the physical layer, but still presents a familiar upper-layer protocol architecture. In these terms, the need for inter-operability with legacy networks may not be regarded as an issue of concern2, at least not as much as deciding when to launch LTE.
LTE device testing is expected to be more challenging for 3GPP2 (CDMA) operators. Although 3GPP have catered for CDMA networks in their specifications, these operators are effectively entering ‘unknown territory'3. It is also important to point out that 3GPP2 operators may have not worked with regulatory bodies such as GCF in the past. As a consequence, and especially if their strategy is to launch LTE soon, these operators are more likely to opt for tests that have not yet been finalised by GCF or for ‘play-safe' tests, i.e. tests that ensure LTE devices will not crash the network.
click to enlarge
Figure 2. A simplified diagram of automated mobile device testing with Anite's ‘SAS'.
In fact, contrary to the launch of 2G and (perhaps, surprisingly) 3G networks, the operator focus for LTE is on data cards and data testing only. Even SMS, the unforeseen ‘killer app' for 2G/3G, is not regarded as high a priority as downlink throughput4. It is therefore no surprise that LTE device testing will concentrate on throughput performance, and on data rate much more than data latency as such. Engineers have used every trick in the book to beef up data throughput for LTE, so operators are now interested to see what subscribers will experience in practice! At the same time, operators have identified other test areas of interest, such as specific frequency bands and bandwidths, consistent with their LTE launch strategy.
Undoubtedly, the lack of finalised LTE standardisation constitutes a major source of uncertainty. With quarterly releases of standards versions still issued as LTE specification is in a state of development and hence instability, some operators could be reluctant to start testing (or launching) devices that may not be compatible with future LTE network/device characteristics. Nevertheless, there is a lot to gain from starting tests soon, and not only for 3GPP2 operators. Also, many subscribers will be glad to get hold of an LTE dongle sooner rather than later, and are unlikely to complain even if they have to physically replace it in a matter of months.
Of course, LTE device testing presents challenges for test equipment vendors too. Some challenges are technology specific; for example, test systems must be able to simulate the rapid LTE network adaptation to channel conditions, based on the short TTI value (1ms) for the new technology. Other challenges are technology agnostic; for example, test automation has become a mandatory requirement of operators, which applies to LTE too. There are also challenges that the uncertainties associated with a new technology accentuate; for example, the ability to identify and rectify what went wrong during an LTE test makes the need for comprehensive and easy-to-use logging tools crucial.
Anite is in the unique position to have addressed these challenges and to be able to help 3GPP and 3GPP2 operators to achieve their goals. With unrivalled experience in providing solutions that accelerate device testing via industry-leading systems such as ‘SAS', distinct features such as TAG (Terminal Automation Gateway) or RTT (Real Time Tracing) and a true focus on customer needs, Anite can make LTE device testing easier.
To conclude, although wireless device testing entails a number of challenges, especially when a new technology such as LTE is introduced, what mobile network operators should do is to learn from past personal or shared experience and make sure that testing starts as early as possible. It is never ‘too early' for operators to take action so that the LTE user experience lives up to expectations. More often than not, things come too late for those who only wait…
Dr. Konstantinos Stavropoulos joined Anite in 2009 as IOT Product Manager, responsible for SAS, Anite's network simulator product for mobile device testing. Konstantinos has worked on areas of diverse nature, ranging from antenna array systems (MIMO, smart antennas) research and mobile network planning/optimisation to the development and management of innovative software products. www.anite.com 
1Anite, the leading wireless test solutions provider, typically employs the term ‘Operator Acceptance Scripts' for collections of such tests. These scripts are written by the operator's engineers or by the Anite Scripts team and run on ‘SAS', the industry-leading network simulator for mobile device testing. The naming of acceptance script packages is usually based on the name of the operator who has specified the use cases of interest in the form of Test Requirements or Test Cases.
2Inter-operability with legacy networks is a high priority for LTE operators, and such test cases have already been specified. Operators do not seem content with LTE existing in isolation from other mobile technologies, which would negatively affect the desirable seamless mobile user experience. This is particularly important since the primary focus of LTE deployments is on data, and the LTE voice/SMS capabilities are likely to be 2G/3G network based for LTE networks launched in the next year.
3There are various 3G concepts in the 3GPP standards that 3GPP2 have not considered in the same fashion. Some of these different concepts can be viewed as ‘equivalent' naming conventions, for example: ‘UE' and ‘AT'; ‘up/down-link' and ‘reverse/forward link'; ‘PDP' and ‘PDN'; ‘NAS layer' and ‘session layer'; ‘RRC layer' and ‘connection layer'; ‘RRC reconfiguration' and ‘traffic channel assignment'. Such a simplifying treatment has been seen to make the introduction to 3GPP standards easier for 3GPP2 operators.
4The low priority of LTE voice/text services can be justified by the fallback capability that existing 2G/3G networks provide. This is why LTE voice standardisation is still unclear (with some operators who want to launch LTE soon keen on IMS) or why LTE texting is not a high-priority subject (although SMS is important for provisioning (OTAP) and natively supported by LTE based on similar mechanisms to GSM/UMTS). It is no coincidence that prominent industry figures do not anticipate the first LTE voice terminal before 2012.