Field-to-Lab Testing Equals Better Real-World Performance for Mobile Devices
This case study will describe an application of the Azimuth Systems Field-to-Lab (FTL) Solution by a market-leading handset manufacturer.
Real-world drive testing is very expensive; lab testing is much less expensive and faster -- provided that the real world can be emulated with sufficient fidelity. The Azimuth Field-to-Lab solution allows service providers, equipment manufacturers, and chipset vendors to take real-world drive test logs collected from around the globe, and replay the data in the ACE MX MIMO channel emulator in order to effectively recreate actual field conditions in any lab, during any phase of the equipment design and qualification cycle for equipment benchmarking, troubleshooting or many other purposes.
Standard channel models are often used to recreate channel conditions in the laboratory environment to perform standards-based testing. Although these statistical representations of channel conditions are very useful in equipment test, certification and benchmarking, they do not adequately capture the unique and specific conditions experienced by a mobile device as it moves through the real world. The Azimuth Field-to-Lab solution was created to easily integrate actual drive test scan logs into the testbeds used in carrier and equipment vendors’ labs to more effectively test devices against field data collected over “golden drive test routes” or at key locations of interest in the actual network, where performance is critical.
Testing with the Azimuth Field-to-Lab solution involves three phases:
* Collecting field data
* Conditioning the data for playback
* Playback of the field data on the Azimuth ACE MX MIMO Channel Emulator
The key components of the Azimuth solution are the AzMapper, AzPlayer and ACE MX Channel Emulator. To collect field data, the Azimuth Field-to-Lab solution is integrated with both industry-standard and custom drive testing software to allow the re-use of data from drive tests. Drive test data is then converted into a playback file for a given number of base stations and can be filtered, smoothed and filled to ensure realistic playback conditions. Next, the data is played back on the channel emulator; a playback control tool configures the ACE MX MIMO channel emulator and streams the playback conditions to the signal path of the channel emulator between the mobile device (e.g. UE) and the base station (e.g. eNodeB).
The use case that will be illustrated in this case study primarily deals with the conditioning of the data to stress important test scenarios. This can be done by simply degrading the signal strength to effectively move the drive test scenario to a cell edge. A second option involves accentuating the signal peaks and valleys. In this manner, challenging environments such as a crossing cell scenario where a device transitions from one base station to another at an area of reduced signal strength could be accentuated (and a deep fade simulated between the two cells) or a rapidly peaking sector, sometimes known as a “rising pilot”, can be accentuated to stress the scenario where the mobile device briefly sees a strong pilot before the pilot fades into the background.
Field-to-Lab was first deployed by a leading handset manufacturer to reduce software development time, as well as impact time to market and product quality.
As the Field-to-Lab methodology was integrated into the handset manufacturers’ 3G test program, the company found that it could test a “crossing cell” scenario (as observed during drive test) and play it back in the lab to pre-qualify products against a particularly challenging field scenario. Testing against a known scenario was a good start, however, the test team used Azimuth’s FTL tools to further exaggerate that scenario. The result of that effort was that the handset manufacturer could now not only test products against the “crossing cell” scenario but could also confirm that the design margin was sufficient to handle a range of field conditions. Software changes necessary for additional performance margin were implemented and then regression tested against multiple scenarios using the end to end testbed management capability of Azimuth’s TestBuilder.
As the handset manufacturer began to utilize Field-to-Lab testing, additional problems were identified where the Field-to-Lab methodology could provide further impact. In particular, the manufacturer was able to use Field-to-Lab to investigate a particular dropped call scenario in a carrier customer’s 3G network.
The carrier had collected data that showed a higher rate of dropped calls in one urban environment. The handset vendor postulated that the pilot from adjacent cells might appear long enough to trigger a handoff but would then disappear, ultimately resulting in a dropped call. This “rising pilot” was apparent in the drive test logs (Figure 2) but prior to using Field-to-Lab testing, there wasn’t a way to easily replicate it in the lab to perform more detailed analysis (Figure 3). After mapping a rising pilot captured in the drive test logs with AzMapper, the handset vendor was able to rapidly test their devices against the drive test profile and identify the specific situations that led to a dropped call. Testers were then able to exaggerate the timing and magnitude of the rising pilot in order to establish the required device performance margins necessary to avoid a higher dropped call rate -- providing valuable feedback to network planners and chipset suppliers (Figure 4).
Both the handset manufacturer and carrier were extremely pleased with the results from Azimuth’s Field- to-Lab test solution as the improved ability to rapidly test many devices against multiple drive test profiles saved them a significant amount of time and money. As an added benefit, Field-to-Lab testing is performed in the privacy and security of the company’s own labs which provides the enhanced security needed when designing and developing groundbreaking new mobile devices as oppose to the public nature of drive testing.
Value to the customer
Testing mobile devices involves many phases, including the validation of the devices in the actual field conditions where subscribers will use them. However, field testing in the diverse conditions of terrain, population density, physical location and motion of devices is extremely time-consuming and costly. As the wireless industry works to introduce advanced new 4G products and solutions, field testing will be particularly important as the MIMO technology used to enhance system performance and achieve the high data rates expected will cause product performance to vary substantially depending on the RF environment. The Azimuth Systems Field-to-Lab solution is accelerating performance testing, benchmarking, and troubleshooting of mobile devices against real world conditions, while avoiding the substantial cost and time required for real world drive testing.
The value of a Field-to-Lab test methodology is not just in the accurate mapping of current drive test profiles to the field, but also in its ability to experiment using measured profiles as a baseline and in its ability to provide a common platform for sharing test environments across the wireless ecosystem. Leading operators and device vendors worldwide are currently benefiting from these powerful capabilities to perform more accurate and efficient testing while enhancing product quality and substantially reducing time to market.