In order for Wireless LANs to achieve the performance demanded by corporate buyers, equipment and chipset manufacturers will need better test equipment.
By Dr. Thomas Alexander, Rick Denker, and Gerard Goubert
Today's WLANs resemble the early days of cell phones, whose first users were driven by the great convenience, and whose later users demanded much more sophisticated features and reliability: the transparent roaming, compact equipment, and reasonable prices we enjoy today.
Early adopters of WLANs have also been attracted by the extreme convenience, as well as ease of installation, relative to wired network infrastructure. As a result, WLANs are experiencing tremendous growth, even in today's tough economy.
However, WLANs are being challenged to provide more sophisticated features in order to evolve into a widespread corporate technology. To understand what WLANs will need to add, it is informative to consider the earlier evolution of Ethernet into a widespread corporate technology.
Ethernet as a Model
Until performance validation test equipment emerged, Ethernet's early throughput rates varied dramatically across vendors, interoperability was not guaranteed, and the user experience was unpredictable. Hence early adopters used Ethernet primarily in engineering or research applications.
However, the first dedicated Ethernet test products were introduced in the early 1990s, and quickly proved indispensable for building demonstrably better Ethernet equipment and robust and interoperable networks. Such test equipment quickly became a requirement for all IC and equipment vendors developing Ethernet products for the corporate market. Within two years there was a drastic improvement in the throughput and interoperability of Ethernet equipment. Ethernet is now the de facto standard in enterprise networking.
Bridging the gap between early WLAN adopters and large corporate users, equipment manufacturers will need to attain higher levels of security, reliability and performance. Validating that enterprise-class equipment standards are being met will require better test equipment than exists today.
WLAN Test equipment: Challenges
Addressing and solving the key issues that keep WLANs out of business networks requires complex test equipment that can identify the problems and provide the detailed information needed to debug them. Compared to wired LANs, the wireless protocols are more sophisticated. In addition, the environment at the customer site influences the performance of wireless LANs. Also the range of product characteristics is broader than for wired products, and the protocols are also rapidly evolving as the industry is, relatively speaking, in its infancy. This is particularly challenging for test equipment, as it must keep up with (and ahead of) new standards and protocols.
There are several extra features in the WLAN protocols, addressing the nature of being wireless, that add complexity to testing requirements compared to wired protocols. The additions relate to the dynamic configuration, the spatial nature, and mobility:
Stations can query for APs (Probe Requests), and an AP can announce services supported (beacons). These services are typically present in higher-layer protocols, but WLANs need to implement them at the MAC layer.
There is also a need to determine which AP to use (association), and ensure that the user is valid (authentication). Any device within range can hear a WLAN data transmission; hence the protocol contains special features to deal with this.
Multiple BSSs can be available to a station at the same time (overlapping BSS).
Stations may both be in range of an AP, but not within range of each other (hidden node).
Optimizing power dissipation modes are required for mobile devices (power management).
Stations can dynamically switch between APs during a conversation or transmission (roaming). This is similar to the cellular concept of roaming between coverage areas of base stations and is completely absent from wired LANs.
The data rate can adapt based on the current signal strength received by the radio received to optimize performance rate adaptation). Customer site influences performance WLAN test equipment must work in a variety of customer environments and the lack of isolation from other WLANs that can affect performance.
Influence of customer site: the location of equipment and physical surroundings, though irrelevant in wired LANs, has significant impact on wireless LANs. WLANs may encounter interference issues, which are proximity dependent. And, the operation of WLAN equipment is affected by a building's structure. Commercial environments may vary from the wall-less cubicle farm to old buildings with heavy concrete walls between offices. Other field situations: home, high-and low-rise apartments, hospitals, warehouses, manufacturing centers each pose their own challenges. Equipment should be tested for the potential environments where it will be used, not just in a lab environment.
Lack of isolation: WLANs cannot be easily isolated from each other. This means that even the WLANs installed in neighboring businesses can affect performance. Equipment needs to be designed and developed to take this into account. One common source of interference is another WLAN client or access point. Packing too many clients into a confined space (e.g., a conference room) can lead to serious interference issues. This is quite unlike wired LANs, where generally switching is used to isolate traffic into easily manageable groups.
Wide Range of Product Characteristics WLAN product characteristics are very broad, and the protocols are evolving rapidly. This places requirements on test equipment to accommodate this great variation in size, power usage, security needs, etc.
Many types of equipment: Access Points, NIC Cards, VoIP phones, WLAN switches, and WLAN gateways, are among just a few of the kinds of WLAN equipment and each differs in significant ways in what is most important. For instance, portable devices like laptops, handhelds, and phones, must conserve power, so they will focus on power management. VoIP phones, in order to maintain voice quality in real-time, require tighter QoS parameters than data transmissions. Business equipment needs the support of security protocols (WEP, WPA, 802.11i, 802.1X).
Rapidly evolving protocols: The standard for 802.11 is evolving at an accelerating pace. In 1990 the IEEE established a working group on wireless LAN and in 1997 the 802.11 standard was approved. Two years later the IEEE approved two Project Authorization Requests for higher rate extensions, the 802.11a and 802.11b standards. Today there is also 802.11g and work on the even higher rate 802.11n, along with several related activities, such as wireless QoS (802.11e), and wireless security (802.11i, 802.1X).
Current WLAN Test equipment
Because of all of these challenges, from protocol to equipment to the environmental factors, validating a WLAN product is a complex challenge, one that is simply not met by current test equipment. In the next year the WLAN IC and WLAN system developers will find testing issues exacerbated by the introduction of new equipment standards, the introduction of advanced product architectures, and the emergence of Bluetooth and WiMAX (802.16) products that can cause interference. All of these issues point to the need for more advanced validation test equipment than is currently available. The limitations of existing WLAN test equipment and methodologies are numerous.
Equipment Limitations: Inflexibility: Current test methods make it hard to keep up with rapidly changing standards. Most companies need to build a new test environment with each generation. This is expensive and eliminates the ability to share tests across equipment generations.
Labor Intensive: Because tests require "real" customer use environments, tests can take a great deal of time and labor to set up. Without clear metrics and standard operating procedures, companies are re-inventing the same tests over and over. Moreover, test equipment has not been designed with test efficiency in mind.
Limited point tools: Each type of tool today is strong at a particular measurement. For instance, sniffers can handle only a single channel at a time, and have limited ability to analyze the cause of any issue detected. Protocol analyzers do not provide a method to recreate the situation that was detected and often have limited mobility. Spectrum analyzers provide only limited directional information and do not work at higher protocol levels.
Limits to Current Test Methods
Very ad-hoc, limited: Tests are typically designed on an ad-hoc basis. Few, if any, test procedures have been defined or agreed upon across vendors. A notable exception to this has been the efforts of the UNH-InterOperabiliity Lab.
Variability between tests: Test setups are built by engineers using standard PCs and software with little account for the variation between hardware and software. For instance, there are variations in the driver software depending on which NIC card is used. If there's a driver upgrade, the entire setup must be recalibrated. This adds to the uncertainty of the test environment. There is a need for tools, which reduce the variability from test to test. This is particularly true in WLANs, where the RF devices used exhibit far more variability and sensitivity to environmental conditions than typical wired LAN devices such as Ethernet.
Inaccurate: No current test equipment can handle timing precisely enough. Timing strongly influences the repeatability of testing. Sniffers are not exact enough to provide the level of accuracy required for repeatability. RF instruments give very accurate timing, but gather too much data to look at more than a few frames.
Unable to scale: Scaling to larger test configurations is expensive and hard to manage. Test engineers who want to look at a scenario with 50 client stations are faced with the daunting task of having to configure for every test. Many test engineers spend a third or more of their time just setting up for testing.
Not repeatable: To get true repeatability, testers need to control both the RF environment and timing accuracy, which is currently very difficult or impossible to do. Current WLAN protocol analyzers still follow the traditional model: a single wireless source, limited traffic generation, and single-channel testing. They provide some analysis of received traffic, but are not be able to reproduce test conditions.
WLAN Testing Must Accommodate Unilaterally
To fully understand what 802.11 testers need to do, one must understand the range of test environments: Faraday cages, RF chambers, cabled, and open air, including their strengths and weaknesses. "Test environment" refers to the setup or environment into which the device(s) being tested and the test equipment are placed. For instance, DUT and tester may be completely enclosed in a single large shielded box, or the DUT and the tester may be placed at selected locations within an actual building. See Figure 1 for a comparison of test environments.
Faraday cages are usually large, hand-constructed, copper mesh wrapped boxes or rooms. Because of the expense of their construction, they are typically found in the labs of large equipment manufacturers, where they are shared for testing and quality assurance. Because Faraday cages assure a fairly noise- and interference-free environment, they are good for a wide variety of individual product tests, especially for antennas. However, test configurations of more than a few devices can quickly congest traffic in a cage. In addition, it is not possible to test effects such as multi-path and diversity in a Faraday cage.
RF Chambers are metal boxes with absorbing material lining the inside to dampen interference. They provide a controlled environment for much lower cost than a Faraday cage. Typically, the DUT is placed into the test chamber, and probes are used to couple signals to/from the DUT through cables to an external test system. In some cases the DUT and the test equipment are placed within the same test chamber, at which point this approximates a Faraday cage. At some point, it ceases to be practical to use chambers as opposed to a Faraday cage. Moreover, because spatial information is lost, some equipment cannot be tested in a chamber, e.g., smart antennas.
Multiple sizes of chambers are required for proper testing in a fully enclosed environment. The lower limit on the size of the chamber is dictated by the distance (referred to as the Fraunhofer distance) at which the RF near-field transitions to the RF far-field. Objects including the walls of the chamber itself that are placed closer than this distance to the unit under test have a significant impact on the radiation pattern and efficiency of the unit; hence it is necessary to ensure that the chamber dimensions are greater than this distance, otherwise the test results may prove to be either irreproducible or erroneous.
The Fraunhofer distance is a function of the RF wavelength used, as well as the physical dimensions of the device under test. For example, the minimum required dimensions of a suitable test chamber for a typical 802.11b/g access point could range up to 1 meter on a side or more, while those for a chamber capable of properly testing a laptop with a 2.4 GHz WLAN card could be very large as much as four meters on a side, or equivalent to a small room.
Cabled tests simply substitute a wired connection for the wireless connection, bypassing the antennas and directly connecting two pieces of equipment. As a result, cabled tests are inexpensive and easy to configure, and provide good isolation from interference. They are not limited to small configurations, like cages and chambers. However, because of the lack of interference, their results in configurations are idealized toward better performance than would actually occur in the randomness of an open air environment. In addition, properly performing cabled testing relies on the DUT itself being well-shielded, which may not always be the case in consumer or low-end enterprise equipment. In addition, equipment with integral antennas (where the antenna cannot be disconnected to gain access to a connector or other attachment point for a cable) cannot be tested using this method.
Open air is the only test environment that truly matches the way the customer will use the equipment. Like cabled environments, open air has no size limitations or limits on the number of pieces of equipment in a configuration. For some tests, it is ideal because it can test both the antenna and the protocol effects. Also it is the only solution for certain location-dependent tests.
Open air test environments can be separated into indoor and outdoor. Indoor environments are normally actual buildings, usually with furnishings and other accouterments characteristic of typical office buildings. Outdoor environments are usually open spaces without obstructions, such as would be found at an antenna range. Of these two, the indoor environment is of the most interest, as it most closely approximates the conditions under which the equipment is expected to function. Outdoor environments are used for applications such as characterizing antenna patterns, setting baselines for range and rate, etc.
Summary of Test Environments
Complete testing requires a combination of test environments; a one-size-fits-all environment does not exist for wireless testing. Situations will arise where a test developer will need to choose one environment over another. For example, antenna tests may require the full accuracy of a Faraday cage. And for protocol testing at Level 2 and above, because these levels are not affected by the antenna, they can be tested by cable, chamber or cage all roughly equivalently, and the test developer can choose whichever is most convenient. Ideally, testing equipment should be able to accommodate all environments.
Note that some approaches may mix two of the above environments to gain the advantages of both. For instance, a test vendor may choose to combine a cabled setup with a shielded test chamber, to offset the issues caused by poorly shielded DUTs in a cabled configuration. There is no single environment, however, that is considered "optimal" for WLAN testing.
Due to the impact of RF effects upon the MAC-layer protocol, and vice versa, users must make tradeoffs between repeatability, accuracy, cost, convenience and end-user performance, when selecting a suitable test methodology. Test equipment vendors therefore need to accommodate all the different types of test environments, and cannot confine themselves to a single type.
Requirements for Better Test Equipment
When considering new WLAN test equipment there are several questions that should be evaluated, including:
Is it cost-effective?
What are the costs per engineer?
What tests can the equipment be applied to?
What productivity improvements are possible?
What processes will be more effective?
How has communication between groups been improved?
Over time how much of test development can be reused in the next project?
Can it keep up with changes in protocol standards?
Will the technology be able to keep up with the fast changing standards?
Is there a lag time for updating software drivers or recalibration of test results?
Is it simple and easy to use?
Is the equipment easy to set-up and use?
Does it require special expertise to run?
Does it require extensive training to be effective?
Is using the equipment error prone, or forgiving?
Technical Requirements for Test Equipment
The requirements placed on wireless LAN test equipment are far more rigorous and stringent than those to which standard consumer and enterprise WLAN devices must conform. Test equipment is required to provide answers to problems encountered while diagnosing malfunctioning devices or systems, as well as to drive such devices to their limits while still accurately measuring their capabilities and responses.
WLAN test equipment must hence provide far more comprehensive and detailed traffic generation and analysis capabilities than can be achieved by simply utilizing off-the-shelf hardware with load generation software. Accurate testing of WLAN devices and systems mandates complex and purpose-built hardware that has been designed "from the ground up" specifically for WLAN scenarios.
The requirements for test equipment are many and vary from the detailed technical nature of timing specifications, to the organizational efficiency issue of sharing tests. A summary of these requirements follows:
Provide All Information Seen on the Air WLAN test equipment must accurately capturing and present all information that is sent to or emitted by a device under test. This includes not just error-free frames, but also frames with errors, partial frames, and interfering signals. Without such information, it is frequently impossible to determine exactly why a device is malfunctioning, or why a particular sequence of transactions took place.
Be Repeatable Repeatability here implies that a specific test, if run over and over again under a constant set of environmental conditions, will yield the same outcome every time. This is extremely difficult to achieve unless great care is taken, while designing the test equipment, to ensure that it can generate test stimuli in a highly controllable manner. For WLAN testing, repeatability necessitates precise control of not only factors such as RF signal strength and attenuation, but also packet timing and protocol handshakes. To achieve full repeatability, the test equipment must be capable of controlling and measuring signals at accuracy substantially better than one symbol time.
Accurate Measurement of Timing and Signal Strength
Accuracy of recording is essential in order to diagnose obscure faults or measure the responses of devices under test at extremes of the protocol. As in the case of repeatability, accuracy must be achieved at both the RF signal strength level (i.e., transmitted and received power and signal characteristics) as well as the bit timing level.
Share Tests and Results Across Distances
In today's multi-site development environment, where teams are spread over different locations, the ability to share information between sites and between departments is critical. For instance, validation engineers must communicate problems found during qualification testing to firmware development engineers in a simple and efficient manner, and manufacturing engineers need to send characterization and QA data back to development engineers in order to improve manufacturing processes and raise product quality. Test equipment is essential to this kind of interdisciplinary communication. WLAN test equipment, therefore, must support the effective sharing of tests and test results between teams, and enable tests to be replicated accurately at different physical locations.
Handle All Testing Environments As part of the need to support a broad-based, interdisciplinary approach to WLAN testing, the test equipment should also lend itself to being used in all types of test environments, including Faraday cages, RF chambers, cabled, and open air. Fully shielded testing (cages, chambers, and cabled) is useful for cases with a high density of users; open-air environments are essential for QA departments that need to completely characterize the behavior of the products being developed under "real-world" conditions. It is also extremely desirable that results obtained under one type of environment be easily portable to another type of environment, preferably even using exactly the same test equipment for both. This not only simplifies the company's validation and testing strategy, but also considerably speeds up test development.
Handle Both Lab Bench and Customer Site
Because certain issues will only be manifested at the customer site, companies must be able to extend the reach of their test equipment from the laboratory to the customer site, so that problems that are only visible at specific customer sites can be diagnosed quickly and easily, and be brought back to the lab for replication and detailed analysis.
Be Spatially Aware
An attribute required by WLAN test equipment that is unnecessary for wired LAN test equipment is spatial awareness. WLANs are inherently spatially aware, in that their behavior is considerably influenced by their position within the physical environment and their location with respect to other devices. WLAN protocols take this into account and hence add to the spatial nature of WLAN systems. To fully test a WLAN device or system, therefore, test equipment must take these spatial characteristics into account.
As an example, consider the common WLAN issue of roaming. Roaming occurs when a WLAN client physically moves from the neighborhood of one access point to another; as the signal strength received by one access point decreases and that received by the other increases, the client de-associates with the first access point and re-associates with the second new access point in order to maintain its network connectivity. This is obviously not a feature of wired networks.
Testing roaming, however, requires that the test equipment be capable of emulating the characteristics of the environment as well as simulating the effect of moving a client from one location to another. In particular, precise and repeatable testing of roaming necessitates highly accurate control of signal characteristics and timing. Essentially, the test equipment must cause a "virtual" movement in space in a highly controlled manner while generating traffic in an accurate and repeatable manner.
Other instances of situations requiring spatial awareness are common WLAN-specific effects such as hidden nodes, the "near/far" problem, coherent and non-coherent interference, diversity antenna switching, etc. The WLAN protocol attempts to compensate for many of these issues with specific functions and capabilities. Complete testing of a WLAN device, therefore, mandates that the traffic patterns corresponding to these effects be precisely generated.
For instance, fully testing diversity antennas requires that the test equipment be set up in some sort of real or simulated multi-path environment, and suitable traffic patterns generated to exercise the device features supporting diversity reception. In many cases (e.g., integrated diversity antennas), complete testing of these features can only be carried out in an open-air environment, with the test equipment placed at fixed and known locations within this environment.
Handle New Switching Architectures
The new generation of WLAN switches being introduced into the marketplace imposes special requirements on test equipment. WLAN switching is fundamentally a spatial phenomenon; for instance, one version of switching uses electronically steered beams that are focused on specific WLAN clients, and follow these clients as they move about.
Clearly, testing WLAN switches is fundamentally different from testing wired LAN switching technologies. Unless the test equipment is capable of spatial awareness and can generate and receive traffic in a spatially distributed fashion; any attempt to test such equipment will be superficial and incomplete. As more companies bring various types of WLAN switching technologies to market, therefore, the requirements placed on test equipment will rise correspondingly.
Handle Combinations of Layer 1 and Layer 2
Finally, WLAN test equipment must be capable of straddling the Layer 1/ Layer 2 boundary. This implies that the test equipment must be able to address problems that occur as an interaction between the physical layer protocol and the MAC (or higher) layer protocols. For instance, consider the issue of testing an adaptive summing diversity receiver being used for a WLAN access point. Such a receiver may make use of information at the bitstream or even the packet layer to adaptively adjust a set of weights within the PHY; however, the combination of firmware, packet-level protocols and RF effects makes this a difficult problem to address with physical layer test equipment alone. Instead, it is necessary to utilize test equipment that is capable of simultaneously handling the needs of testing the device at the bit level, the packet level, the MAC level and even the driver level. Without such capabilities, the user may be reduced to solving problems by guesswork, or may have to resort to expensive ad-hoc test setups.
The issue of combined Layer 1 and Layer 2 testing is required by several critical areas of WLAN protocols and technology. For instance, roaming is a Layer 2 function that is triggered by a Layer 1 effect (a reduction in signal strength), that may have consequences even at the Layer 3 level depending on the security protocol in use.
A similar problem manifests itself in Voice over IP (VoIP) technologies, but along a different dimension; VoIP testing requires accurate measurement of timing variations in order to estimate the worst-case latency of a voice channel, but these timing variations may be caused by Layer 1 effects such as interference bursts causing retransmissions and back-off. It is hence necessary for the test equipment to provide the user with both visibility and control of the data stream at multiple levels, and not focus merely on a specific protocol layer while sacrificing controllability or observability at other layers.
What The Future Holds
A new generation of testing tools, architected for wireless, is starting to become available. Such tools will be critical to WLANs addressing widespread corporate applications, and for handling the more complex requirements of future WLAN technologies.
There are also several new technologies that are on the near horizon and the testing needs for these should be considered in a test equipment purchase, including smart antennas, higher bandwidth standards, new features, and advanced switching architectures.
About the Authors
Dr. Thomas Alexander is the Chief Technology Officer of VeriWave, Inc.. He has extensive experience in the networking and computation industries. Prior to VeriWave, he was the chief architect for Ethernet products for PMC-Sierra. He is active in communications standards being the editor for the IEEE 802.3ae 10G Ethernet standard, and the Chief Editor, IEEE 802.17. He has invented widely-applied Ethernet products and has a history of founding and guiding successful companies.
Rick Denke is the Vice President of Marketing for VeriWave, Inc. Rick has broad experience in high technology product marketing and business development. Prior to VeriWave, he was the Director of Product Marketing for WeSync.com, which was acquired by Palm. He has held senior marketing positions at Synopsys, and PMC-Sierra.
Gerard Goubert is the Manager of the Wireless consortium, UNH-InterOperability Lab. Gerard has many years of experience in a variety of networking technologies. He currently manages the Bridge Functions, Voice over X, and Wireless consortiums for the University of New Hampshire InterOperability Lab (UNH-IOL). UNH-IOL is the leading independent test lab for interoperability and conformance testing.Glossary of Acronyms
AP - Access Point
BSS - Basic Service Sets
DUT - Device Under Test
IC - Integrated Circuit
MAC - Media Access Layer
NIC - Network Interface Card
QoS - Quality of Service
WEP- Wired Equivalent Privacy
WLAN - Wireless Local Area Network
WPA - WiFi Protected Access
VoIP - Voice Over Internet Protocol