Open Switch Fabric Technology for Wireless Communication Equipment
A building block that must be put in place to support the advent of multimedia high speed Internet access and VoIP capabilities has specific requirements.By Tim Miller, StarGen, Inc.
The infrastructure of the worldwide wireless network is in flux. The 3G wireless promise of media rich wireless voice, video and data services delivered seamlessly to the mobile subscriber is compelling. Yet the underpinnings of this network present fundamentally different requirements than the voice oriented circuit switched 1G/2G networks or even the data service overlay networks of 2.5G. Much of the discussion today around 3G infrastructure is focused on protocol and processing requirements in the move toward IP based data services and preparing for Voice over IP (VoIP) services in the future. Less of a focus, but the subject of this article, is a required building block that must be put in place to support the advent of multimedia high speed Internet access and VoIP capabilities. That is an open, cost effective, high capacity data highway within the infrastructure equipment itself. These highways provide the communication link between the various processing and I/O elements within wireless network communication nodes that are required to process and manage the wireless communication streams from the mobile subscriber devices to the wireline Internet or PSTN infrastructure. This system level plumbing is fundamental in being able to build out 3G networks in terms of scale, numbers of users, total bandwidth, reliability, differentiation of services, and cost effectiveness. This article will explore the requirements of this plumbing and describe the solution provided by the StarFabric interconnect technology.
Figure 1 illustrates the major components of the 3G network from base stations to the wireline Internet and PSTN backbones including, radio network controllers, SGSNs (service GPRS serving nodes), GGSNs (gateway GPRS service nodes), and media, signaling, and billing gateways. Each provides its unique function in the process of delivering mobile services to the subscriber from mobility management, session control, data transcoding, switching, routing, policing, traffic management, and services tracking and billing. In the 3G world, with potentially more than a billion subscribers and data rates of multi-megabits per second, this equipment must be highly scaleable, highly reliable and able to provide a wide range of differentiated levels of service. In addition, these services must be easily provisioned to meet the extensive needs of users and enable service providers to track and charge for the services provided.
The architecture of these systems has some fundamental similarities. Figure 2 provides a high level architectural diagram. Systems are made up of a collection of processing elements used for system control and data stream processing. These processing elements, depending on function, are standard general-purpose microprocessors, off-the-shelf network or packet processors, DSPs, voice processors, or fixed function ASICs. They provide system control, packet processing, traffic management, protocol translation, and data processing. Additionally, with VoIP and the move of voice applications from the legacy circuit switched infrastructure to the data services networks, utilization of DSPs is typical for functions like voice coding, compression, echo cancellation, etc. Signal processing also provides spread spectrum, interleaving and error control functionality. Additionally, I/O capabilities like storage and network connections are present. Control and data stream traffic flowing through these systems move from processing element to processing element and access IO resources via the system level interconnect. The system interconnect provides connectivity inside a system chassis and can scale up to a rack of equipment typically within a Central Office, POP, or enterprise network center. Today, many of the systems in 2G and 2.5G networks are implemented using bus-based technology as the system interconnect.
In today's architectures, multiple interconnect planes typically exist in a system with each optimized for a specialized purpose. A control plane carries system control and signaling traffic separate from the data transport plane. The characteristics of the data transport plane are optimized for the type of payload traffic carried. With 2.5G the data services overlay network is tuned for asynchronous data traffic, which is typically bursty and does not have real-time delivery requirements. This type of traffic is very different from voice traffic, which has lower bandwidth requirements but strict deterministic real-time delivery requirements. Video traffic has high bandwidth requirements as well as real-time requirements. Many typical video applications will also have a multicast requirement. With 3G and the convergence of multimedia applications, real time traffic must coexist with asynchronous traffic.
For most wireless communication equipment today, the interconnect architecture is proprietary or leverages widely available bus standards like PCI, H.110 or ATM. Proprietary approaches have generally been utilized to address performance requirements of high-end solutions. Communication equipment vendors have used in-house ASIC resources to develop custom semiconductors to implement these solutions. Because the solutions are unique, they cannot achieve the volume scales to drive cost down. This results in expensive solutions only suitable for the high end of the marketplace.
Standards-based solutions benefit from compatibility across vendor platforms and can achieve low cost through high volume. With PCI, a wide spectrum of components are available to communication equipment designers as low cost system building blocks. Direct PCI interfaces (or through standard interface chips) are available for most network controllers, DSPs, and processors. PCI is well understood and well supported in the hardware and software space. In traditional TDM circuit switched systems, a standard like H.100 provides the same benefits. Downsides of using these standards for designers are the inherent limitations they have in terms of scalability and robust feature sets making them unsuitable to support the increasing demands of 3G equipment. Bus based architectures are not up to the requirements of 3G network infrastructure equipment.
A new open interconnect standard needs to be developed to meet the demanding performance, functionality and reliability of 3G communication equipment requirements. It must also have the cost and ease of design benefits of a widely adopted open standard and provide an elegant migration path from the existing wireless equipment interconnect standards.
There are several top-level requirements for this interconnect that are driven by the common needs across the wireless communication market.
1. Compatibility with existing standards
2. Quality of Service functionality
4. High Availability
5. Multi-vendor support
6. Low cost
The fundamental architecture required to meet these requirements is switch fabric technology. Switch fabrics have many benefits over bus-based interconnects. Critical among these are scalability and reliability. Unlike the shared medium of a bus architecture, switch fabrics are point-to-point. Each end-point is connected to every other end-point through one or a series of switches. End-points can be considered 'bridges' to existing standard buses or components. In a switch fabric, many devices can be transmitting and receiving simultaneously. Through the building of systems with series of end-points and switches, a diverse and flexible universe of system typologies can be created. As more connections are added to the system, the total bandwidth of the system increases. Figure 3 illustrates a replacement of the bus-based backplane architecture shown in Figure 2 with a switch fabric interconnect. The central switching capability joins all of the processing and I/O elements in a point-to-point fashion.
A switch-based design allows simple scaling of connections and bandwidth. Additionally, it has flexible routing capabilities. Topologies that have multiple routes between the same two end-points can be created. If one route fails or becomes unavailable, traffic can be redirected onto the alternative route. With point-to-point connections, a single end-point failure does not impact the rest of the system, as is the case with a bus model where a bad device can bring down the entire bus. With switch fabrics, the point-to-point connection is friendly to device insertion and removal.
The point-to-point nature of switched interconnect is well suited to serial physical layer technology. The transmission line characteristics of serial technology are superior to bus technologies. To gain bandwidth, buses increase their parallel nature. This limits frequency and introduces tight tolerance requirements that are unfriendly to off board transmission. Serial physical technology can allow much longer distances between end-points and is measured in meters rather than inches, depending on the cabling characteristics. The physical layer technologies that embed clocks with data have the added advantage of eliminating the skew control required when clocks travel separate from the data.
The StarFabric technology provides the general benefits of switch fabric while also providing a simple migration path from existing open platform architectures. It offers 100% backward compatibility with PCI, allowing the use of existing device drivers, BIOS and operating system support. It provides this functionality in an easily adoptable way by not requiring exotic system design in terms of power or signal integrity, and by allowing use of standard cabling and connector technology. Its cost structure is also in-line with traditional bridging technology. StarFabric provides the performance, scalability, reliability, and flexibility needed for 3G equipment interconnect while delivering the cost effectiveness, and ease of design of open technology. Table 1 provides a summary of the features of StarFabric.
The initial silicon components leveraging StarGen's StarFabric technology include a high throughput switch, providing 30 Gbps-switching capacity with six ports. Bridge chips provided by StarGen and others will provide access from existing standard interconnects to the advanced functionality of the switch fabric. These devices offer manufacturers a new option for building high-speed, scalable and highly reliable systems. In addition to the 30 Gbps switch, StarGen is developing a bridge chip to PCI that translates between two 5 Gbps fabric ports and a 64 bit/66 MHz 5 V-tolerant PCI bus. The products provide sufficient SRAM on-chip to support flow through operation. These components are illustrated in Figure 4.
Agere Systems is currently designing a bridge chip that translates two 5 Gbps fabric ports to the H.110 TDM bus. Discussions and planning are underway with partners for additional bridges to buses including ATM, Utopia, network processor buses, GigaBit Ethernet and DSPs.
The switch fabric utilizes point-to-point LVDS connections. This technology is ideally suited for chip-to-chip, through backplane connectors and chassis-to-chassis interconnect up to 5 meters with standard PCB construction, 2 mm existing connectors and CAT5 twisted-pair cable.
A solution can have as few as 2 ports up to >200 with initial components. 3G networks will need to be able to support large numbers of subscribers and service providers will be able to scale there equipment up as the number of subscribers increases. They will not be forced to over invest initially as they can incrementally add capacity to their equipment as demand increases. As processing requirements change, line cards can be upgraded without impacting the rest of the system or requiring forklift upgrades of the entire system. Future components will enable solutions with >1000 ports.
Each StarGen port consists of four 622 Mbps LVDS pairs in each direction to provide an aggregate bandwidth of 5 Gbps. The 622 Mbps physical layer was chosen to balance bandwidth, power, number of pins, cost and ease of design. Standard FR4 PCB technology can be used with several feet of etch controlled at 50 ohms. For backplane interconnect, proper signaling has been simulated with several standard C-PCI 2 mm hard metric connectors and 50 ohm etch at distances over 3 feet. Chassis-to-chassis interconnect is accomplished with RJ45 connectors and CAT5 twisted-pair wire. Cable simulations up to 5 meters have been successfully completed. Distances greater than this are possible with higher quality cable. Also, the technology is well suited for interface to other physical layers, like fiber, to accomplish even greater distances. Future StarGen components will integrate higher bandwidth physical layers.
StarGen's switch fabric architecture is intended to support seven traffic classes, including asynchronous classes, isochronous classes, multicast, and high-priority. For 3G networks, this will allow the efficient support of data services and VoIP services simultaneously using the same infrastructure. VoIP services can be added to the equipment incrementally to meet business opportunities. StarGen allows the unification of traditionally separate interconnects for control traffic and data payload traffic. This simplifies design and lowers cost.
The fabric uses variable packet size method that a switch uses to process data depends on the class of service and routing method used for the specific frame. Each port on a switch can simultaneously source and sink data on every clock tick. 8B/10B encode/decode and CRC generation and checking are performed on each frame. With the exception of 8B/10B, CRC and some header processing, data is not processed in switches. It is efficiently routed from an incoming port to the appropriate outgoing port. The fabric features CRC-16 protection at each hop. Switches are self-routing with a fully integrated scheduler. With all overhead including 8B/10B and protocol, the fabric is ~70% efficient for a net full-duplex bandwidth capability of ~3.5 Gbps per port.
For 3G networks, reliability and high availability are critical requirements. The switch fabric provides several features that enable high-availability solutions including redundancy, fabric hot swap, C-PCI hot swap and fault detection/isolation. Implementers can create parallel fabrics for full redundant interconnect of switches and links or they can provide N+1 redundancy with fewer components. Redundant paths between endpoints in the fabric can be pre-configured so that switchover to an alternate path can be done automatically in silicon when a primary path fails. The time for this switchover is in the several microsecond range. Solutions can be optimized based on the type of traffic class. For example, isochronous traffic classes with real-time requirements would typically provide for ASAP switchover to a redundant path without concern for loss of a single packet. Asynchronous traffic classes typically have higher data integrity requirements. For this traffic type, switchover time may be longer to enable recovery and re-send of data.
The fabric links are friendly to device insertion and removal. The physical interface can be short-circuited or open-circuited without damaging the device or adversely affecting the fabric. The PCI-to-serial bridge implements Compact-PCI Hot Swap for card insertion and removal in a functioning system.
StarGen's switch fabric is lossless in the absence of component failure and complete link outage. The fabric can tolerate the loss of a link within a bundled 10Gbps pair and all but one 622Mbps differential pair within a link (normally, four differential pairs operate within a link.) The fabric automatically detects the loss of one or more differential pairs within a link. Traffic will continue to flow over the remaining differential pairs and notification of the 'fragile' link can be directed to a controlling node. The fabric is a lossless mezzanine-level interface with no tail drop. Retransmission is not required unless a CRC transmission error occurs.
Open switch fabric technology can provide the logical evolutionary path from existing bus-based architectures for wireless communication equipment. To be successful, the switch fabric technology must provide backwards compatibility with existing bus-based open standards and provide an elegant migration path so that system designers can adopt the new technology at the pace that makes sense for their business. In addition to providing the scaling and performance requirements of 3G networks, the interconnect must inherently support Quality of Service so that it can effectively handle all classes of traffic simultaneously, while meeting each one's unique delivery requirements. The interconnect must be cost effective, on the same order of magnitude of existing standard interconnect solutions, both at the silicon level as well as at the system level. It must not require exotic or expensive packaging, connectors or cabling. Finally, it must be open and widely available so that multiple vendors can supply devices, thus insuring a wide and ever-evolving set of compatible products and a ready and diverse supply base. StarFabric, developed by StarGen, but now available as an open technology, is an example of a technology that meets the system interconnect needs of the new 3G network infrastructure.
Tim Miller is the vice president of marketing for StarGen, Inc., a fabless semiconductor company developing new switch fabric technology for the communications industry. He can be reached by phone at (508) 786-9950 or by email at firstname.lastname@example.org.