PCI_Bus_Overview

PCI BUS OVERVIEW (http://www.adaptec.com/technology/whitepapers/pcibus.html#background) INTRODUCTION The PCI (Peripheral Component Interconnect) architecture is without doubt the best broad-application computing platform available to the public for meeting today's PC computing needs. Legacy architectures such as ISA (Industry Standard Architecture), EISA (Extension to Industry Standard Architecture) or MCA (Micro-Channel Architecture) have served their purpose but are no longer viable platforms for running the demanding applications that are now being produced. The ISA, EISA and MCA architectures are simply less efficient than PCI in comparison, especially in operations such as I/O and DMA (direct memory access). These operations are key areas, and are two of the primary factors affecting overall system efficiency. The limitations that are were inadvertently built in to legacy architectures have now been fully relieved by the industry's acceptance of PCI. Aside from resolving issues encountered with legac y systems, PCI offers the industry an open standards interface for device/application development. PCI is an open specification, all host systems that are compliant with the PCI specification are capable of hosting any PCI compliant peripheral device. The same is true for the I/O level core of some software. This broad compatibility has lead the PC industry to attain an overall system efficiency that is unrivaled by any other system architecture. HISTORIC BUS AND LOCAL BUS DESIGN The ISA and EISA architectures were in use until only recently. They were the most popular for use in PCs and even provided a base architecture which became the PC server. The two architectures differ in many ways, however they share a common structure relative to how CPU and memory are accessed. The block diagram below shows how ISA and EISA systems communicate with devices on the expansion bus. Note that there is a CPU local bus, an expansion bus, and a memory bus. Devices that are inserted into the system's peripheral component bus must communicate via an expansion bridge, memory bus, bus cache, and the CPU local bus before finally communicating with the CPU. The latencies inherent to a design of this nature severely restrict the system's ability to perform work efficiently. Consider the following situation: As a request for processing passes to the processor from an expansion device, each link that the request passes along the way causes a brief delay. As traffic on the expansion bus increases, and requests to the CPU increase, each link becomes it's own bottleneck, and serves to restrict traffic flow even more. In systems that experience very high levels of traffic on their host bus, the latency involved with getting I/O and data requests processed can become so pronounced that individual devices on the expansion bus wind up timing out while waiting for their traffic to be processed. This can cause problems with intensive I/O devices such as network adapters or SCSI controllers. If the communication latency between a peripheral device and the CPU is too great, expansion devices can actually loose or corrupt data. The expansion bus is so far removed from the CPU and memory on legacy bus designs that high traffic conditions can barely tolerable for peripheral devices. It makes excellent sense to attempt to place expansion devices on the CPU bus directly in order to reduce the latencies inherent in the ISA and EISA bus architectures. One successful attempt was the "VESA A" local bus (Video Electronics Standards Association rev. A). The first VESA devices were video controllers, and they were placed on a VESA A local bus which was connected directly to the host CPU bus. There is an inherent problem with connecting devices directly to the CPU local bus however. The interface between the CPU bus and the VESA A bus must be CPU dependant - the controller that communicates between the CPU and the host bus speaks only to the particular family of CPUs it was designed for (486, 386, etc.). This means that if you upgrade the CPU, you need to upgrade the VESA A bus interface to the CPU bus. In most cases CPU upgrades meant changing system boards or designing bus/CPU interface products that were capable using of a number of different formats to access the CPU. In addition to this issue, VESA A architecture proved to run slower and slower as loads on the system increased. This is due to VESA expansion devices being CPU intensive by design, they communicate directly with CPU, they are right on the CPU bus. VESA A was a good idea and an excellent effort but it was not flexible enough to survive, and it was very inefficient with regards to CPU usage. Another way to resolve the bus latency issues is by creating a virtual local bus and connecting it to the host CPU bus via a buffer. VESA A simply attached a peripheral device bus to the CPU local bus. The introduction of an intermediate buffer between the CPU bus and VESA bus in the VESA A architecture is known as VESA B design. The buffer between the VESA and CPU busses re-drives all local bus signals to the entire VESA local bus. It also re-drives the signals to the CPU local bus. This means all devices on the bus can know when communication is occurring, and avoids some arbitration issues and collisions. The buffers allow signals to be stored for brief periods if the target bus should be busy. The buffering idea works fairly well and even took care of some performance issues associated with the VESA A. This bus design introduced new issues however, such as what to do about device arbitration. What happens when more than one device needs to use the CPU local bus at the same time? The answer turns out to be fairly time consuming, one device needs to request the bus and then wait until the device actually using the bus surrenders it. Devices involved in this arbitration include the CPU as well as expansion devices. Some latency is necessarily designed into the system by using buffers and/or bridges. If the traffic on the bus causes the latency to become too large however, it can cause the system to operate inefficiently. The PCI architecture is a local bus design, meaning PCI devices have access directly to the CPU local bus. This is similar to the VESA designs. The PCI design is much more elegant and complete than any previously offered though. The PCI local bus is connected to the CPU local bus and system memory bus via a PCI-HOST bridge. This is a caching device and provides the sole interface between the CPU, memory and PCI local bus. Bus mastering devices and target(slave) devices can both reside on the same PCI local bus successfully. The PCI architecture allows the CPU to continue to fetch information from the caching bridge while still allowing the cache controller to provide an expansion device access to system memory. This is more than one communication on more than one bus occurring at once: concurrent bus operation, something that previous architectures (like VESA B) could not allow. Additionally, PCI expansion devices are fully independent of the CPU local bus, there is no CPU dependency at all. This design allows the CPU to be upgraded without impacting devices on the CPU or expansion busses. As long as the new processor maintains legacy architecture support, no bus design modifications are necessary. Communication between devices on the PCI bus use burst transfers. A burst transfer consists of an I/O cycle (in order for the initiator of the burst to attain master status on the bus) and two or more data cycles. The length of the burst is negotiated at the beginning of the transfer, and may be of any length. At burst completion, the receiving end (target) terminates the communication after the pre-determined amount of information has been received. Only one bus master device can communicate on the bus at a time. Other devices cannot interrupt the burst process because they do not have "master" status. Other architectures used arbitration and streaming techniques in order to allow communication on their expansion bus. VESA local bus designs also used a burst technique, but a burst session could be inopportunely interrupted by another expansion device. Another advantage to the PCI design is that up to 256 devices can be attached to one PCI local bus, and up to 256 PCI busses can exist in one system. This is a huge model for scalability in comparison to previous bus designs. PCI is a Plug and Play architecture - it is auto-configured by system BIOS. This means that system resource assignments destined for expansion devices (such as IRQ assignments and address space) is all automated, there is a greatly reduced chance of resource allocation conflicts. There are many more features of the PCI bus which support the claim that it is the most suitable architecture available to the PC industry today. Most of these features deal with how PCI handles high traffic situations in such an elegant and efficient fashion. HOW DOES THE PCI BUS WORK By breaking PCI down into discrete components it becomes easier to see how the architecture works. The basic components of the PCI bus include: PCI BIOS CPU CPU Cache System Cache System Memory PCI Bridge Peripheral bus Each component performs a specific task associated with accomplishing communication via the PCI bus. When all components work together, the resulting system is very efficient and extraordinarily dependable. Each component of the PCI system adheres to an "open standards" concept. This means that each component has a specific method that it uses to interact with adjacent components. Because these methods of component interaction are the same in all PCI systems, the PCI design is inherently adaptable to a variety of configurations. It is an easy matter to include embedded devices and peripheral devices on the same bus for this reason. Finally, the PCI bus is ultimately scaleable. Because it supports expansion by an extraordinary factor (256(busses) x 256(devices/bus) = 66536 logical PCI units) the PCI local bus design is well equipped to survive for a long time. Points such as the details of CPU and cache operation are beyond the scope of this document, only the most innovative of the PCI architectural components will be covered. PCI BIOS The PCI BIOS initializes and maintains the environment of the PCI bus. It is responsible for initializing all of the devices attached to the bus as well as maintaining some degree of order between the attached I/O devices. The PCI BIOS is responsible for assigning resources to devices during initialization. It is also responsible for system self-checks and, with Plug and Play extensions, ensuring that the information used to initialize the system is appropriate. The host OS (operating system) may not directly use PCI configuration registers in the controlling logic or peripheral devices in order to perform it's regular functions. To do so would be to create platform specific software. The host OS has the need to access the system from the bus level at times however, and this has been provided for. The PCI specification describes a universal PCI BIOS extension, which has resulted in a fairly wide scale register compatibility between PCI systems. PCI configuration space, initialized at system boot and held in system RAM, is one way for the OS to obtain information about system hardware. The PCI BIOS extensions also offer the OS a way to access hardware. PCI configuration space holds data from PCI initialization in specific formats. The interface that the host OS uses to access PCI configuration space must be of a format that is recognized by the entire system. The PCI local bus specification dictates how the interface the OS uses should be constructed in order to comply with the specific formats used for PCI configuration space. PCI configuration space is held in system RAM. Any access requiring information about how a PCI device is configured may be accessed faster from system RAM than via a direct hardware access. After PCI BIOS has initialized the hardware and devices on the PCI bus are all powered up, control of the bus is momentarily passed to the OS. This is the point where the OS begins to load device drivers into memory. The driver first calls the PCI BIOS requesting a search for it's related device using vendor and device specific ID numbers. The BIOS responds to the call and scans the PCI and devices for the requested information. The driver can obtain information about where resource information is stored and what values have been assigned during this scan. This is how the device driver "knows" how to locate the device's particular region of PCI configuration space, which INT line and processor interrupt has been assigned, etc. Once the device is located, the OS finishes the load process and continues to load drivers for any remaining devices. When driver load is complete, BIOS resumes it's duty of watching over the PCI bus. It is important to note that during device driver load, the operating system is controlling the PCI bus and it's accesses. After driver load, control is returned to the BIOS. This allows the OS to have some control over how hardware operates, which is very useful for plug and play operating systems. Each OS may access the PCI BIOS in a number of ways. For instance Win95 can actually access the PCI configuration registers directly in addition to using PCI configuration space, allowing resource assignment modifications while interfacing directly with PCI BIOS. Windows NT® never uses the PCI BIOS at all, rather the Hardware Abstraction Layer, a sub-layer of the NT operating system, uses calls directly to PCI configuration registers in order to obtain information about the system. Both techniques have benefits and drawbacks, however both are valid means of obtaining device/resource specific information from the PCI bus. A unique aspect of the PCI BIOS is the ability to share interrupts. The actual hardware implementation of shared interrupts is unique to each PCI system, but the concept is the same for all. While the PCI design employs local bus architecture, and is therefore closer to the CPU and host bus, it does not directly use IRQ's to interrupt the CPU as other architectures did: INT lines and IRQs are two different entities. An IRQ (interrupt request) is a direct means of getting the CPU's attention. Legacy busses required each expansion device to use a different, unique IRQ in order to communicate with the CPU. The user was required to set the IRQ (and other settings) via jumpers on the peripheral device. The PCI bus uses INT lines, which are then routed to IRQs by the PCI BIOS. The INT lines are named INTA, INTB, INTC and INTD. The BIOS assigns an INT line to a PCI peripheral device at initialization time. PCI devices may then use a granted INT line in order to access a system IRQ, assisted by the PCI BIOS. Using this method of INT routing, the BIOS is capable of assigning more than one IRQ to a particular INT line, thus allowing IRQs to be shared amongst PCI devices if required. To compound things, only one bus mastering device is active at any given time, therefore devices can share the same INTx. This concept is key in understanding why the PCI bus is so efficient. The PCI BIOS is in control over all system resources used by all PCI controllers. This design "intelligence" allows IRQ sharing in addition to other means of adjusting a device's behavior. The BIOS can use user setting to control PCI bus latency for example. This type of open design makes it possible to create peripheral devices that do not require jumper configuration for resource assignment, a notoriously dirty job on many legacy bus systems. The design also reduces the growing pains associated with expanding legacy systems, it used to be possible to literally run out of system resources (like IRQs) before running out of expansion slots. The PCI bus does not experience this limitation. PCI BRIDGE PCI-PCI and PCI-System bridges operate in the same way. A PCI bridge facilitates communication between the different busses in the PCI system. PCI bridges function as traffic coordinators, and reside between two distinct logical busses. Bridges never initiate a transaction on their own, whatever their designated function - they must be requested to do so. One of the tasks of the PCI bridge is to monitor transactions on either bus and to decide whether or not to pass a transaction through to the other bus. If it is determined that the bridge needs to pass a transaction through, the bridge acts as a target for the initiating device and, once the transaction has been received, the bridge initiates a new transaction on the destination bus. The bridge operation is entirely transparent to the original initiator and the final target. This is the only time a PCI bridge has the capacity to initiate a signal on one of the connected busses: a bridge initiates a signal on one of it's busses or to a device only when it is ordered to do so by another bus or device. Other bridge functions include monitoring and passing of specific control signals from one bus to the other. The SERR# signal is used by the PCI bus and devices to report to the PCI BIOS that an error has been encountered, and is one of the signals that the bridge is designed to respond to. The PCI bridge monitors for this signal and passes it to the next bus if it is present. The PCI bridge is versatile, providing it's own specific memory mapped configuration registers which can be used to customize it's functionality. A PCI bridge can include a ROM that contains hardware parameters used to control the bridge. A PCI bridge can even offer I/O mapped control registers, allowing software device driver interaction and control. The PCI bridge is registered by the system (PCI BIOS) as a PCI device, however it is not a mastering device and therefore does not receive an INT line from BIOS. PCI devices use a collection of signals in order to accomplish communication over the bus. This collection of signals is common to all PCI I/O controllers, which is why different PCI devices can function in placed in the same system. Some of the more important signals relative to PCI I/O transactions across the include: Memory Read - Used when transaction involves less than one whole cache line of data. Memory Read Line - Used when an entire cache line of data is to be transferred. Memory controller can pre-fetch the entire line from memory rather than accessing memory on a phase by phase basis. Memory Read Multiple - Used when more than one cache line of data is to be retrieved from memory. Memory Write - This command is used to transfer a cache line to memory. Memory Write and Invalidate - This command is used to guarantee the transfer of a complete cache line to memory in the event that the target memory block is "dirty." I/O Read/Write - These commands are used to transfer data between initiator and target devices on the PCI bus. Devices use these signals to pass information to and from each other. The memory controller is a PCI device, and usually resides on the CPU local bus. All devices must use the PCI-System bridge to access system memory. Any device needing to perform I/O needs to get it's signals across the system bridge to the CPU. The system bridge handles many transactions, making full use of transmit and receive buffers that have been included in the different system bridge designs that are available. Typically these buffers are fairly modest at 256 bytes, however the bridge controller is performing its duties fairly quickly (at 33MHz, or whatever frequency the host PCI bus is oscillating at) and the signals that need to be stored are fairly small with regards to physical size. CONCLUSION The PCI architecture is the best design available today for the PC industry. Previous architectures have not been able to provide adequate compatibility, scalability or dependability, and have actually impeded the growth of the PC industry. The advent of the PCI architecture has made it possible to develop new applications for the PC, such as enterprise class servers. The fundamental advantages of the PCI bus lie in it's inherently efficient, BIOS controlled local bus design as well as it's open standards aspects. Peripheral component manufacturers today develop products such as SCSI controllers and network adapters that take advantage of PCI bus design features. The open design of the PCI architecture provides a means of scalability. The underlying framework which dictates the architecture can be applied to new hardware designs which promote bus efficiency even more. Industry initiatives such as I2O are evidence of the PCI architectures scalability. It should be apparent that the PCI architecture will be an integral part of PCs for many years to come.