# Lecture_24_2009.pptx

Document Sample

```					        CMSC 611-101

Lecture 25
I/O System

November 30, 2009

www.csee.umbc.edu/~younis/CMSC611/CMSC611.htm

Mohamed Younis      CMCS 611, Advanced Computer Architecture   1
Lecture’s Overview
 Previous Lecture:
 Optimizing performance of main memory
•Wider main memory bus
•Interleaving of memory access
•Banked memory
•Bank conflict avoidance
•DRAM interleaving
 Virtual memory and supporting access protection
•Designing simple and small cache
•Pipelined cache write
 This Lecture
 Effects of I/O on performance
 Storage systems:
 Types of of storage devices
 Anatomy and technology trends of magnetic disks
 Highly available storage systems
 Interfacing I/O devices
Mohamed Younis         CMCS 611, Advanced Computer Architecture   2
Computer Input/Output
Computer
 I/O Interface
 Device drivers                Processor           Memory           Devices
 Device controller                Control                           Input
 Service queues
Datapath                            Output
 Interrupt handling

 Design Issues
 Performance
 Expandability
 Standardization
 Resilience to failure     Computer
Processor
 Impact on Tasks                                      Memory           Devices

 Blocking conditions              Control                           Input
 Priority inversion
Datapath                            Output
 Access ordering

Mohamed Younis          CMCS 611, Advanced Computer Architecture             3
Impact of I/O on System Performance
Suppose we have a benchmark that executes in 100 seconds of elapsed time, where 90
seconds is CPU time and the rest is I/O time. If the CPU time improves by 50% per year
for the next five years but I/O time does not improve, how much faster will our program
run at the end of the five years?
Answer:                Elapsed Time = CPU time + I/O time
After n years              CPU time          I/O time         Elapsed time     % I/O time
0                    90 Seconds     10 Seconds        100 Seconds         10%
90
1                    60 Seconds    10 Seconds          70 Seconds        14%
1 .5
60
2                    40 Seconds    10 Seconds          50 Seconds        20%
1 .5
40
3                    27 Seconds    10 Seconds          37 Seconds        27%
1 .5
27
4                    18 Seconds    10 Seconds          28 Seconds        36%
1 .5
18
5                    12 Seconds    10 Seconds          22 Seconds        45%
1 .5

Over five years:
CPU improvement = 90/12 = 7.5                  BUT System improvement = 100/22 = 4.5
Mohamed Younis                     CMCS 611, Advanced Computer Architecture                4
Typical I/O System
interrupts
Processor

Cache

Memory - I/O Bus

Main            I/O                I/O                 I/O
Memory        Controller         Controller          Controller

Disk      Disk       Graphics              Network

 The connection between the I/O devices, processor, and memory are
usually called (local or internal) bus
 Communication among the devices and the processor use both protocols
on the bus and interrupts
Mohamed Younis                 CMCS 611, Advanced Computer Architecture      5
I/O Device Examples
Device                Behavior             Partner                  Data Rate (KB/sec)
Keyboard               Input                Human                                0.01

Mouse                  Input                Human                                0.02

Line Printer           Output               Human                                1.00

Floppy disk            Storage              Machine                           50.00

Laser Printer          Output               Human                           100.00

Optical Disk           Storage              Machine                         500.00

Magnetic Disk          Storage              Machine                         5,000.00

Network-LAN           Input or Output       Machine                     2000 – 1,000,000

Graphics Display       Output               Human                           30,000.00

* Slide is courtesy of Dave Patterson
Mohamed Younis              CMCS 611, Advanced Computer Architecture                                       6
Storage Technology Drivers
 Driven by the prevailing computing paradigm:
1950s: migration from batch to on-line processing
1990s: migration to ubiquitous computing
• Computers in phones, books, cars, video cameras, …
• Nationwide fiber optical network with wireless tails
 Effects on storage industry:
Embedded storage: smaller, cheaper, more reliable, lower power
Data utilities: high capacity, hierarchically managed storage
Famous storage devices:
 Hard drive: random access, magnetic based, various density and speed
 Tape: sequential access, huge storage capacity, cheap and replaceable
 Helical scan tapes: diagonal storage of bits to allow high speed tape rotation
(used also for VCR and camcorders)
 Flash DRAM: little slower than DRAM, expensive and limited in capacity
 Optical disk: high density, read-only, CD ROM is the only success story
* Slide is partially a courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                            7
Disk History
Data density in
Mbit/square inch

Capacity of Unit
Shown in Megabytes

source: New York Times, 2/23/98, page C3
Mohamed Younis     CMCS 611, Advanced Computer Architecture          8
Magnetic Disk
 Purpose:

Registers
 Long term, nonvolatile storage

Cache

Memory
 Large, inexpensive, and slow
 Low level in the memory hierarchy

Disk
 Two major types:
 Floppy disk
 Hard disk
 Both types of disks:
 Rely on a rotating platter coated with a magnetic surface
 Advantages of hard disks over floppy disks:
 Platters are more rigid ( metal or glass) so they can be larger
 Higher density because it can be controlled more precisely
 Higher data rate because it spins faster
 Can incorporate more than one platter
* Slide is courtesy of Dave Patterson
Mohamed Younis            CMCS 611, Advanced Computer Architecture                                         9
Organization of a Hard Magnetic Disk

Platters

Track

 Typical numbers (depending on the disk size):     Sector
 500 to 2,000 tracks per surface
 32 to 128 sectors per track
• A sector is the smallest unit that can be read or written to
 Traditionally all tracks have the same number of sectors:
 Constant bit density: record more sectors on the outer tracks
 Recently relaxed: constant bit size, speed varies with track location
* Slide is courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                      10
Magnetic Disk Operation
Track
 Cylinder: all the tacks under the head                                                 Sector
at a given point on all surface
 Read/write data is a three-stage process:
 Seek time: position the arm over proper track                                               Cylinder
 Rotational latency: wait for the desired sector                         Head                    Platter
 Transfer time: transfer a block of bits (sector)
 Average seek time
 (Sum of the time for all possible seek) / (total # of possible seeks)
 Typically in the range of 8 ms to 12 ms (as reported by the industry)
 Due to locality of disk reference, actual average seek time may only
be 25% to 33% of the advertised number

* Slide is courtesy of Dave Patterson
Mohamed Younis            CMCS 611, Advanced Computer Architecture                                      11
Magnetic Disk Characteristic
 Rotational Latency:                                                          Track

 Most disks rotate at 3,600 to 7,200 RPM                                        Sector
 Approximately 16 ms to 8 ms
per revolution, respectively
 An average latency to the desired
information is halfway around the disk:
Cylinder
8 ms at 3600 RPM, 4 ms at 7200 RPM
Platter
 Transfer Time is a function of :
 Transfer size (usually a sector): 1 KB / sector
 Rotation speed: 3600 RPM to 7200 RPM
 Recording density: bits per inch on a track
 Diameter: typical diameter ranges from 2.5 to 5.25 in
 Typical values: 2 to 12 MB per second
* Slide is courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                        12
Example
Calculate the access time for a disk with 512 byte/sector and 12 ms advertised seek
time. The disk rotates at 5400 RPM and transfers data at a rate of 4MB/sec. The
controller overhead is 1 ms. Assume that the queue is idle (so no service time)
Disk Access Time = Seek time + Rotational Latency + Transfer time
+ Controller Time + Queuing Delay

= 12 ms + 0.5 / 5400 RPM + 0.5 KB / 4 MB/s + 1 ms + 0

= 12 ms + 0.5 / 90 RPS + 0.125 / 1024 s + 1 ms + 0

= 12 ms + 5.5 ms                  + 0.1 ms                + 1 ms + 0 ms

= 18.6 ms

If real seeks are 1/3 the advertised seeks, disk access time would be
10.6 ms, with rotation delay contributing 50% of the access time!
* Slide is courtesy of Dave Patterson
Mohamed Younis               CMCS 611, Advanced Computer Architecture                                      13
Historical Trend
Characteristics                IBM 3090        IBM UltraStar          Integral 1820

Disk diameter (inches)           10.88              3.50                   1.80

Formatted data capacity (MB)    22,700             4,300                       21

MTTF (hours)                    50,000           1,000,000               100,000

Number of arms/box                 12                  1                         1

Rotation speed (RPM)             3,600              7,200                    3,800

Transfer rate (MB/sec)              4.2               9-12                    1.9

Power/box (watts)                2,900                 13                        2

MB/watt                              8                102                    10.5

Volume (cubic feet)                 97                0.13                   0.02

MB/cubic feet                      234               33000                   1050
* Slide is courtesy of Dave Patterson
Mohamed Younis         CMCS 611, Advanced Computer Architecture                                      14
Reliability and Availability
 Two terms that are often confused:
 Reliability: Is anything broken?
 Availability: Is the system still available to the user?
 Availability can be improved by adding hardware:
 Example: adding ECC on memory
 Reliability can only be improved by:
 Enhancing environmental conditions
 Building more reliable components
 Building with fewer components
• Improve availability may come at the cost of
lower reliability

* Slide is courtesy of Dave Patterson
Mohamed Younis        CMCS 611, Advanced Computer Architecture                                      15
Disk Arrays
 A new organization of disk storage:
 Arrays of small and inexpensive disks
 Increase potential throughput by having
many disk drives:
 Data is spread over multiple disk
 Multiple accesses are made to several disks

 Reliability is lower than a single disk:
 Reliability of N disks = Reliability of 1 Disk ÷ N
(50,000 Hours ÷ 70 disks = 700 hours,
 Disk system MTTF: Drops from 6 years to 1 month
 Arrays (without redundancy) too unreliable to be useful!
 But availability can be improved by adding redundant disks (RAID):
Lost information can be reconstructed from redundant information
 Hot spares support reconstruction in parallel with access: very high
media availability can be achieved
* Slide is partially courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                            16
Disk Product Families

Conventional:
1 disk
3.5” 5.25”                    10”                     14”
designs

Low End                                  High End

Disk Array:
4 disk design
3.5”

Replace Small # of Large Disks with Large # of Small Disks!
* Slide is partially courtesy of Dave Patterson
Mohamed Younis       CMCS 611, Advanced Computer Architecture                                            17
Redundant Arrays of Disks
Redundant Array of Inexpensive Disks (RIAD)
 Widely available and used in today’s market
 Files are "striped" across multiple spindles
 Redundancy yields high data availability despite low reliability
 Contents of a failed disk is reconstructed from data redundantly stored in the
disk array
 Drawbacks include capacity penalty to store redundant data and bandwidth
penalty to update a disk block
 Different levels based on replication level and recovery techniques
RAID level                     Failures survived          Data disks Check disks
0 Non-redundant                                   0                      8          0
1 Mirrored                                           1                        8      8
2 Memory-style ECC                                   1                        8      4
3 Bit-interleaved parity                             1                        8      1
4 Block-interleaved                                  1                        8      1
5 Block-interleaved distributed parity               1                        8      1

Mohamed Younis                CMCS 611, Advanced Computer Architecture               18
recovery
group

 Each disk is fully duplicated onto its "shadow“
 Very high availability can be achieved
 Bandwidth sacrifice on write: Logical write = two physical writes
 Most expensive solution: 100% capacity overhead

Targeted for high I/O rate , high availability environments
* Slide is courtesy of Dave Patterson
Mohamed Younis       CMCS 611, Advanced Computer Architecture                                      19
RAID 3: Parity Disk
10010011
11001101                                                                     P
10010011
...
logical record                   1              1              1                 0
0              1              0                 0
Striped physical                  0              0              0                 1
records                      1              0              1                 1
0              1              0                 0
0              1              0                 0
1              0              1                 0
1              1              1                 0
 Parity computed across recovery group to protect against hard disk failures
 33% capacity cost for parity in this configuration: wider arrays reduce
capacity costs, decrease expected availability, increase reconstruction time
 Arms logically synchronized, spindles rotationally synchronized
(logically a single high capacity, high transfer rate disk)
Targeted for high bandwidth applications: Scientific, Image Processing
* Slide is courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                      20
Block-Based Parity
 Block-based party leads to more efficient read access compared to RAID 3
 Designating a party disk allows recovery but will keep it idle in the absence
of a disk failure
 RAID 5 distribute the party block to allow the use of all disk and enhance
parallelism of disk access

RAID 4                                            RAID 5
Mohamed Younis           CMCS 611, Advanced Computer Architecture           21
Problems of Small Writes
RAID-5: Small Write Algorithm
1 Logical Write = 2 Physical Reads + 2 Physical Writes

D0'          D0      D1          D2          D3            P

new               old                                       old

+ XOR

+ XOR

(3. Write)                               (4. Write)

D0'      D1          D2          D3           P'

* Slide is courtesy of Dave Patterson
Mohamed Younis             CMCS 611, Advanced Computer Architecture                                      22
RAID 5+: High I/O Rate Parity
Increasing
A logical write       D0         D1          D2          D3            P                   Logical
becomes four                                                                                Disk
D4         D5          D6           P            D7
Independent writes
possible because of
interleaved parity    D8         D9           P         D10            D11
Reed-Solomon
Codes ("Q") for       D12         P         D13         D14            D15
protection during
reconstruction                                                                                Stripe

P         D16         D17         D18            D19                    Stripe
Unit

D20       D21         D22         D23             P
Targeted for mixed      .           .         .      .                   .
applications            .           .         .
Disk Columns.                   .
.           .         .      .                   .
* Slide is courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                           23
Subsystem Organization
single board
host       array                                 disk
host

manages interface                                                 single board
to host, DMA                                                          disk
controller
control, buffering,
parity logic
single board
disk
physical device                                                     controller
control

striping software off-loaded from                               single board
host to array controller                                      disk
controller
no applications modifications
often piggy-backed
no reduction of host performance                    in small format devices

* Slide is courtesy of Dave Patterson
Mohamed Younis                 CMCS 611, Advanced Computer Architecture                                      24
System Availability: Orthogonal RAIDs
String
Controller                                        . . .

String
Controller                                        . . .

String
Array         Controller                                         . . .
Controller
String
Controller                                         . . .

String
Controller                                         . . .

String
Controller                                         . . .

Data Recovery Group: unit of data redundancy
Redundant Support Components: fans, power supplies, controller, cables
End to End Data Integrity: internal parity protected data paths
* Slide is courtesy of Dave Patterson
Mohamed Younis                CMCS 611, Advanced Computer Architecture                                         25
System-Level Availability
host                                                              host
I/O Controller         Fully dual redundant            I/O Controller

Array Controller                                      Array Controller
...                                                                      ...

...
Goal: No Single
Points of
...                             Failure

...

.         with duplicated paths, higher performance can be
Recovery           .                obtained when there are no failures
Group              .
* Slide is courtesy of Dave Patterson
Mohamed Younis              CMCS 611, Advanced Computer Architecture                                        26
I/O Devices’ Interface
Two methods are used to address the device:
 Special I/O instructions: (Intel 80X86, IBM 370)
 Specify both the device number and the command word
•   Device number: the processor communicates this via a
set of wires normally included as part of the I/O bus
•   Command word: this is usually send on the bus’s data lines
•   Each devices maintain status register to indicate progress
 Instructions are privileged to prevent user tasks from directly accessing
the I/O devices

 Memory-mapped I/O: (Motorola/IBM PowerPC)
 Portions of the address space are assigned to I/O devices
 Read and writes to those addresses are interpreted as commands to the
I/O devices
 User programs are prevented from issuing I/O operations directly:
* Slide is courtesy of Dave Patterson
Mohamed Younis              CMCS 611, Advanced Computer Architecture                                      27
Communicating with I/O Devices
 The OS needs to know when:
 The I/O device has completed an operation
 The I/O operation has encountered an error
 This can be accomplished in two different ways:
 Polling:
• The I/O device put information in a status register
• The OS periodically check the status register
 I/O Interrupt:
• An I/O interrupt is an externally stimulated event, asynchronous to
instruction execution but does NOT prevent instruction completion
• Whenever an I/O device needs attention from the processor, it
interrupts the processor from what it is currently doing
• Some processors deals with interrupt as special exceptions
These schemes requires heavy processor’s involvement and
suitable only for low bandwidth devices such as the keyboard
* Slide is partially a courtesy of Dave Patterson
Mohamed Younis           CMCS 611, Advanced Computer Architecture                                            28
Polling: Programmed I/O
CPU
Is the
data                 busy wait loop
way to use the CPU
unless the device
Memory                                yes    no                  is very fast!
IOC
data
but checks for I/O
device                                        completion can be
dispersed among
store                   computation
data                   intensive code
done?            no
yes
 Simple: the processor is totally in control and does all the work
 Polling overhead can consume a lot of CPU time
* Slide is courtesy of Dave Patterson
Mohamed Younis              CMCS 611, Advanced Computer Architecture                                      29
Interrupt Driven Data Transfer
CPU                                                 sub                    user
(1) I/O                  and                    program
interrupt                or
nop
(2) save PC

Memory
IOC              (3) interrupt
store                     interrupt
device                                        ... :                     service
(4)       rti                       routine

 User program progress is only halted during actual transfer
 Disadvantage: special hardware is needed to:
 Cause an interrupt (I/O device)
 Detect an interrupt (processor)
 Save the proper states to resume after the interrupt (processor)
* Slide is courtesy of Dave Patterson
Mohamed Younis              CMCS 611, Advanced Computer Architecture                                        30
I/O Interrupt vs. Exception
 An I/O interrupt is just like the exceptions except:
 An I/O interrupt is asynchronous
 Further information needs to be conveyed
 Typically exceptions are more urgent than interrupts
 An I/O interrupt is asynchronous with respect to instruction execution:
 I/O interrupt is not associated with any instruction
 I/O interrupt does not prevent any instruction from completion
•   You can pick your own convenient point to take an interrupt

 I/O interrupt is more complicated than exception:
 Needs to convey the identity of the device generating the interrupt
 Interrupt requests can have different urgencies:
•   Interrupt request needs to be prioritized
•   Priority indicates urgency of dealing with the interrupt
•   high speed devices usually receive highest priority

* Slide is courtesy of Dave Patterson
Mohamed Younis               CMCS 611, Advanced Computer Architecture                                      31
Direct Memory Access
 Direct Memory Access (DMA):                              CPU sends a starting address,
 External to the CPU                                   direction, and length count
to DMAC. Then issues "start".
 Use idle bus cycles (cycle stealing)
 Act as a master on the bus
 Transfer blocks of data to or from memory                               CPU
without CPU intervention
 Efficient for large data transfer, e.g. from disk
 Cache usage allows the processor to leave
enough memory bandwidth for DMA
 How does DMA work?:                                          Memory          DMAC                    IOC
 CPU sets up and supply device id, memory
device
 DMA controller (DMAC) starts the access
and becomes the bus master                                DMAC provides handshake
signals for Peripheral
 For multiple byte transfer, the DMAC                     Controller, and Memory
 DMAC interrupts the CPU upon completion                  signals for Memory.

For multiple bus system, each bus controller often contains DMA control logic
* Figure is courtesy of Dave Patterson
Mohamed Younis               CMCS 611, Advanced Computer Architecture                                     32
DMA Problems
 With virtual memory systems: (pages would have physical and virtual addresses)
 Physical pages re-mapping to different virtual pages during DMA operations
 Multi-page DMA cannot assume consecutive addresses
Solutions:
 Allow virtual addressing based DMA
 Add translation logic to DMA controller
 OS allocated virtual pages to DMA prevent re-mapping until DMA completes
 Partitioned DMA
 Break DMA transfer into multi-DMA operations, each is single page
 OS chains the pages for the requester

 In cache-based systems: (there can be two copies of data items)
 Processor might not know that the cache and memory pages are different
 Write-back caches can overwrite I/O data or makes DMA to read wrong data
Solutions:
 Route I/O activities through the cache
 Not efficient since I/O data usually is not demonstrating temporal locality
 OS selectively invalidates cache blocks before I/O read or force write-back prior
to I/O write
 Usually called cache flushing and requires hardware support
DMA allows another path to main memory with no cache and address translation
Mohamed Younis                 CMCS 611, Advanced Computer Architecture           33
I/O Processor
 An I/O processor (IOP) offload the CPU
CPU           IOP                 D1
 Some of the new processors, e.g.
main memory                   D2         Motorola 860, include special purpose
Mem     bus                       . . .      IOP for serial communication
Dn
I/O
bus                                       target device
where cmnds are

(1) Issues                   (4) IOP interrupts
instruction                      CPU when done
to IOP           IOP                                     IOP looks in memory for commands
(2)

memory
Device to/from memory                           what                                       special
transfers are controlled                        to do                                      requests
by the IOP directly.                                        where        how
to put       much
IOP steals memory cycles.                                    data
* Slide is courtesy of Dave Patterson
Mohamed Younis                   CMCS 611, Advanced Computer Architecture                                      34
Operating System’s Role
 Operating system acts as an interface between I/O hardware and programs
 Important characteristics of the I/O systems:
 The I/O system is shared by multiple program using the processor
 I/O systems often use interrupts to communicate information about I/O
•   Interrupts must be handled by OS because they cause a transfer to supervisor mode
 The low-level control of an I/O device is complex:
•   Managing a set of concurrent events
•   The requirements for correct device control are very detailed

 Operating System’s Responsibilities
 Provide protection to shared I/O resources
•   Guarantees that a user’s program can only access
•   Supply routines that handle low-level device operation
 Handles the interrupts generated by I/O devices
•    All user programs must have equal access to the I/O resources
 Schedule accesses in order to enhance system throughput
 Provides abstraction for accessing devices through allowed set of I/O services

Mohamed Younis                 CMCS 611, Advanced Computer Architecture              35
Conclusion
 Summary
 I/O systems architecture
•    I/O design issues
•    I/O devices
 Magnetic Disk
•    Access time and performance characteristics
•    Theory of operation and historical trend
•    Disk non-functional attributes (reliability and availability)
•    Redundant array of inexpensive disks
 Interfacing I/O devices
•    Interfacing with I/O devices
•    Communication with I/O devices
•    Operating System’s role
 Next Lecture
 Memory to processor interconnect
 Performance of I/O systems

Read: Section 6.1, 6.2 & 6.5 in textbook (3rd Ed.), or 6.1-6.4 (4th Ed.)

Mohamed Younis                 CMCS 611, Advanced Computer Architecture          36

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 0 posted: 5/18/2013 language: Latin pages: 36
yaofenji