CS1104 Computer Organisation

							CS1104: Computer Organisation http://www.comp.nus.edu.sg/~cs1104

School of Computing National University of Singapore

PII Lecture 8: Memory Hierarchy
    
Memory Technologies Memory Hierarchy Analogy Illusion of Large, Fast, Cheap Memory Principle of Locality Terminology

CS1104-P2-8

Memory Hierarchy

2

PII Lecture 8: Memory Hierarchy
 Reading:
 Chapter 8 of textbook, which is Chapter 5 in
“Computer Organization” by Hamacher, Vranesic and Zaky.

CS1104-P2-8

Memory Hierarchy

3

Memory Technologies

 RAM (Random Access Memory): access time

is the same for all locations (in nanoseconds). DRAM: Dynamic RAM  High density, low power, cheap, slow (access
time: 60-120 ns).  Dynamic: needs to be “refreshed” regularly.

 SRAM: Static RAM

 Low density, high power, expensive, fast (access
time: 5-25 ns).  Static: non-volatile – content lasts “forever” (until power is removed).

CS1104-P2-8

Memory Hierarchy

4

Memory Technologies (2)

 ROM (Read-Only Memory): non-volatile,
   
allows for read only. PROM: Programmable ROM EPROM: Erasable PROM EEPROM: Electrically EPROM Flash Memory  Similar to EEPROM  Difference: read a single cell, but write a block

CS1104-P2-8

Memory Hierarchy

5

Recap: Memory Organisation
21-bit addresses A0 A1 19-bit internal chip address

A19 A20

2-bit decoder

512K x 8 memory chip
19-bit address 8-bit data input/output

512k X 8 memory chip

D31-24

D23-16

D 15-8

D7-0

Chip select

Another example: Organization of a 2M  32 memory module using 512K  8 static memory chips.
Memory Hierarchy 6

CS1104-P2-8

Technology Trends
Processor: DRAM: Disk: Year 1980 1983 1986 1989 1993 1997
CS1104-P2-8

Capacity -4x in 3 yrs 4x in 3 yrs

Speed (latency) 4x in 3 yrs 2x in 10 yrs 2x in 10 yrs Cycle Time 250 ns 220 ns 190 ns 165 ns 145 ns 120 ns 2:1!
7

DRAM Size 64 Kb 256 Kb 1 Mb 4 Mb 16 Mb 64 Mb 1000:1!
Memory Hierarchy

Technology Trends (2)
1000 µProc 60%/yr. (2X/1.5yr)

CPU

100

Processor-Memory Performance Gap: (grows 50% / year)

10
DRAM

1
1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

DRAM 9%/yr. (2X/10 yrs)

Time
CS1104-P2-8 Memory Hierarchy 8

Analogy: Term Paper in Library

 Working on a paper at a desk in library.  Option 1: Every time a book is needed…
      
CS1104-P2-8

Leave desk to go to shelves (or stacks) Find the book Bring one book back to desk Read section interested in When done, leave desk and go to shelves carrying book Put book back on shelf Return to desk to work

Memory Hierarchy

9

Analogy: Term Paper in Library (2)

 Option 2: Every time a book is needed…

 Leave some books on desk after fetching them  Only go to shelves when a book not on desk is 
needed At the shelves, fetch related books in case you need them; sometimes you will need to return books not used recently to make space for new books on desk (replacement algorithm) Return to desk to work

  Illusion: whole library on your desk

CS1104-P2-8

Memory Hierarchy

10

Illusion of Large, Fast, Cheap Memory

 Registers in processor: Very fast, in order of    
nanoseconds to picoseconds. Disks: Very slow, in order of milliseconds. Large memories (disks) are slow; fast memories (registers) are small. How do we create a memory that is large, cheap and fast (most of the time)? Memory hierarchy.

CS1104-P2-8

Memory Hierarchy

11

Memory Hierarchy Pyramid
Central Processor Unit (CPU)
“Upper”
Level 1

Levels in memory hierarchy

Level 2
Level 3

Increasing distance from CPU, decreasing cost/bit

...

“Lower”

Level n

Size of memory at each level (Data cannot be in level i unless it is also in i+1.)
CS1104-P2-8 Memory Hierarchy 12

Principle of Locality

 Principle of locality (or locality of
reference):  Program accesses a relatively small
portion of the address space at any instant of time.  Temporal locality and spatial locality.
Probability of reference

0
Address Space
CS1104-P2-8 Memory Hierarchy

2 -1

n

13

Principle of Locality (2)

 Temporal locality (locality in time):


  Keep most recently accessed data items closer
to the processor. Library analogy: Recently read books are kept on desk.

 Spatial locality (locality in space):
 

  Move blocks consisting of contiguous words to
„upper‟ levels. Block is a unit of transfer. Library analogy: Bring back nearby books on shelves when fetching a book, hoping that you might need them soon.
Memory Hierarchy

CS1104-P2-8

14

Principle of Locality (3)

 What programming constructs lead to
principle of locality?
Spatial arrays no branch/jump Temporal loop counters loop

Data Code

CS1104-P2-8

Memory Hierarchy

15

Memory Hierarchy: Idea

 Temporal locality: keep recently accessed

data items closer to processor. Spatial locality: move contiguous words in memory to upper levels of hierarchy. Use smaller and faster memory technologies closer to the processor If hit rate is high enough, hierarchy has access time close to the highest (fastest) level and size equal to the lowest (largest) level.
Memory Hierarchy 16




CS1104-P2-8

Memory Hierarchy: Terminology

 Hit: data appears in some block in the upper
level.  Hit rate: the fraction of memory access found in
the upper level. Analogy: fraction of time you find the book on desk.



 Miss: data is not at upper level and needs to
be retrieved from a block in the lower level.  Miss rate: 1 – Hit rate  Analogy: fraction of time you need to go to
shelves for the book.
CS1104-P2-8 Memory Hierarchy 17

Memory Hierarchy: Terminology (2)

 (Average) Hit time: time to access the upper
level which consists of  Time to determine hit/miss + memory access
time. Analogy: time to find and pick up book from desk.

  (Average) Miss penalty: time to replace a block in the upper level + time to deliver the block to the processor.  Analogy: time to go to shelves, find needed book,

CS1104-P2-8

and return to your desk. Hit time << Miss penalty.
Memory Hierarchy 18

Current Memory Hierarchy
Processor Register
Primary L1 cache

Increasing size

Increasing Increasing speed cost per bit

Secondary cache L2

Main memory

Magnetic disk secondary memory

CS1104-P2-8

Memory Hierarchy

19

Current Memory Hierarchy (2)
Processor Control Data path L2 Cache Main Memory Secondary Memory

L1 $

Speed(ns): Size (MB): Cost ($/MB): Technology:

Regs

0.5ns 0.0005 -Regs

2ns 0.05 $100 SRAM

6ns 100ns 1-4 100-1000 $30 $1 SRAM DRAM

10,000,000ns 100,000 $0.05 Disk

CS1104-P2-8

Memory Hierarchy

20

Management of the Hierarchy

 Registers  Memory
 By compiler (or assembly programmer).  Cache  Main memory  By the hardware.  Main memory  Disks  By the hardware and operating system.  By the programmer (through files).

CS1104-P2-8

Memory Hierarchy

21

Virtual Memory

 Main memory smaller than address space
 Parts of program not in main memory are 

 Example: 32-bit address allows an address space
of 4G bytes, but main memory may only be a few hundred megabytes.

stored on secondary storage devices, such as disks. Operating system moves programs and data automatically between the physical main memory and secondary storage (virtual memory).
Memory Hierarchy 22

CS1104-P2-8

Things To Remember

 Principle of locality and hierarchy of
memories of different speed and cost; exploit locality to improve costperformance.  Terms: Hit, miss, hit rate, miss rate, hit time, miss penalty, block, upper-level memory, lower-level memory.

CS1104-P2-8

Memory Hierarchy

23

End of file

24