Computer Memory Organization
Elephants don’t forget… but do computers?
GOAL: to best supply programmers with (almost) infinitely extensible memory that is as fast as possible to use The general design is a hierarchy
The most expensive, fastest memory is in close proximity to the processor Memory contents are duplicated and expanded in banks lower in the hierarchy Successive memory banks increase in storage capacity and access latency, but decrease in cost and operational speed
Performance measurements between any two levels of hierarchy
Hit rate = ratio of times a desired element is in that level of memory to the number of total accesses to memory = (1-miss rate) Hit time = time to access the upper level of memory and deliver the item to the requesting device Miss penalty = time to bring the item into upper level and deliver the item
Memory and Design
Average memory access time is equal to (hit time + miss rate * miss penalty) Computer designers favor a block size with the lowest average access time rather than lowest miss rate Memory hierarchies also complicate the design of the CPU
Variable access time prevents memory from being directly on the datapath
The Memory Hierarchy
Smallest and closest to processor Many different organization strategies May be multiple levels of cache
The motherboard RAM you know and love
Switch out memory to disk Essential for shared, multiprocess systems
Where can you put a block in a cache?
direct mapped: a function maps a block to a unique location in the cache set associative: a function maps a block to a unique group of locations in the cache, but block can go anywhere in the cache fully associative: put the block anywhere
How do you find things in a cache?
For direct map, might invert map function Otherwise, must check each block (in parallel)
What block is replaced on a miss?
Random selection promotes uniform allocation, easy to implement Least recently used takes advantage of temporal locality (recently used blocks are more likely to be used again)
What happens on a write?
Write through - also writes to lower-level block Write back - doesn’t write lower-level block until the corresponding cache block is replaced
dirty bit on a cache block indicates update is necessary
We try to make main memory contain the entire state of the currently running system process; otherwise, we swap
Penalty for swapping with virtual memory very high (magnetic, mechanical storage)
We don’t worry about sync between main memory and virtual memory because we do not replicate main memory in virtual memory (very costly)
Improving Main Memory
Make memory bus wider (more at one time)
Also must widen cache Must multiplex before entering the datapath (convert many signals into one) May be wasteful (especially on writes)
Interleave memory banks (parallelize reading)
Successive words rotate through memory banks (word 1 = bank 1, word 2 = bank 2, …, word 5 = bank 1, word 6 = bank 2, …) Must still multiplex into the datapath Allows parallel writes (if one bank not overloaded)