CSE5343 Summer 1997
3
CSE 5343 SUMMER 1997 OPERATING SYSTEMS AND SYSTEMS SOFTWARE
Professor Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist University Dallas, Texas 75275 214 768-3087 fax: 214 768-3085 email: mhd@seas.smu.edu www: http: www.seas.smu.edu mhd June 1, 1997
I. INTRODUCTION AND HISTORY What is an Operating System? Why OS? OS History
1
CSE5343 Summer 1997
2
CSE5343 Summer 1997
4
CSE5343 SUMMAER 1997 OUTLINE I. Introduction and History II. Processes III. Memory Management IV. Secondary Storage and File Systems V. Additional Topics Protection, Security, Distributed OS
WHAT IS AN OPERATING SYSTEM? Software that allows computer to be used easily provides a virtual machine for users manages the system resources Alternative OS Views Resource Allocator Manager Translator Levels of Abstraction Why OS? Convenient for users to use computers E cient use of computer resources
CSE5343 Summer 1997
5
CSE5343 Summer 1997
7
WHAT ARE THE RESOURCES THAT ARE MANAGED? Memory CPU Processes I O Devices Files Processors Fig 2.1 p30
WHAT FUNCTIONS DOES AN OS PERFORM? Memory management Secondary storage management Process management Processor management Interrupt handler Error handling Protection Statistics Accounting
CSE5343 Summer 1997
6
CSE5343 Summer 1997
8
WHAT ARE THE MANAGEMENT FUNCTIONS? Keep track of resource Determine who process gets what, when, and how much Allocate resource Reclaim resource
INTERRUPT Response to asynchronous or exceptional event Automatically save current CPU status IC,Registers,Files Transfer to speci ed routine associated with interrupt Fig 2.2 p31 Context switch between programs Interrupt types Program event unexpected I O I O Completion Interrupt di erent from I O Request Interrupt Multiprocessor SVC Time clock
CSE5343 Summer 1997
9
CSE5343 Summer 1997
11
ACCESSING OS - SYSTEM CALLS Interface between program and OS Request supervisor activities Fig 3.2 p68 Process job control Device manipulation File manipulation Information maintenance Communications
OS KERNEL Primitive basic OS functions Precise de nition depends on system Basic building blocks Kernel is the one program running at all times on the computer, with all else being applications programs." p. 5 in your textbook Process management, memory management, basic I O, protection OS structure Simple Fig 3.6 p77 - MSDOS Layered Fig 3.7 p78 - Unix
CSE5343 Summer 1997
10
CSE5343 Summer 1997
12
ACCESSING OS - SYSTEM PROGRAMS Command interpreter File manipulation Status information File modi cation Programming language support Program loading execution Communications Applications Programs DBMS
OS HISTORY 1st Generation - 1940s No OS Only hardware Programmer operator Sign up Introduction of software tools compilers, assemblers, linkers, loaders, device drivers, etc. Fig 1.1 p4
CSE5343 Summer 1997
13
CSE5343 Summer 1997
15
OS HISTORY 2nd Generation - 1950s Simple Batch - Evolved over time Reduced job set-up and tear-down time Batch jobs together Programming languages Automatic job sequencing Control cards Fig 1.3 p10 Fig 1.2 p8 Interrupts Standard I O routines Error recovery O -line printing and card reading Fig 1.4 p11 Spooling Fig 1.5 p13
OS HISTORY 4th Generation - 1960s 1970s Time-Sharing Systems OS switches between jobs completion, I O, time slice, SVC User can interact with job while it is running Use of CRTs On-line commands
CSE5343 Summer 1997
14
CSE5343 Summer 1997
16
OS HISTORY 3rd Generation - 1960s Multiprogrammed Batch Multiprogramming Overlap CPU and I O operations Partition memory for multiple jobs Fig 1.6 p14 Compensate for di erence in CPU and I O speeds channels, bu ers, spooling, interrupts Device independence IBM OS 360
OS HISTORY 5th Generation - 1980s,1990s Personal Computers, Work Stations, Supercomputers Single user multitasking Distributed systems Parallel systems Real-time systems
CSE5343 Summer 1997
17
CSE5343 Summer 1997
19
THREAD II. PROCESSES Introduction What is a process? How does OS manage a process? Process scheduling Interprocess communication Process synchronization Deadlocks Case Studies MS-DOS, Windows 95, MVS, Unix Basic unit of CPU utilization Shares with peer threads code, data, les Dispatching unit Heavyweight process is a task with one thread Process does nothing if no threads Switching between peer threads faster and less expensive then a normal context switch between processes Thread context switch requires register set switch, but no memory management or le related work
CSE5343 Summer 1997
18
CSE5343 Summer 1997
20
WHAT IS A PROCESS? Program in execution Not the code, but the state Related Terms Program - static Job - Batch program in execution. Task - Time sharing program in execution; Atomic unit of computation Thread - Lightweight process; shares code with other threads
PROCESS STATES Running - currently executing Waiting - blocked waiting for resource Ready - waiting to be scheduled New - being created Terminated - nished execution Fig 4.1 p98 Suspended - waiting for non I O event; external request caused suspension Exact states depend on OS For each state transition there is an action that caused it
CSE5343 Summer 1997
21
CSE5343 Summer 1997
23
PROCESS CONTROL BLOCK Information used to represent process Information needed to restart process Contents Thread Level:
PROCESS CREATION Fork - System call to create a new child process Parent and child may may not execute in parallel Child process may be a duplicate of the parent or have a new program New PCB created and placed into correct queue When OS booted a set of initial processes is created All other processes including OS ones are descendents of these Fig 4.5 p103 shows how child process forked Fig 4.7 p 106 tree of processes
ID state register values instruction counter scheduling information
Process Level:
memory management information accounting information I O and le information
CSE5343 Summer 1997
22
CSE5343 Summer 1997
24
HOW DOES OS MANAGE PROCESSES? One Approach PCB implemented as record Linked list of PCBs per state; Alternative structure could be priority queue Fig 4.4 p102 State header at xed location in memory PCBs do not move around May have speci c memory area set aside for PCBs
PROCESS TERMINATION Exit - System call to terminate a process Data returned to parent process PCB freed up and other resources released Abort - System call from parent to terminate child
CSE5343 Summer 1997
25
CSE5343 Summer 1997
27
PROCESS SCHEDULING Which process should get the CPU next? Long Term switch between new and ready states MP level Pick new Process FCFS, Priority Create PCB Allocate memory, read in program Primarily batch Executes infrequently Intermediate Level Temporarily suspending and activating processes Short Term Which processor gets which process? Dispatcher - Gives control of CPU to selected process Executes frequently
PREEMPTIVE SCHEDULING CPU can be taken away High priority processes require rapid attention Overhead high Fig 4.1 p134
CSE5343 Summer 1997
26
CSE5343 Summer 1997
28
NONPREEMPTIVE SCHEDULING Process keeps CPU until it decides to give it up Give up at I O or SVC or termination When it chooses Fair treatment Response times more predictable Simple
MEASURING CPU SCHEDULING PERFORMANCE Response Time - Time to rst response Turnaround Time - Total elapsed time including IO Normalized Turnaround - Turnaround divided by run time Throughput - Number of processes per unit time Waiting Time - Time in ready queue For multiples processes you need the average value CPU Utilization
CSE5343 Summer 1997
29
CSE5343 Summer 1997
31
SCHEDULING CONSIDERATIONS Availability of limited resources Program needed resources Priority User speci ed requirements deadline I O or CPU Bound balance resource usage Variance of response times Minimize average response time Minimize maximum response time Maximize throughput Avoid starvation
TIME SLICE Program gives up CPU at prede ned service time Avoids Convoy e ect How Choose? Large - approaches FCFS Small - frequent context switches May cause thrashing Maximum Wait Time = N-1 *quantum Typical values 10-100 milliseconds
CSE5343 Summer 1997
30
CSE5343 Summer 1997
32
SCHEDULING ALGORITHMS FCFS SJF SRT Priority Round-Robin Multilevel Queue Multilevel Feedback Queue Example No I O, Context Switch time is 0, Suppose quantum = 0.5 where applicable : Process 1 2 3 Arrival Runtime 10.00 2.0 10.10 1.0 10.25 .25
10 PR1 PR2 PR3
FCFS FIFO Simplest - Nonpreemptive FIFO Queue - Ordered according to time process arrives Convoy E ect Poor response time; Large variance Example:
11 12 1 1.25 Waiting Running
Process Turnaround Normalized Turnaround 1 2 1 2 2.9 2.9 3 3 12 Average 2.63 5.3
CSE5343 Summer 1997
33
CSE5343 Summer 1997
35
SHORTEST JOB FIRST SJF Schedule process with shortest CPU burst total time Nonpreemptive Queue ordered according to CPU burst time Minimum average waiting time
10 PR1 PR2 PR3 11 12 1 1.25 Waiting 10 Running PR1 PR2
SHORTEST REMAINING TIME SRT Preemptive version of SJF Schedules process with shortest burst time Queue ordered according to burst time Example Burst time assumed to be time to completion. Process allowed to continue if shortest time. :
11 12 1 1.25 Waiting Running
Process Turnaround Normalized Turnaround 1 2 1 2 3.15 3.15 3 2 8 Average 2.38 4.05
PR3
Process Turnaround Normalized Turnaround 1 3.25 1.625 2 1.65 1.65 3 .5 2 Average 1.8 1.758
CSE5343 Summer 1997
34
CSE5343 Summer 1997
36
PREDICTING BURST TIME How do you know CPU time? Burst time Programmer estimate Historical Similar processes Average of past bursts Suppose bursts are t1; t2 ; :::; t . What is t +1?
n n
PRIORITY Schedule process with highest priority lowest number Priorities 0 best .... n worst Preemptive and nonpreemptive versions Queue ordered according to priorities of processes Internal system External user Static Dynamic Aging - Longer process in system, higher priority. Avoids starvation.
T T T
n
+1 +1 +1
=t =
n
n
Pn
i
=1 ti=n
n n
n
= t + 1 , T where 0 1
Older times have the least weight
CSE5343 Summer 1997
37
CSE5343 Summer 1997
39
PRIORITY EXAMPLE Assume priorities are 1,3,2 and preemption:
10 PR1 PR2 PR3 11 12 1 1.25 Waiting Running
IMPACT OF VARYING TIME SLICES This table seems to indicate you want quantum as small as possible. Is this really true? RR 1 CPU Util 100 Throughput .92 Ave Turn 2.63 Ave Wait 1.55 Ave Norm 5.3 RR 1 100 .92 2.38 1.3 3.84 RR .5 RR .25 100 100 .92 .92 2.133 1.967 1.05 .883 2.592 1.925
Process Turnaround Normalized Turnaround 1 2 1 2 3.15 3.15 3 2 8 Average 1.967 1.925
CSE5343 Summer 1997
38
CSE5343 Summer 1997
40
ROUND-ROBIN Preemptive version of FCFS Queue ordered according to time process arrives Each process allocated a time slice quantum Popular in time-sharing systems Example:
10 PR1 PR2 PR3 11 12 1 1.25 Waiting Running
MULTILEVEL QUEUE Multiple queues for scheduling Priority among queues
Process Turnaround Normalized Turnaround 1 3.25 1.625 2 2.15 2.15 3 1 4 Average 2.133 2.592
CSE5343 Summer 1997
41
CSE5343 Summer 1997
43
MULTILEVEL FEEDBACK QUEUE Multilevel queues Processes move among the queues
DIRECT COMMUNICATION Messages Send Receive Sender needs to know receivers address Receiver may name process symmetric or receive from any process asymmetric Producer-Consumer example on p110 and p119 One link per process pair
CSE5343 Summer 1997
42
CSE5343 Summer 1997
44
INTERPROCESS COMMUNICATION IPC Techniques through which processes communicate "Link" needed between processes Implementation issues Link for more than two processes? Link capacity? Variable vs Fixed size Bi-directional, direct vs indirect Copy vs reference Security Shared memory
INDIRECT COMMUNICATION Mailbox Port Create Send Receive Destroy "Link" is established if mailbox is available More than two processes can access the same mailbox More than one link can exist per process pair Every process can be the receiver, but only one at a time Garbage collection can be a problem
Send Receive
CSE5343 Summer 1997
45
CSE5343 Summer 1997
47
BUFFERING Zero capacity: Synchronous message transfer Bounded capacity: Bu er has limited capacity, therefore, sender has to wait when the bu er is full Unbounded capacity: Bu er is unlimited Asynchronous communication: sender never waits for acknowledge Synchronous communication: sender waits for reply then continues
REMOTE PROCEDURE CALL RPC Abstract the basic procedure call mechanism for use between processes and processors Usually built on top of basic IPC structures Synchronous
CSE5343 Summer 1997
46
CSE5343 Summer 1997
48
PIPE Serial unidirectional ow of bytes between processes Create Write Read Access like le Two way communication requires two pipes Named and unnamed
Read
COMMUNICATION ERROR CONDITIONS If receiver is dead, communication sender informed With automatic bu ering sender will not be blocked if no acknowledgment OS detects a lost message and resends
Write
CSE5343 Summer 1997
49
CSE5343 Summer 1997
51
PROCESS SYNCHRONIZATION Problem is to synchronize execution of processes Can be solved by some IPC solutions Examples: Bounded-Bu er variation of ProducerConsumer ; Dining Philosophers Semaphores Monitors
CS ALTERNATIVES Two Processes TURN - No progress Fig 6.2 p167 FLAG - No progress; No mutual exclusion Fig 6.3 p168 Modi cation to algorithm by changing assignment and while is no mutex FLAG & TURN - Correct Fig 6.4 p169 Multiple Processes Bakery Algorithm
Fig 6.5 p171 Could have two get same number Still problems: Programmer controlled, Busy Wait,
Overhead Synchronization Hardware
Disable interrupts
CSE5343 Summer 1997
50
CSE5343 Summer 1997
52
CRITICAL SECTION PROBLEM Determine synchronization method to enforce correct interleaving of programs using shared data Critical Section: Section of code where data shared with another process is located Example: Suppose two programs each update x:=x+1 where x has starting value of v. What is value for x? v+1 or v+2 Fig 6.1 p166 How to allow only one process in CS at a time? NO mutual blocking Speeds unknown CS nite time Requirements Mutual Exclusion Must make progress Delayed only nite time
Test-and-Set Machine must support Atomic instruction Fig 6.6 p173 - does not necessarily satisfy bounded waiting
CSE5343 Summer 1997
53
CSE5343 Summer 1997
55
SEMAPHORES Nonnegative integer, S, variable accessed only through atomic operations Wait - Decrement S if possible Entry, P Signal - Increment S Exit, V Initial value of S is the number of processes in CS at same time More general than previous approaches. Solves mutual exclusion, but also other synchronization problems Busy-Wait solution p. 176 Non Busy-Wait solution p. 178 Binary semaphore 0,1 . Mutual Exclusion. General Counting semaphore 0,1,2,3,...,n . Abstract Data Type Initialize, Wait, Signal Disadvantages: Hard to write, Each program must use them, Programmer controlled
DINING PHILOSOPHERS PROBLEM Philosophers think and eat each of nite time n philosophers, n plates, n chairs, 1 table, n bowls of rice, n chopsticks Chopsticks placed in between philosophers When philosopher is hungry he she picks up chopsticks on either side Philosopher needs two chopsticks to eat, but only one may be available Object: No philosopher starves Fig 6.14 p184 Semaphore solution uses an array of semaphores one per chopstick Problem with this solution is that a deadlock may occur Fig 6.15 p185 This solution can have deadlocks.
CSE5343 Summer 1997
54
CSE5343 Summer 1997
56
BOUNDED-BUFFER PROBLEM Fixed size bu er producer-consumer problem Both producer and consumer may have to wait Imagine circular queue with n bu ers Semaphore solution uses empty, mutex, and full semaphores Fig 6.10 and 6.11 p182 Mutex initialized to 1; Binary semaphore Empty initialized to n; Counting semaphore Free Full initialized to 0; Counting semaphore Full
MONITORS Control access to shared data Only way to access data is through monitor procedure As with semaphores can solve CS problem but also more general problems Abstract Data Type Only one process at time allowed in monitor Data in monitor is hidden from processes Processes only know entry procedure protocol Monitor consists of: Data, Entry procedures, Initialization Code, and possibly Nonentry procedures Monitor data only accessed by internal procedures
CSE5343 Summer 1997
57
CSE5343 Summer 1997
59
MONITOR EXECUTION Sequence: Process calls Entry Procedure E Place request in queue for E Wait Enter monitor E Exit monitor Return to original code How use for CS? Simple solution - put each CS in monitor as procedure To enter CS or to exit need a monitor procedure
MONITOR SOLUTION FOR BOUNDED BUFFER PROBLEM Data: queue : array 1..n free : integer full : integer rear : integer head : integer Condition Variables: full, empty Initialization: free := n; full := 0; rear := 1; head := 1 Entry Procedures: Producer : empty.wait; full.signal Consumer : full.wait; empty.signal Fig 6.10 and 6.11 p182 Other Procedures: None
CSE5343 Summer 1997
58
CSE5343 Summer 1997
60
MONITOR EXECUTION WITH CONDITIONS Once in monitor may discover a condition that causes the process to wait Since only one process is active in the monitor at time, this process can not wait inside the monitor Solution: Process issues Wait operation to suspend itself until some condition is satis ed Another process enters monitor, satis es the condition, and wakeups the waiting process using Signal Wait Signal operations performed on condition variables Process that issued the Signal waits until signaled process completes Priority of entry into monitor: Signaled process, Process that did signal operation, Outside process Dining Philosopher Solution is deadlock free
SIMILAR CONSTRUCTS Abstract Data Types Objects OOPL, OODB Monitors Encapsulation Hiding Entry Procedures
CSE5343 Summer 1997
61
CSE5343 Summer 1997
63
DEADLOCKS Processes some can not make progress Cyclical wait on resource among a set of processes Necessary Conditions But not su cient : Mutual exclusion Hold and wait No preemption Circular wait Deadlock issues Prevention Avoidance Detection Recovery Applies only to serially reusable resources as opposed to consumable resources Fig 6.15 p185 may cause a deadlock. How to avoid????
DEADLOCK PREVENTION Remove any one of the 4 conditions Very pessimistic Deny Mutual Exclusion: share resources not always feasible Deny Hold and Wait: Allocate all resources at one time Pessimistic Deny No Preemption: Remove a resource from a process If resource can not be granted immediately, then take away all held resources Deny Circular Wait: Group resources into ordered classes Process must request resources in order of the groups Must request all of one type at one time
CSE5343 Summer 1997
62
CSE5343 Summer 1997
64
RESOURCE ALLOCATION GRAPH Directed bipartite graph to pictorially show resource allocation in system Vertices: Resources Box - Dots in box indicate number Processes Circle Fig 7.1 p220 Arcs:
AVOIDANCE Detects and prevents unsafe states. Safe State: System can allocate resources to all processes in some order such that they will nish Unsafe State: No such sequence for allocating processes exists Unsafe state is not a deadlock Deadlock is an unsafe state However, unsafe is not necessarily deadlock Less pessimistic than prevention Requires knowledge of resources and processes Dynamically examine resource allocation graph to ensure acyclic Avoidance algorithms do not allow any unsafe state
hP ,R i - Process P is waiting for resource
i j i
R P
j
hR ,P i - Resource R is allocated to process
j i j i
If resource allocation graph is acyclic then no deadlock Note, a cycle may or may not indicate deadlock
CSE5343 Summer 1997
65
CSE5343 Summer 1997
67
AVOIDANCE EXAMPLE Request Pr1 Alloc Max 6 Pr2 Alloc Max 6 0 6 0 6 Pr1 2 2 4 0 6 Pr2 3 2 4 3 3 Pr1 2 4 2 3 3 Pr2 1 4 2 4 2 No Avail 9 7 Safe 4 Safe 2 Safe 1 Unsafe
Algorithm "simulates" making request then determines if safe state
CSE5343 Summer 1997
66
CSE5343 Summer 1997
68
BANKER'S ALGORITHM Another algorithm in book on p 229 if only 1 occurrence of each resource Algorithm: 1. Declare in advance maximum resources 2. If resource is allocated and all processes can complete 3. then allocate else don't allocate Data Structures n processes, m resources : Available 1..m - Number of available resources Max 1..n,1..m - Max demand per process Allocation 1..n,1..m - Number resources per process Need 1..n,1..m - Remaining needs of each process Work 1..m - Used to simulate available Finish 1..n - Used to determine if all processes can nish
DEADLOCK DETECTION & RECOVERY Optimistic Check for cycle in resource allocation graph When check? Periodically Process inactive for period of time Algorithm: 1. Build resource allocation graph 2. Simulate request allocation release actions of processes 3. If deadlock then pick process and abort
CSE5343 Summer 1997
69
CSE5343 Summer 1997
71
DEADLOCK DETECTION ALGORITHM Algorithm very similar to Banker's Data Structures n processes, m resources : Available 1..m - Number of available resources of each type Allocation 1..n,1..m - Number resources currently allocated to each process Request 1..n,1..m - Current request by each process Work 1..m - Used to simulate available Finish 1..n - Used to determine if all processes can nish III. MEMORY MANAGEMENT Addressing Memory Allocation Release Free Space
CSE5343 Summer 1997
70
CSE5343 Summer 1997
72
DEADLOCK RECOVERY Abort all deadlocked processes Abort one process at a time until deadlock cycle eliminated Abortion kills process and removes revokes all resources Requires some rollback of changed data Victim selection: Find smallest number of processes needed to kill Process making request Minimum number of allocated resources Shortest execution time Lowest priority Ease of revocation Most recent checkpoint
ADDRESS TRANSLATION Bind physical address to logical address Compile - Absolute Load - Relocatable Execution - Dynamic Fig 8.1 p251
CSE5343 Summer 1997
73
CSE5343 Summer 1997
75
STATIC ADDRESS TRANSLATION Faster execution Only translate once More error checking before running
SWAPPING and OVERLAYS Programs in memory may be swapped out to disk Fig 8.4 p257 If not enough memory sections of code may overlay others Fig 8.2 p254 Overlays aren't used much since virtual memory
CSE5343 Summer 1997
74
CSE5343 Summer 1997
76
DYNAMIC ADDRESS TRANSLATION DAT hardware Relocation Registers Cache frequent addresses Fig 8.3 p256
SIMPLEST MEMORY ALLOCATION No address translation Logical address = Physical address Early batch
CSE5343 Summer 1997
77
CSE5343 Summer 1997
79
CONTIGUOUS ALLOCATION Simple Batch Fixed Partitions Variable Partitions
FIXED PARTITIONS Multiple partitions of xed size Di erent partitions may be of di erent sizes Which partition should a program be loaded into?
CSE5343 Summer 1997
78
CSE5343 Summer 1997
80
SIMPLE BATCH ALLOCATION Single Partition Allocation OS in low memory User in high memory Static or dynamic address translation Fig 8.5 p260 Fig 8.6 p261
VARIABLE PARTITIONS Multiple partitions of variable size Size determined when program loaded Allocate contiguous block large enough for program Fig 8.8 p263 Free Space linked list Coalesce free space Boundary Tag approach
CSE5343 Summer 1997
81
CSE5343 Summer 1997
83
FRAGMENTATION Unusable memory Can not be allocated Internal Fixed - Allocated more space to a program than it needs External Variable - Space not allocated but not usable
VARIABLE PARTITIONS ALLOCATION First Fit Best Fit Increasing Size Worst Fit Decreasing Size Either may perform best under di erent circumstance Reduce fragmentation - worst EXAMPLE Memory 1000 words : A Get 400; B get 300: A Free 400; C get 300; D get 350 Worst Fit - D blocked Best Fit - Works
CSE5343 Summer 1997
82
CSE5343 Summer 1997
84
COMPACTION Solution to external fragmentation Put all free space together in one big block Fig 8.10 p266 Problems - e ciency, frequency, impact on executing programs Quiesce system Dynamic compaction
NONCONTIGUOUS ALLOCATIONS Paging Segmentation Combination
CSE5343 Summer 1997
85
CSE5343 Summer 1997
87
PAGING Fixed size partitions of program memory Memory divided into frames Long term scheduler loads pages into frames Program not contiguous in memory Page Table - For each page in process has associated frame number Pointer to page table in PCB Dynamic address translation hardware Fragmentation - 1 2 page Internal; Usually no External Page Size - 256-4K; Block size Small - Less fragmentation, Less I O time Large - Fewer I Os. Smaller page table Multilevel Paging Fig 8.12 p268, Fig 8.13 p269, Fig 8.18 p278
PAGE MAPPING METHODS Direct Basic Page Table Register Two memory accesses Fig 8.12 p268 Associative Associative registers for page tables Expensive Faster Combined Both page table and associative registers Translation Lookaside Bu er TLB TLB has active pages only Go to page table only if page not in TLB Fig 8.16 p 274
CSE5343 Summer 1997
86
CSE5343 Summer 1997
88
PAGE TABLE Size Constant - Maximum Size Variable - Program Size Dynamic - Change during execution Valid Bit Read-Write protection
PHYSICAL MEMORY MANAGEMENT WITH PAGING Free frames in bit map Frame Table Inverted page table - one entry per frame; less space than page tables Fig 8.20 p281
CSE5343 Summer 1997
89
CSE5343 Summer 1997
91
PAGING ADVANTAGES DISADVANTAGES ADVANTAGES noncontiguous eliminate external fragmentation simple free space management no compaction sharing Fig 8.21 p262 protection DISADVANTAGES how to pick good page size internal fragmentation more overhead - page table, mapping hardware program size and structure has no relationship to page size
COMBINED Use segmentation and paging Segment composed of pages Memory divided into frames item Fig 8.26 p291
CSE5343 Summer 1997
90
CSE5343 Summer 1997
92
SEGMENTATION Divide program into logical units based on structure Segment table Fig 8.23 p285 Segment table register More logical than pages No fragmentation Address translation must verify that the address is valid Fig 8.24 p286 Protection and sharing easy IV. VIRTUAL MEMORY What is Virtual Memory? Page Replacement Algorithms
CSE5343 Summer 1997
93
CSE5343 Summer 1997
95
WHAT IS VIRTUAL MEMORY? All schemes so far: load entire program into memory prior to execution program size is limited by memory size need room in memory for all programs Virtual Memory: Process need not all be in memory Process may be larger or smaller than main memory No need for overlays Secondary storage required to store VM Uses paging May use segmentation Need to modify page table structure
VIRTUAL MEMORY ISSUES Fetching When and How much Placement Replacement Global or Local Page Fault - Accessing a page not in memory
valid invalid memory or not pointer to secondary storage modify bit
Fig 9.1 p303, Fig 9.2 p304, Fig 9.3 p305
CSE5343 Summer 1997
94
CSE5343 Summer 1997
96
WHY DOES VIRTUAL MEMORY WORK? Access only subset of entire program Locality of reference: High probability that next instruction to be executed is near" current one. Temporal - Time; Rereference likely in near future Spatial - Space; Highly likely that nearby locations will be referenced Small number of pages needed to execute program Memory Hierarchy: Cache - TLB - Cost high, very small, fast Primary - MM Secondary Storage - VM - Cost low, very large, slow
PAGE FAULT Page table indicates page not in memory Block process Find frame in memory to put new page May need to replace one already lled Load page from secondary storage to frame Update page table Unblock process Fig 9.4 p307 What prevents multiple page faults on same operation? Suppose there is a page fault getting to page table?
CSE5343 Summer 1997
97
CSE5343 Summer 1997
99
FRAME TABLE For every frame in memory: Free or not free Clean or dirty Usage statistics Process using this frame Lock in memory
REPLACEMENT When no frames are available to place a needed page, which of the occupied frames should be replaced? Fig 9.5 p313 Replace page not needed in future Use free frame if one available If no free frame use page replacement algorithm to select a victim frame If replaced page has been modi ed it must be written out to secondary storage swap out Use locality to help predict At extreme may decide to swap out a complete process Use free frame if one available
CSE5343 Summer 1997
98
CSE5343 Summer 1997
100
FETCH PAGE Demand Paging - at page fault Anticipate Predict - guess future usage Prefetch - prior to execution
REPLACEMENT OBJECTIVES Reduce future page faults and swaps out Measures: Total page faults Swap in - F Swaps out - S Failure Frequency Function f = F number of references
CSE5343 Summer 1997
101
CSE5343 Summer 1997
103
PAGE REPLACEMENT ALGORITHMS Random Optimal FIFO LRU Least Recently Used LRU Approximations - Clock LFU Least Frequently Used Working Set Reference String - String of memory references Time 1 2 3 4 5 6 7 8 9 10 11 12 Page 4 3 2 1 4 3 5 4 3 2 1 5
LRU Replace page not used for longest time Extreme overhead - Must maintain priority every reference No anomaly Time 1 2 3 4 5 6 7 8 9 10 Page 4 3 2 1 4 3 5 4 3 2 Memory 4 3 2 1 4 3 5 4 3 2 4 3 2 1 4 3 5 4 3 4 3 2 1 4 3 5 4 F=10; S7; f=9=12=83 11 1 1 2 3 12 5 5 1 2
CSE5343 Summer 1997
102
CSE5343 Summer 1997
104
FIFO Replace page in memory the longest Belady's Anomaly - Some page reference patterns cause more page faults when number of page frames increases Time 1 2 3 4 5 6 7 8 9 10 Page 4 3 2 1 4 3 5 4 3 2 Memory 4 3 2 1 4 3 5 5 5 2 4 3 2 1 4 3 3 3 5 4 3 2 1 4 4 4 3 F=9; S6; f=9=12=75 Time 1 2 3 Page 4 3 2 Memory 4 3 2 4 3 4 4 5 6 7 8 9 10 1 4 3 5 4 3 2 1 1 1 5 4 3 2 2 2 2 1 5 4 3 3 3 3 2 1 5 4 4 4 4 3 2 1 5 F=10; S6; f=10=12=83 11 1 1 2 5 11 1 1 2 3 4 12 5 1 2 5 12 5 5 1 2 3
CLOCK Second Chance - LRU Approximation Variation on FIFO Little overhead Need reference bit per frame Pointer moves through memory pointing to target frame to replace When a frame is needed, the pointer advances until it nds a frame with a 0 reference bit When a nonzero reference bit is found, the bit is changed to 0 and the pointer advances Whan a frame is referenced, the bit is set to 1 Fig 9.13 p324
CSE5343 Summer 1997
105
CSE5343 Summer 1997
107
LFU How intensive use has been Replace page used the least frequently Hot Set No Anomaly Time 1 2 3 4 5 6 7 8 9 10 Page 4 3 2 1 4 3 5 4 3 2 Memory 4 3 2 1 4 3 5 4 3 3 4 3 2 1 4 3 5 4 4 4 3 2 1 4 3 5 2 F=10; S7; f=9=12=83 11 1 3 4 1 12 5 3 4 5
THRASHING More time paging than execution Excessing swapping Usually indicates processes have too few pages Solution - Suspend process and swap out pages to disk If local replacement - increase number of frames per process May help to lock pages in memory
CSE5343 Summer 1997
106
CSE5343 Summer 1997
108
WORKING SET
V. I O And File Concepts What is a File? File Systems Other File Topics Secondary Storage Management Disk Scheduling Related Topics
4 - Working Set Window, Most recent 4 page
references WS t = f page j page one of the last 4 references g OS must allocate enough frames to process to store WS A new process can be initiated if there are enough page frames WS replacement - keep working set in memory; When a process needs a new page in memory, replace one from its own WS Local Page Replacement as opposed to global Problems: Expensive, WS size, initial WS, vary size per process, wastes frames fragmentation
CSE5343 Summer 1997
109
CSE5343 Summer 1997
111
TERMS Record - Logical unit Block Physical record - Physical unit of I O Sector - Physical division of a track Fixed length = Records Variable length ; : Blocks Blocking factor Spanned Unspanned
98
LOGICAL vs PHYSICAL Logical: Attribute Field Record File Database Physical: Bit Byte Word Block Sector Track Cylinder Device
CSE5343 Summer 1997
110
CSE5343 Summer 1997
112
ADDRESSES Logical: Key - Field s in record whose value s is are unique across all records Relative Record Block - Relative to start of le Ine cient Physical: Device Cylinder Track Sector Block O set E cient Di cult to program Pinned records Pointer on disk could be either physical or logical address Dangling pointer
WHAT IS A FILE? Named collection of persistent data. File Metadata: Name, Type organization , Location, Size, Protection, Time of last use update,create , Owner ID
CSE5343 Summer 1997
113
CSE5343 Summer 1997
115
FILE MANAGEMENT SYSTEM FMS Software that creates and provides access to data les on secondary storage Removes detail and speci cs concerning I O from program Statistics Tools Coordinate access - Simple locking Bu er Pool management Directory Placement on disk Organizations - Storage structure Access routines - Software to read write data in les Protection
BUFFER POOL Area in memory set aside to read write from to disk May contain many bu ers Overlap lling of bu er and reading of another Double Triple Bu ering Anticipatory I O
CSE5343 Summer 1997
114
CSE5343 Summer 1997
116
LAYERED FILE SYSTEM User logical to Device physical Generic I O controllers - facilitate uniform view and access Device driver Interrupt handlers
BUFFER POOL MANAGEMENT More important for databases than les DBMS may provide own bu er pool and management Issues: Size - Number of blocks Fetching - When to bring records into bu er
Minimize I Os Anticipatory I O - Anticipate future access based on past. Prefetch blocks. When need will already be there. Replacement - When to replace records
May depend on access. Keep hot set in bu er Fix some bu ers in memory
CSE5343 Summer 1997
117
CSE5343 Summer 1997
119
DIRECTORY Index to les Indicates organization Indicates by le where stored Describes each extent Contains statistics about le Types: Single-level Two-level Tree Acyclic Graph General Graph
TWO-LEVEL DIRECTORY Lowest level contains le names Root groups les Root usually by userID Allows duplicates across users
CSE5343 Summer 1997
118
CSE5343 Summer 1997
120
SINGLE-LEVEL DIRECTORY Simple list of le names Linear Search Binary Search
TREE DIRECTORY Node is directory or le Can view directory as just another le which is a group of les directories Path name Current directory Default starting directory MS-DOS
CSE5343 Summer 1997
121
CSE5343 Summer 1997
123
ACYCLIC GRAPH DIRECTORY Generalization of tree Allows sharing of directories or les May not have a path name Useage count UNIX
PLACEMENT ON DISK Allocation: Static vs Dynamic Extent - Unit of allocation; group of blocks Contiguous - All blocks in contiguous disk locations Linked - Each block in le points to next Indexed - Special index blocks point to extents Clustering - Placing data accessed together close together on disk Partitioning - Divide le and store sections separately. Horizontal and Vertical.
CSE5343 Summer 1997
122
CSE5343 Summer 1997
124
GENERAL GRAPH DIRECTORY Allows cycles in graph Avoid in nite loop in searching for le How delete?
MULTIPLE SMALLER DISKS Disk access can become a bottleneck Increasing size and even speed does not always solve problem Channel limits the amount of data to be transfeRred to memory Solution: Many small disks RAID - Redundant Array of Inexpensive Disks Striping - Placing data across all disks in interleaved fashion Striping Granularity - Size of striped unit
CSE5343 Summer 1997
125
CSE5343 Summer 1997
127
ORGANIZATION vs ACCESS Organization: Manner in which blocks are arranged on secondary storage May or may not dictate access techniques allowed Access: Software that provides logical access based on speci c organization Choose based on access needs of applications Some organizations are di cult to maintain
ORGANIZATION Sequential Ordered, sorted - Records stored in logical order of key. Access: sequential, relative, binary search. Heap Unordered - Records added to end. Access: direct. B Tree - Multiway balanced search tree. Access: sequential, random, skip sequential. Hashing - Store and access record based on address determined when key is hashed. Access random. Accesses provided depend on FMS
CSE5343 Summer 1997
126
CSE5343 Summer 1997
128
ACCESS Sequential - Retrieve records in logical order Random - Retrieve record based on key Direct - Retrieve record based on physical address Relative - Retrieve record based on relative position in le Binary Search - Randomly retrieve record doing binary search Skip Sequential - Position at speci c record in le then read sequentially Access Lists
PROTECTION
Access Groups owner,group,all others
CSE5343 Summer 1997
129
CSE5343 Summer 1997
131
OPEN and CLOSE OPEN: Prepare le for access Build tables Allocate bu ers Request lock Prefetch CLOSE: Indicate program is nished with le Free tables Flush bu ers Free bu ers Unlock
FILE MOUNTING Must open a le before access Must mount a le system before use Allows many di erent le systems Could have di erent le system per device May mount a directory. This integrates the directory into the directory hierarchy.
CSE5343 Summer 1997
130
CSE5343 Summer 1997
132
OPEN FILE TABLE Directory information Pointer to bu ers Disk address Read write location File Descriptor
DISK SCHEDULING How to scheduling I O requests to same disk No starvation Reduce waiting time mean, per process Increase throughput reduce seek time Reduce response time mean, per process Types: FCFS SSTF SCAN C-SCAN
CSE5343 Summer 1997
133
CSE5343 Summer 1997
135
FCFS Peform requests in order received Extreme overhead and head movement
SCAN Eliminates discrimination of SSTF Reduced variance No starvation OUT-IN-OUT-IN... Pick up in both directions Elevator No wasted head movement LOOK only go as far as needed in that direction
CSE5343 Summer 1997
134
CSE5343 Summer 1997
136
SSTF Perform request with the minimum seek distance Extreme variance in response time Starvation
C-SCAN Less variance in wait time than SCAN Only picks up data in one direction Additional head movement C-LOOK - if no more requests return to start
CSE5343 Summer 1997
137
CSE5343 Summer 1997
139
FREE SPACE MANAGEMENT Free File Free List linked list Bit Maps Index Backup Roll Forward Roll Back Log Checkpoint
RECOVERY
CSE5343 Summer 1997
138
CSE5343 Summer 1997
140
BAD" BLOCK MANAGEMENT Bad" File Bit map
VI. ADDITIONAL TOPICS Protection Security Distributed Operating Systems
CSE5343 Summer 1997
141
CSE5343 Summer 1997
143
PROTECTION vs SECURITY Security External Physical - Disaster, Intruders Operational - Policies, Procedures Measure of the con dence that integrity is preserved Broader than protection Must identify potential threats Viruses Protection Controls in HW and OS Internal How to provide controlled access to programs and data May depend on proper authentication Defend against misuse Multics
RINGS
Levels of domains D0 ; :::; D Fig 13.2 p 436
n
Object le, process belongs to domain Can access segments in higher rings Priviledges D Priviledges D where ij.
i j
Access rights are given at fork same or subset of parent
CSE5343 Summer 1997
142
CSE5343 Summer 1997
144
TYPES OF PROTECTION Login Instruction Supervisor User mode Storage Bounds registers, Length, Page Segment tables, Access rights File Access rights, Password Access Rights - De ne access subject has to object Domain - Collection of access rights associated with object object-name,rights-set Access Matrix - Model of protection Fig 13.1 p 438
MOVEMENT BETWEEN RINGS To allow controlled movement between domains, each domain has the following: Access bracket - Integers b1; b2 such that b1 b2. Range of ring numbers which can enter this procedure. Otherwise OS is called and it determines if call is allowed. Limit - Integer b3 such that b3 b2; Some calls from rings beyond b3 are allowed but only to named entry points. List of gates - Entry points to segment procedure ;
CSE5343 Summer 1997
145
CSE5343 Summer 1997
147
CAPABILITIES Protected name for object which indicates access rights To access object, process must have capability to do so C-List - List of capabilities granted to process When object is created, initial capability for it including all access rights is created. Subject possessing capability may pass it to another subject and possibly reduce rights.
DISTRIBUTED OPERATING SYSTEMS Operating systems at multiple sites cooperate to create one global operating system. Local and remote resources accessed in same manner. Each site may only have a piece of an operating system. Data vs Computation vs Process migration
CSE5343 Summer 1997
146
CSE5343 Summer 1997
148
NETWORK OPERATING SYSTEMS Each site maintains own autonomous operating system Users may access data at remote sites telnet - Connection to port at remote site. Creates a transparent bidirectional link. ftp - Transfer les, Anonymous login, get put ls cd
DISTRIBUTED FILE SYSTEM NAMING Location transparency - Name does not reveal location. Location independence - Name is not linked to physical location. UNIX United - Single naming structure for entire system. Root directory of component systems now named. Virtual root connecting all systems. SUN NFS - Based on client server relationship. File systems at di erent machines are autonomous. To access a remote machine, a client must mount the le system for the desired server.
CSE5343 Summer 1997
149
DISTRIBUTED DEADLOCK Deadlock may be local or across machines Prevention Based on timestamp Wait-die - Nonpreemptive Older process waits on younger one. Younger process aborts itself instead of waiting. Wound-wait - Preemptive Older process preempts process that currently has the resource. Newer process waits. Detection Global and local wait-for graph Hierarchical detection Distributed detection Centralized Fig 18.6 p 593
CSE5343 Summer 1997
150
DISTRIBUTED CONCURRENCY CONTROL Locking - central site or site associated with data Timestamping Replicated data - single coordinator, majority, primary copy