intro

Document Sample
intro Powered By Docstoc
					          CPS110:
Intro to (Operating) Systems

       Author: Landon Cox
       Instructor: Jeff Chase
       August 24, 2009
Syllabus: prerequisites
 CPS 100
   Basic data structures and memory layout
   Allocating memory on the stack versus from the heap
 CPS 104
     Basic computer architecture, ISAs
     Registers: stack pointer, PC, general-purpose
     Virtual memory translation
     Page tables
     TLB, caching
 Other: C/C++
Syllabus: lectures and textbook
 Lecture notes on the web (125 pages)
   Exams based on content of lectures
 Textbooks
     Only suggested
     “Modern Operating Systems”
     Easy to find on-line
     New book: Saltzer and Kaashoek
Syllabus: discussion sections
 One section, starting this week
    F 2:50 – 4:05
 Teaching Assistant
    Jie Xiao (“Jennifer”)
    (jiexiao@cs.duke.edu)
 Undergraduate Teaching Assistants
    Matt Jacobson
Syllabus: homework problems
 Posted on web on Monday of each week
 Should be done before discussion section
 Not graded, but count toward participation
Syllabus: projects
 Where you will learn the most
 4 projects
      0: very simple intro to C++
      1: build a user-level concurrency package (thread library)
      2: build a virtual memory manager
      3: hack into a vulnerable system
 Projects aren’t long, but are difficult (and “exacting”)
    Only 100-1,000 lines/code, but could be many hours
    Everything is in C++
 Project 0 will be posted in a few days
Syllabus: project environment
 Linux/GNU environment
 You need a CS account
   Don’t have one? Send me e-mail.
   chase@cs.duke.edu by August 28
 Login to linux.cs.duke.edu to submit
Syllabus: project groups
 All projects done in groups of 2 or 3
 Email groups to chase@cs.duke.edu
    By Friday (August 28)
    {name, NetID, CS login} for each member
 Group members may be asked to rate each other
 Procedure for firing, quitting in syllabus
Syllabus: project auto-grading
 All projects are auto-graded
   Immediate feedback
   Use submit110 script on cs machines
 One submission/group/day gets feedback
   Can’t use to debug your project
   + three bonus submissions/group/project
 Any group member’s submission counts
More on the auto-grader
 Very narrow feedback: correct or incorrect
    Doesn’t say what is wrong
       Follow specifications carefully
    Still have to write a test suite (except P0)
    Don’t rely on auto-grader feedback alone
 To get more useful feedback
    Come talk to us!
    We will provide many office hours every week
    (double office hours week before a deadline)
Syllabus: project timelines
 Due at 6pm, accepted until 11:59:59pm
   Auto-grader clock is the one that counts
   Last submission to auto-grader is final
 3 late days/group/semester
   Intended for unexpected problems
 No extensions
 Start early!
Syllabus: project collaboration
 Ok, among groups
    C++ syntax, course concepts
    “What does this part of the handout mean?”
 Not ok, among groups
    Design/writing of another’s program
    Includes prior class solutions
    “How do I do this part of the handout?”
 We use automated similarity-detection software
    Just changing the variable names won’t save you
    http://theory.stanford.edu/~aiken/moss/
 If in doubt, ask me
Syllabus: grades, exams
 Projects: 35%
 Midterm: 30%
   Monday, February 23
 Final: 30%
   Wednesday, April 29, 7-10pm
 Participation: 5%
Projects and exams
 The two are not independent
   Familiarity with projects is critical to doing well on exams
 I like to ask questions about projects on exams
    “Extend Project X to include this functionality”
 Know your project!
    You can assign roles to different people
    But each member must understand all aspects
Syllabus: getting help
 Newsgroup
   http://courses.duke.edu
 Office hours
   With me: Tu, 1:00 – 3:00, etc…
   With Jie: Tu, 1:30 – 2:30, Th 4:00-5:00
   UTAs…
 Post to the newsgroup
   And you may also cc: us
Questions about the course?
Goals for CPS 110
 First part: demystify the operating system
   How does my computer start running?
   How does a program load into memory?
 Second part: demystify the Internet
   How does my email know where to go?
   Why is Google so fast?
Duke curriculum: one view

  CPS 1,6,100,108
                        Applications
                                 Ideas 
                    high-level programming languages
                     compiling, reading programs off disk,
                     getting program into memory, reading
     CPS 110
  What’s missing?   keyboard, starting the computer, saving
                           files, filenames, networking



     CPS 104
                           Hardware
                      Assembly language program 
                                  gates
“Systems”: A Bigger Picture
 Programmable platforms to enable
  sharing of data and resources
 “Textbook” example: operating systems
   “Cloud” clusters
   Etc…?
Systems as Art




   http://www.sammlung.daimler.com/sculpt/potsdamerplatz/potsd_tinguely500.jpg
Thinking about interfaces
 Consider the Java language and its key word “interface”
 What is a Java object?
    List of methods and collection of internal state
 What is a Java interface?
    Set of methods associated with an object that a programmer can call
 What do those methods do?
    Invoke code (let the object do work on the caller’s behalf)
    Modify the object’s public/private state
 Why are interfaces useful?
    They provide an “abstraction” or simplification
    Callers don’t have to know an object exact type or implementation
OS terminology
   Key terms: interface, resource (cpu, mem, etc), abstraction, virtual
 What is an interface?
      An interface is a set of primitives or operations
      Interfaces provide access to resources and/or data
 What do we mean by abstraction?
      How resources are presented to a client
      Can think of as an illusion that makes resources easier to program
 What does it mean to virtualize something?
      Provides an abstraction (simple way to manipulate resources)
      (mostly) disallow direct access to reality/resources
Interface and abstraction
Virtualization?
Standards, wrappers, adapters
What is an operating system?
 Program that runs on CPU, (mostly) like any other

        Applications
                                  “Virtual machine”
                                       Interface
                OS
                                  “Physical machine”
                                      Interface
          Hardware
 Virtual interface should be “easier” than physical
What is an operating system?

                        What interface does
     Applications       the OS present?
                      “Virtual machine”
                           Interface

          OS
                      “Physical machine”
                          Interface

      Hardware          What interface does
                        the hardware present?
                        Instruction set:
                        Load/store, mem, regs
Hardware-software stack
Applications              Virtual   System    Files,
               Threads
                          Memory     Calls   Sockets




                          OS
               Atomic      Page              Ether,
               Test/Set   Tables
                                    Traps
                                             80211
Hardware
OS vs user-level programs
    Familiar view                          Alternate view
       User program                                   OS


            OS                         User program         User program

How do programs start?              OS runs first, calls program
Tasks outside program? (net recv)   Programs run until they return control to OS
                                      (by themselves or forced by hardware)
How do prevent CPU hogging?
                                    Then OS calls another program

                       Who calls whom?
Functions of the OS
1. Illusionist
     Makes hardware seem “nicer” than it really is
   Examples?
       Programs seem to have their own CPU
       AFS: single, unified file system
       Name data with human-readable names
       Directories
       Packets get lost; OS makes net look reliable
       Disk is slow; OS makes it look fast via caching
Functions of the OS
1. Illusionist
     Makes hardware seem nicer than it really is
2. Government
     Divides hardware resources among competing programs
   What hardware resources does the OS manage?
       Processor
       Memory
       Network
       Disk
Functions of the OS
1. Illusionist
    Makes hardware seem nicer than it really is
2. Government
    Divides hardware resources among competing programs
    Taxes programs (OS needs CPU, memory to run)
    Taken for granted when it works, cursed when it breaks
Why study operating systems?
 Very few of you will ever write one …
1. Illusionist, govn functions appear in many domains
    Google provides the illusion of a single web server
    How should basketball seats be allocated?
2. Design principles
    Proper abstractions, caching, indirection
    Concurrency, naming, atomicity, authentication
    Protection, resource multiplexing (fairness)
 How does OS create the illusions we know/love?
Hints for designing systems
 What is a system?
   Components, interconnections
   Interfaces, environment
 Systems do something for their environs
   Export this behavior via an interface
 Cleanly divides the world in two
   Parts of the system + the environment
Systems from 10,000 feet

              Component




                           Component
          Component


          Component

                  System

       Environment aka “the client”
Why is designing systems hard?
1. Emergent properties
   Can’t predict all component interactions
   Millennium bridge
      Synchronized stepping leads to swaying
      Swaying leads to more forceful synchronized stepping
      Leads to more swaying …
2. Propagation of effects
3. Incommensurate scaling
4. Trade-offs
Why is designing systems hard?
1. Emergent properties
2. Propagation of effects
     Want a better ride so increase the tire size
     Need a larger trunk for the larger spare
     Need to move the back seat forward
     Need to make front seats thinner
     Leads to worse driver comfort than before
3. Incommensurate scaling
4. Trade-offs
Why is designing systems hard?
1. Emergent properties
2. Propagation of effects
3. Incommensurate scaling
     Consider the giant mouse
     Weight ~ size3 (volume)
     Bone strength ~ size2 (cross section area)
     An elephant sized mouse is not sustainable
4. Trade-offs
Why is designing systems hard?
1.   Emergent properties
2.   Propagation of effects
3.   Incommensurate scaling
4.   Trade-offs
        “Waterbed effect”
        Push on one end, and the other goes up
        Spam filters and smoke detectors
        False positives vs false negatives
Why is designing systems hard?
1.    Emergent properties
2.    Propagation of effects
3.    Incommensurate scaling
4.    Trade-offs
     In the immortal words of HT Kung
      “Systems hard. Must work harder.”
History of operating systems
 History dominated by two trends
    Hardware: better, cheaper, faster
    Software: sprawl
 Microsoft embodies tension between these trends
    MS gained 90% market share by running on cheap hw
    Supporting all that hardware complicates the OS
    (3rd-party drivers responsible for vast majority of crashes)
 How is Apple’s strategy different?
    Jobs chooses the hardware you will run
    HW-to-app control reduces complexity, choice, discount
First phase: single operator
 One goal: make it work
    Interactive (user has entire machine to herself)
    Users sign up, get room for two hours at a time
 “OS” is really just a library linked into your program




          What is wrong with this timeline?
          CPU utilization is awful
          Since CPUs were expensive, this mattered
Second phase: batch processing
 Goal: improve CPU, I/O utilization
    Machine is no longer interactive
    Users submit program (stack of cards) to queue
    One job at a time, CPU idle during I/O, I/O idle during CPU




 OS is a batch monitor + library of services
    Loads program, runs program, prints results
    Loads next program …
Second phase: batch processing
 Goal: improve CPU, I/O utilization
    Machine is no longer interactive
    Users submit program (stack of cards) to queue
    One job at a time, CPU idle during I/O, I/O idle during CPU




 What key OS function starts to matter now?
    Protection: programs must not corrupt monitor
    Programs must relinquish CPU to monitor
Second phase: batch processing
 Goal: improve CPU, I/O utilization
    Machine is no longer interactive
    Users submit program (stack of cards) to queue
    One job at a time, CPU idle during I/O, I/O idle during CPU




 Why wasn’t protection an issue before?
    No batch monitor to corrupt
    Person in lab coat took CPU back from program
Third phase: multi-program batch
 Goal: overlap CPU, I/O
     When one job is reading from disk, run another job on CPU
     Use DMA + interrupts to allow background I/O
     DMA: devices write to program memory
     Interrupts: devices can tell CPU the I/O is done


      Job 1

      Job 2
Third phase: multi-program batch
 Goal: overlap CPU, I/O
 What are the OS’s new responsibilities?
    Switch between processes
    Manage multiple I/Os across devices
    Protect processes from each other

    Job 1

    Job 2
Fourth phase: time-sharing
 Goal: keep efficiency, restore interactivity
 Key insight: humans are really just slow I/O devices
    Switch between programs during think-time

 Job 1
                                    Increased complexity:
                                       • Many jobs
 Job 2                                 • Outstanding reqs
                                       • Many job sources
 Job 3
4a: computing as a social medium




            JCR Licklider, “The Computer as a Communication Device”
            Science and Technology, April 1968.
Fifth phase: personal computing
 What are PC operating systems most like?
    As PC prices dropped, single-operator became feasible
    OS was again just a library of services (MS-DOS)
 With one user, do jobs need to time-share?
    Early PC OSes could only do one thing at a time
    Everything waited while printing/loading a program (Mac < X)
 Need protection if I’m the only one using the PC?
    Protect me from myself (or my buggy software)
    Early PCs provided no protection
    (why Windows before XP, Mac before X were awful)
 PC operating systems are basically time-sharing OSes now
Operating system complexity
 Windows XP
   > 40 million lines of code
   Most of this code is device drivers (not written by MS)
 Windows NT took 7 years to develop
   Only worked well years after it shipped
 Windows 2000
   Shipped with 63,000 “potential known defects”
 Hot research area
   Simplify, automatically find OS bugs
OS Complexity
 Lines of code
   Vista: 50 million
   Linux 2.6: 6 million
   (mostly driver code)
 Sources of complexity
   Multiple instruction streams (processes)
   Multiple interrupt sources (I/O, timers, faults)
 How can we keep everything straight?
Dealing with complexity
 Program decomposition
   Functions                 int main () {
                                getInput ();
                                computeResult ();
   OO: Classes, types          printOutput ();
                              }
                              void getInput () {
                                cout << “input: ”;
 int main () {                  cin >> input;
                              }
   cout << “input: ”;
                              void computerResult () {
   cin >> input;                output = sqrt (input);
   output = sqrt (input);       output = pow (output,3);
   output = pow (output,3);   }
   cout << output << endl;    void printOutput () {
 }                              cout << output << endl;
                              }

				
DOCUMENT INFO