Docstoc

PPT slides - MACHINE PERCEPTION LABORATORY.ppt

Document Sample
PPT slides - MACHINE PERCEPTION LABORATORY.ppt Powered By Docstoc
					Computer Science:
   A Whirlwind Tour
Jacob Whitehill (jake@mplab.ucsd.edu)
   Machine Perception Laboratory,
       http://mplab.ucsd.edu
                 UCSD
  Goals of This Talk
1. Introduce you to the science of
   computers & computation.
2. Give a super-fast tour of various sub-
   areas of computer science.
3. Tell you a little about our lab’s research.
4. Show some demos.
Finding the science in
  Computer Science
 Computer Science:
  Misconceptions
• Computer science is related to, but
  distinct from, the following fields:
 • Information Technology (IT).
 • Computer & network management
 • Computer programming
 Computer Science:
  Misconceptions
• Information Technology (IT):
 • The study, development, and
   management of information systems -
   computer & software systems that
   manage large amounts of information.
   • Example: design an efficient
     database for Verizon to handle
     billing information for 100 million
     customers.
 Computer Science:
  Misconceptions
• Computer & network management -
 set up and maintain computers &
 computer networks
 • Example: figure out why the Internet
   is not accessible from a particular
   school classroom, and fix the
   problem.
 Computer Science:
  Misconceptions
• Computer programming - create
  software for a computer to execute to
  perform a useful task.
 • Example: write software to
   automatically download music by your
   favorite artist on iTunes.
     My definition of
• Computer Science
  My definition: the development of new
  techniques using computers &
  computation to solve problems that
  were fundamentally not solvable before.
• Contrast with IT: there are no
  fundamental challenges in setting up a
  billing system for Verizon that have not
  yet been solved - it’s clear that it’s
  achievable.
• Contrast with programming: though
  programming is by no means trivial, it is
  My definition of
 Computer Science
• Computer programming represents the
  means by which (empirical) computer
  science research is conducted.
 • Analogous to conducting an assay,
   using a microscope, staining DNA,
   etc.
 Computer Science
• Computer science is (for the most part)
  an engineering science:
 • We have a task we’d like to
    accomplish.
 • We don’t know if it’s solvable, let
    alone how to solve it.
 • We (try to ) find computational
    methods to accomplish our task.
  Computer Science

• Contrast with the natural sciences (e.g.,
  biology, chemistry, physics,
  mathematics):
 • We are interested in discovering how
    the world works, without necessarily
    accomplishing a concrete task.
Sample of Various
Computer Science
Research Problems
        Computer Vision

•   Google Earth - How do you
    “stitch together” satellite
    photographs taken above the
    Earth into a navigable 3-D
    “world” (Google Earth)?
               Graphics


•   Pixar - How do you
    animate a graphical
    video character (e.g.,
    Shrek) so that it
    looks alive?
    Cryptography and
        Security

•   How do you encrypt a message so
    that only the “intended persons”
    can read it and eavesdroppers
    cannot?
•   How do you prevent people from
    copying movies & music illegally?
            Networking

•   How can you transmit
    data reliably and at
    high-speed using
    ordinary power-line
    cables?

    (this is important in
    developing countries
    without telephone
    infrastructure)
    Machine Learning

•   Can computers learn to
    recognize the same
    kinds of patterns in
    nature that humans
    can?
•   Example: where are the
    faces in the video?
    Bioinformatics and
      Computational
          Biology

•   How can we analyze human
    DNA sequences to determine
    the risk factors of certain
    diseases?
    Complexity Theory


•   How can we use quantum
    computers to solve
    mathematical problems
    more quickly than with
    conventional machines?
      Computer Science:
•     Sub-areas (many)
    These are all the ones I can think of off-hand:
•   Graphics
    Vision
    Machine learning
    Databases
    Compilers
    Networking
    Operating systems
    Distributed systems
    Software engineering                 Some of these are
    Robotics
    Numerical computing                mostly empirical, some
    Human-computer interaction
    Graphics                           are mostly theoretical,
    Computer architecture
    Security                            and some are both.
    Cryptography
    Algorithms
    Image & video processing
    Bioinformatics
    Complexity theory
    Computational geometry


•   By the way - some of these areas also fit into Electrical
        Theoretical vs.
   Empirical use
• Theoretical research -Research
  mathematics/logical reasoning to
  prove what you are saying is
  definitely true.




• Empirical research - research
  through observation - run many
  experiments to show that what
  Theoretical vs.
 Empirical Research
• Example:
 •   Theoretical proof that a particular casino gambling
     strategy will give you the highest possible winnings.

 •   Demonstration over many experiments that a particular
     strategy works better than others.

• Important: It’s not always easy/possible
  to prove something mathematically.
 Algorithms - the
  foundation of
computer science
            Algorithms
•   The notion of algorithm is fundamental to all
    of computer science.
•   An algorithm is a step-by-step procedure
    (“recipe”) for how to complete a particular
    task.
•   Computers can perform tasks very quickly,
    but they require a very precise description
    of what to do.
                Algorithms
•   Example:
    •   INPUT: flour, eggs, milk, butter, baking powder,
        salt, chocolate chips
    •   PROCEDURE:
        1. Beat eggs.
        2. Add all other ingredients.
        3. Stir everything until smooth.
        4. Drop in 5” circles on hot frying pan.
        5. Fry on both sides until brown.
              Algorithms


•   Example (continued):
    •   OUTPUT: chocolate chip pancakes
               Algorithms

•   More interesting algorithms:
    •   If you type a phrase into Google, what is the fastest
        way of finding all of the webpages (on the whole
        Internet!) that contain that phrase?
    •   How do you automatically find the faces in a digital
        video?
    •   How do you animate Shrek’s mouth to match his
        speech?
            Algorithms
•   Algorithm descriptions are typically written
    in a computer programming language (e.g.,
    C, Java, Python). (This is where
    programming comes in.)
•   Algorithm descriptions are called code.
•   They are then converted (by a compiler or
    interpreter) into machine language (the
    computer’s native language, written in 0’s
    and 1’s).
              Algorithms

•   Example: counting from 1 to 100.
•   PROCEDURE (in English):
    1. Start with the number 1.
    2. Print the number.
    3. Add 1 to the number.
    4. Go back to Step (2) until we’ve reached 100.
               Algorithms

•   Example: counting from 1 to 100.
•   PROCEDURE (in C, a programming language):
    1. int number = 1;
       while (number <= 100) {
           printf(“%d\n”, number);
           number += 1;
       }
                      Algorithms

•    Example: counting from 1 to 100.
•    PROCEDURE (in PowerPC G5 machine language,
     binary):
    1.   1111111011101101111110101100111000000000000000000000000000010010000000000000000000000000000
         0101000000000000000000000000000000010000000000000000000000000000010100000000000000000000001
         0000010100000000000000000000000000100001010000000000000000000000000000000100000000000000000
         0000000001110000101111101011111010100000100000101000111010001010101101001000101010100100100
         1111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
         0000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000
         0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
         00000000000000000000000000000000000000000000000000000000000000000010000000000000000

         ...
                Algorithms
•   Several sub-areas of computer science
    study the science of algorithms:
    •   Algorithm design
        •   Come up with new fundamental
            algorithms.
    •   Complexity & computability theory
        •   Study the kinds of algorithms that can
            be written, and how fast they are.
Computability Theory
• There are some tasks which are
  fundamentally impossible to solve.
• One of the most famous is the “halting
  problem”.
 • It is possible to write a program that
    never ends. Consider:
   1. Print out “hello”.
   2. Go back to step 1.
        The “Halting
         Problem”
• It would be useful to know whether a
  particular computer program will ever
  finish.
• What we would like is the following:
 • INPUT: any computer program.
 • OUTPUT: either Yes or No according
    to whether the specified computer
    program will (eventually) come to an
    end.
        The “Halting
         Problem”
• It is provably impossible to create such
  an algorithm.
• Hence, no such algorithm can ever be
  created, no matter how sophisticated
  computers ever become.
• This research result comes from the
  field of computability theory.
• Determining whether any computer
  program will terminate is incomputable.
 Complexity Theoryif
• Some computer science tasks, even
  there exists algorithms to solve them,
  may take prohibitively long to finish - for
  instance, a google-google-google years
  (this is a long time).
• Other algorithms may finish their work
  in less than a second.
• Which tasks can be solved quickly, and
  which take more time?
• The amount of time that an algorithm
  takes to finish is called the complexity
    Two Algorithms -
  Which takes longer to
         finish?

• “Stable Marriage”: How can we pair n
  men with n women so that divorces
  their spouse for someone else?
• “Graph 3-Coloring”: How can we color
  the circles in a graph so that connected
  circles have different colors?
    Stable Marriage
       Algorithm
• Suppose we have 10 men and 10
  women who wish to marry each other
  (heterosexual).
• Each person ranks all 10 persons of the
  opposite gender in order of preference.
• We want to pair people so that no one
  will divorce their current spouse to
  marry someone else. Show on board.
   Graph 3-Coloring
• A “graph” has circles and lines.
• Some circles are connected to others
  by lines.
• If two circles are connected by a line,
  then the circles cannot have the same
  color.
• Can we color all the circles in the graph
  using just 3 different colors?
         Algorithmic
         Complexity
• Which problem is harder? “Stable
  Marriage”, or “Graph 3-Coloring”?
• These algorithms may seem contrived,
  but they actually find many uses in
  computer science.
           Algorithmic
           Complexity
•   It turns out that the problem of “Stable
    Marriage” has a very efficient algorithm.
•   Finding marriages for, say, 1000 people
    will take a fraction of a second.
•   Graph 3-Coloring is a hard problem - no
    known efficient algorithm exists (although
    slow ones do exist).
•   For a graph with 1000 circles, finding a 3-
    coloring will take longer than the universe
    will continue to exist!
         Algorithmic
         Complexity
• Let n describe the “size” of the problem.
 • n men/women who wish to marry.
 • n circles in the graph we wish to color.
• The Stable Marriage algorithm is
  quadratic in n.
• The best known Graph 3-coloring
  algorithm is exponential in n.
Machine Learning
     My Research:
    Machine Learning

•   “Machine learning” - the study of algorithms
    that enable computers to “learn” the same
    things that humans can do easily (and not
    so easily).
•   Machine learning is similar to pattern
    recognition.
   My Research:
  Machine Learning
• Examples:
 • How to hear & understand speech.
 • How to see & recognize people.
 • How to determine whether an email
    contains a virus.
• The science involved in machine
  learning is coming up with ways to
  perform such tasks (or to improve their
  performance).
   My Research:
  Machine Learning

• My work:
 • Automatic facial expression
    recognition (and other kinds of face
    processing).
 • Automated teaching systems.
   Automatic Facial
     Expression
     Recognition
• INPUT: an image/video containing
  faces.
• OUTPUT: the locations of all faces in
  the image, along with their facial
  expressions.
• PROCEDURE: unknown (but we’re
  working on it)
   Automatic Facial
     Expression
     Recognition
• Let’s define “facial expression” more
  clearly:
 • One definition is the emotion - happy,
    sad, angry, etc.
 • Another definition (ours) is the kinds
    of facial muscle movements that
    occur in the face.
      Facial Muscle
       Movements
• There are 46 independent muscle
  groups in the face. Here are a few
  examples:




               Source: http://www.cs.cmu.edu/afs/cs/project/face/www/facs.htm
   Automatic Facial
        Expression
• Why would we want to recognize facial
       Recognition
  muscle movements?
 •   Psychological research - how do people
     respond to certain stimuli?
 •   Computer games - make the game
     responsive to how the user is feeling.
 •   Lie detection.
 •   Perceptual computer interfaces (more
     later).
 •   Remote control using the face (more
      Automatic Facial
        Expression
        Recognition
•   The basic flow of how things work:
    1. Find all the faces in the image (face
       detection).
    2. For each face you find, determine its
       facial expression (expression recognition).
     Face Detection

• Contrast face detection and face
  recognition.
  • Detection - where are the faces in the
    image?
  • Recognition - whose face is that?
      Face Detection
• In 2001, Paul Viola and Michael Jones (from
  Mitsubishi Electric Research Labs)
  revolutionized the field of face detection.
• Their design is now called the Viola-Jones
  Face Detector.
• PROCEDURE:
  1. Divide the image into thousands of
     different square regions.
  2. For each square region, check if it
     contains a face. (illustrate on board)
     Does the region
     contain a face?
• We must classify each square region as
  either a face, or a non-face. This is
  performed by a classifier.
• Classifiers: examines a square region of an
  image, and outputs either 1 (“face”) or 0
  (“non-face”).
  • Must operate extremely quickly!
 Classifying a region
   as face/non-face
• We must “train” a face classifier using
  many “training examples”.
 • ~10,000 examples of faces.
 • ~1,000,000,000 examples of non-
     faces.
• The classifier training algorithm
  compares the faces to the non-faces
  and finds the image properties which
  distinguish these two kinds of images.
  (illustrate on board)
    Classifying a region
     as face/non-face
•The classifier training
algorithm automatically finds
“features” of the image that
distinguish faces from non-
faces.
•Once trained, the classifier
can decide (very quickly)
whether the square region
contains a face.
        Expression
       Recognition we
• Now that we’ve found the faces,
  must determine their facial expressions.
• We examine each facial muscle
  movement separately:
 •   Is the person wrinkling their nose? Yes/No.
 •   Is the person smiling? Yes/No.
 •   Is the person blinking? Yes/No.
 •   ...
             Expression
             Recognition
•   To detect expressions, we use a similar
    approach as for face detection:
    •   Train a classifier to distinguish between
        smile/non-smile, frown/non-frown, etc.
    •   For each classifier, we need many
        examples for both the presence, and the
        absence, of the expression.
    •   The classifier examines the face and
        determines whether a particular expression
        occurred or not.
           Our System
•   Our lab (MPLab) has developed a state-
    of-the-art facial expression recognition
    system.
•   “CERT” = Computer Expression
    Recognition Toolbox.
•   CERT rates each expression in terms of
    intensity
    (e.g., not smiling at all ==> very smiley).
•   Demo.
       Unexpected
    Application: Art
            Exhibit detector to
• We deployed CERT’s smile
  “force” people to smile for 1.5 hours.
• Whenever the people stopped smiling,
  CERT would beep at them.
 Unexpected
Application: Art
   Exhibit

         QuickTime™ an d a
  Sorenson Video 3 decompre ssor
   are need ed to see this p icture .
Automated Teaching
     Systems
• Facial expression recognition (and
  other forms of “machine perception”)
  have many applications.
• In my research, I’m interested in one
  particular application: automated
  teaching systems.
• I define automated teaching systems
  very generally - any machine that
  teaches, or helps to teach, a human.
Automated Teaching
     Systems
• Why would we want to do this?
 •   There are many parts of the world in which
     good teachers (especially in specialized
     subjects) are scarce.
 •   The current model of education is one teacher
     for many students. Automated teaching
     systems could help reduce this imbalance.
Automated Teaching
     Systems
•   Some aspects of teaching (e.g., drilling of key
    concepts) are highly repetitive and could be
    automated.
•   (My research): Online learning has become
    very popular in higher education; students
    watch pre-recorded lectures on video streamed
    over the Internet. Can we make these video
    lectures more responsive to the individual
    student?
    •   Facial expression recognition may be useful
        here.
Automated Teaching
           Systems - the basic
• “Smart Video Lecture Player”
 idea:
 •   Some parts of a lecture are already
     familiar to you, or easy to understand.
     You can watch these faster than normal
     and still follow.
 •   Other parts are harder to understand;
     slow these down!
 •   We want to adjust the playback speed
     automatically using facial expression.
Smart Lecture Video
      Player
• Why not just control the speed manually
  (with the keyboard)?
 • The student may be taking notes with
    his/her hands.
 • May be more natural or convenient to
    use the face, similarly to how students
    interact (subconsciously) with their
    teacher.
 Smart Lecture Video
• Player: Experiment
  Question/hypothesis: Does the student’s
  facial expression tell us how difficult he/she
    finds the lecture to be?
•   Experimental procedure:
    1. Each student watched a short video lecture (3
       mins). He/she could adjust the speed (1-10) with
       the keyboard.
    2. Each student’s facial expressions were recorded in
       real-time using the CERT software.
    3. Quiz.
    4. Each student then watched the video again, this
       time giving his/her assessment of how easy/difficult
Smart Lecture Video
 Player: Experiment
• Experimental procedure:
 5. We employed statistical regression algorithms to
    convert the student’s facial expressions into:
      The preferred viewing speed of the student.
      The perceived difficulty of the lecture.

    (Note: the true “computer science” research in this
    project is developing the algorithm to map from
    facial expression to Difficulty and Preferred
    Viewing Speed.)
Smart Lecture Video
 Player: Experiment
• Question:
 • How accurately can our automated
   computer algorithm predict Difficulty
   and Preferred Viewing Speed from
   the facial expression information?
   • We already collected this data, so
     we can determine this
     quantitatively.
   Smart Lecture Video
    Player: Experiment



Difficulty prediction:   0.42 mean Pearson correlation (validation).
Speed prediction:             0.29 mean Pearson correlation
(validation).
Results (in English):
• Using automatic facial expression
  recognition, we can get an approximate
  sense of how Easy/Difficult the student
  finds the lecture to be.
• Can also infer how Fast/Slow the
  student wants to watch the video.
• Applications:
 • Smart Video Lecture Player (as
    intended).
Smart Video Lecture
      Player


• Demo.
Other Work on Automated
Teaching Systems at Our
          Lab
• The RUBI project: deploy a robot
  among pre-school children at UCSD’s
  Early Childhood Education Center.
• Children are between 18-24 months
  old.
• Teach them basic shapes, letters,
  vocabulary (English, Finnish).
RUBI in action



RUBI 1.0   RUBI 2.0
Science as a Career
    Quick Blurb About
           Me
•   Graduated from Stanford (2001); majored in
    computer science.
•   Worked as a software engineer (two years) and
    college lecturer (two years) after finishing college.
•   Been working at UCSD as a researcher &
    graduate student since 2005.
•   Aim to finish my PhD in computer science by 2011.
•   Hope to eventually become a college professor.
Scientific Research:
   My Personal
    Perspective
•   In research, you will never stop learning, by
    definition.
•   Usually lots of freedom: the emphasis is on
    getting results.
•   You will expand the frontiers of human
    knowledge (but usually only in small ways).
•   You will become an “expert” in something.
Scientific Research:
   My Personal
    Perspective
•   Greed for money is replaced by greed for
    prestige.
•   Research can be lonely - you will (often)
    spend lots of time in the lab.
•   You won’t get rich (unless you start a
    company).
From High Schooler
   to Researcher
•   Attending a prestigious research university
    (e.g., MIT, Harvard) is always helpful.
•   BUT - going to a lesser known school by no
    means shuts you out. You just have to take
    more initiative to get involved in research
    projects during college.
•   UC schools are all excellent (in certain
    fields) and are much cheaper (for CA
    residents) than private schools.
    From High Schooler
       to Researcher
•   In my opinion, the #1 most important thing to do in
    college is:
    GET TO KNOW YOUR PROFESSORS.
    •   Go to office hours.
    •   Ask them questions.
    •   Ask them about their research.
    •   Ask them how you can get involved in one of their
        research projects.

•   Professors are the reason why famous schools are
    famous.
•   A strong letter of recommendation from a professor
    is worth its weight in platinum.
The End

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:50
posted:4/25/2011
language:English
pages:81
handongqp handongqp
About