analysis of algorithms
You’re in SEM 261.
This is analysis of algorithms
Please make sure you’re in the right place.
We have to get through some boring stuff today.
who I am
quizzes & exams
we’ll also do a little bit of fun stuff.
No heavy lifting today.
course webpage & syllabus
The main course page is
You’ll find the following there:
Possibly other sites, as demand dictates
Blog with lecture notes?
meet the course staff
I’m Richard Kelley
call me Richard
PhD Student – don’t call me “Dr” or “Professor”
Social Artificial Intelligence & Machine Learning
I took this class as an undergrad at UW.
I’ve TA’ed for this class for a few years here.
no required text
two (highly) recommended texts:
Algorithm Design by Kleinberg & Tardos
Introduction to Algorithms by Cormen, et. al.
instead of required textbooks
slides from previous semesters
links to online resources
all available from the course website.
when I use a resource, I will point you to it.
I’m aiming for at least one per week
some really easy
as in, “make sure you can compile this program.”
some more involved
first (easy) one today . . . maybe.
quizzes and exams
regular quizzes to make sure you’re paying attention
these should be easy, count as participation
about half way through the material
homework (55% of your grade)
midterm (15% of your grade)
final (25% of your grade)
attendance & participation (5% of your grade)
what’s this course about?
we want to write good programs.
most advanced courses you take cover topics
some courses are about the processes that lead to better code
the goal is that at the end of this course, your programs will be
regardless of topic
regardless of process
what is a good program?
I know it when I see it?
some common qualities
easy to read
easy to modify
lots of tests
no memory leaks
what’s wrong with these measures?
but we’re going to focus (mostly) on the last two.
we want to write programs that are fast.
we want to write programs that use little memory.
we’re going to take a slightly more general view, though.
not programs. algorithms.
it turns out that focusing on programs is a bad idea.
computers tend to get faster over time.
one way to make your program faster is to buy a better
although faster machines are good, we can do better if
we’re more careful in our design.
this means choosing better algorithms
what is an algorithm?
we can be really picky here
but we won’t be
for our purposes, an algorithm is
a step-by-step process for a well-defined problem
takes zero or more inputs
produces zero or more outputs
is clearly defined
we’ll use pseudocode to describe algorithms
what is a good algorithm?
most problems can be solved in many different ways,
using many different algorithms.
some algorithms are better than others
faster or more space efficient
we want to choose the “best” algorithms
let’s focus on time and try to work out what this means.
(I’m following Kleinberg & Tardos in this process)
even though there are a few quantities we want to
optimize our program for, we’re going to spend most of
our energy on running time.
this is pretty standard.
many of the ideas here carry over to other types of
we need to have notation for
the size of an input
the running time of the algorithm
we’ll use variables like n and m to talk about input sizes.
we focus on input sizes instead of inputs
running time often depends more on the size of the input than the
actual value of the input.
we’ll use T( ) to talk about running times.
for example, T(n) is the running time of an algorithm on an
input of size n.
attempt 1: good means fast
idea: implement the algorithm. run it on some inputs,
record the times
this is acceptable in some cases
in the real world, profiling our programs is a common
this approach has fundamental limitations that we want to
(and can easily) avoid.
at least one homework assignment will involve this kind
of analysis though.
why does this fail?
sensitive to the computer we run on
sensitive to the quality of our implementation
sensitive to the inputs we choose
doesn’t tell us how running time scales with increasing problem
if I run two different algorithms on inputs of size one, and then
I run them on inputs of size two, how much longer do things
rather than look at implementations, we need to think
attempt 2: better than brute force
let’s start by making an agreement:
we will focus on worst-case performance.
we want to find a bound on the largest possible running time
an algorithm could have, over all inputs of size n.
this has a nice property:
not sensitive to inputs we choose.
next, let’s define “good” as “better than a brute-force
search through all possible solutions.”
most problems have a natural “solution space”
path finding in graphs
most optimization problems
with discrete (finite) problems, we can often just look at
every possible solution to see if it’s what we want.
this is usually not tractable.
solution spaces tend to grow quickly
in sorting, brute force requires looking at n! possible
what’s wrong with this definition?
“good” as “better than brute force” is certainly better
than our first attempt, but it’s not perfect.
it’s too vague
we solve vagueness by making our definition more
attempt 3: polynomial time
in brute-force search, when we increase the problem size
by one, we increase the number of possibilities
in sorting, we go from n! to (n+1)!
a good algorithm should scale better.
if we double the input size, we’d like the algorithm to slow
down by only a constant factor.
we get this behavior by agreeing that good algorithms are
an algorithm is efficient if its running time is bounded
above by some fixed polynomial.
if an algorithm’s running time is bounded above by a fixed
polynomial, we may say:
the algorithm has a polynomial running time.
the algorithm runs in polynomial time.
the algorithm is a polynomial-time algorithm.
if a problem has an algorithmic solution that runs in
polynomial time, we might say:
the problem is in the complexity class P.
the problem is in P.
For instance, you probably know that sorting is in P.
bubble sort’s running time is bounded above by n^10.
our goal: revised
based on this discussion, we can be more precise about
the main goals of this course:
we want to find polynomial-time algorithms for important
we want to be able to prove that our algorithms run in
we also want to answer the questions
are there interesting problems (that are solvable) that are not
for such problems, what do we do?
this is a theory course
I’ll provide opportunities to delve into more theoretical items
as the course progresses.
optional for undergrads, less optional for grads.
this is a practical course
you should be able to use this stuff every day.
most of what we’re learning is at the core of the C++ standard
library & the boost library.
as the course progresses, we’re going to learn how to use the
standard library to do a lot for us.
we’re also going to cover C++11 (the standard formerly
known as C++0x)
algorithmic portions only – no variadic template craziness.
the schedule: main topics
(discrete) optimization – dynamic programming & greedy
randomness and approximation
we’ll cover a few topics that are not directly related to
algorithms, but are useful to know.
unit testing – to make grading easier.
profiling – to figure out what parts of your program are taking
static & dynamic analysis tools – to find bugs and general
badness in your code.
extra #1: std::vector
full instructions online
download assignment0.cpp from the course webpage.
make sure you can compile it.
the program illustrates the use of std::vector, which you
can think of as a dynamically-resizable array.
you should get in the habit of using std::vector whenever
you need a contiguous block of storage.
basic usage is similar to arrays, and is illustrated in the
from now on in this class, no more raw arrays.
next time: math review
hopefully I’ve convinced you that we need to do some
the next few lectures will review the most important
ideas and techniques that we have to apply to prove that
algorithms run in polynomial time.
also, assignment 0 is due. Assuming we can get g++ to
work in the ECC.