Back of the Envelope
Reasoning
Praveen Paritosh
Ken Forbus
QR04 Practice Talk
Thursday, July 8, 2004
Outline of the talk
• Examples
• Motivations for modeling BotE reasoning
• Hand-worked out examples
• Implementation of BotE-Solver
• Results and current status
• What next
Some examples
• How many K-8 school teachers are in the USA?
• How much money is spent on newspapers in USA per year?
• How much time would be saved per year nationwide by
increasing the speed limit from 55 to 65 mph?
• What is the annual cost of healthcare in USA?
• How much tea (weight) is there in China?
• Last summer, the US Army bought Microsoft
Windows/Office/Server software for 500,000 computers.
The deal included the software and six years of support.
How much did the army pay for this?
Back of the Envelope Reasoning
• Numeric answers
• Specificity-Resources tradeoff
• Only thing you can do in many domains
– Environmental science, Biophysics, etc.
– New, unfamiliar domains.
• Useful
– Engineers, Scientists, Policymakers, everyone who
reads a newspaper
• Problem solving + feel for numbers
[contrast first principles reasoning]
Constraints guiding Common Sense QR
1. Incompleteness
Domain theories incomplete in coverage.
2. Concreteness
Knowledge of concrete, specific situations (made use of by
analogical reasoning) in addition to first-principles
reasoning.
3. Highly experiential: Experience improves
- ability to reason through similar scenarios.
- intuitions for quantity: what is reasonable, high, low in a domain.
4. Focused reasoning
Tight reasoning, as opposed to maintaining ambiguity for
completeness
5. Pervasively quantitative
Real-world actions require that estimates manifest as exact values.
How many K-8 school teachers
are in the USA?
Number of teachers = number of students / students per teacher
Number of students = population * fraction in the age range of K-
8 students * fraction of kids who go to school
Number of students = 300 mil * 9/75 * 1
Number of teachers = 40 mil / 25 = 1.6 mil
1.9 million, source: Statistical Abstracts, 2003.
How much money is spent on
newspapers in USA per year?
Total money spent = Money spent per buyer * number of buyers
Annual expense per buyer = Units bought per year * cost per unit
Annual expense per buyer = 365 * 0.75 = 250
Number of buyers = 300 mil * ¼ = 75 mil
Total money spent = 75 mil * 250 = 20 billion
26 billion, source: Statistical abstracts, 2003.
What is the total annual gasoline
consumption by cars in US?
Total consumption = Total miles driven/ miles per gallon
Total miles driven = Number of cars in the US * Miles driven
per car per year
Miles driven per car per year = Miles driven per day * 365
If we say that every household owns a car, since some don’t
and some might have more than one, then
Number of cars in the US = number of households =
population / average size of American household.
Now we have a model, and using the following numbers,
Population ~ 300 million, Average size of household ~ 3,
Daily miles driven ~ 20, Miles per gallon for a car ~ 20.
We get an estimate of 36.5 billion gallons.
35 billion gallons, Statistical Abstracts, 2003.
Formalizing BoTE reasoning
Estimate parameter directly 1
Use known value
Feel for numbers
if available
2
Estimate based
Create estimation model on similar situation
Problem solving
Find modeling strategy
3
Find values for
parameters
in model
Problem Solving
• Representation
– Of problems, strategies and domain knowledge.
• Retrieval
– Of relevant knowledge and strategies
• Reasoning
– Workspace: keeping track of progress made.
– Agenda: figuring out what to do next.
BotE-Solver
• Representation
– Domain knowledge: 1.2 million fact subset of
Cycorp’s CYC knowledge base.
– Strategies: Suggestions.
• Retrieval
– Pattern matching and backward chaining.
• Reasoning
– Built on top of FIRE reasoning engine
– AND/OR tree as workspace.
– Difficulty ordered agenda.
AND/OR Decomposition
Total annual
gasoline
consumption
MileageStrategy
Suggestion/OR-node
Total miles Miles per
driven Gallon
Goal/AND-node
PerUnitStrategy
Number of Annual miles
cars driven per car
HouseholdStrategyForCountingUnits PerDayForYearStrategy
Number of cars Number of Miles driven
per house households per day
Workings of the Solver
Control Status Flags Solution generation
• SOLVED • Incremental
• FAILED • Two modes
• MOOT – Agenda processing
– MOOT-VIA-FAILURE – IN-PLAY processing
– MOOT-VIA-SUCCESS Subgoals
• Fully ordered
An example suggestion
(defSuggestion HouseholdStrategyForCountingUnits
:trigger (unitsTotal ?obj ?place ?time ?total-units)
:test (ownedBy ?obj FamilyCohabitationUnit)
:subgoals ((numberOfHouseholds
?place ?time ?num-households)
(unitsPerHousehold
?obj ?units-per-household))
:result-step (evaluate ?total-units
(TimesFn ?num-households
?units-per-household)))
Axioms for HouseholdStrategyForCountingUnits
(ist-Asserted SuggestionMT
(suggestionResultStep HouseholdStrategyForCountingUnits
(evaluate ?total-units (TimesFn ?num-households ?units-per-household))))
(ist-Asserted SuggestionMT
(suggestionSubgoals HouseholdStrategyForCountingUnits
(TheList (numberOfHouseholds ?place ?time ?num-households)
(unitsPerHousehold ?obj ?units-per-household))))
(ist-Asserted SuggestionMT
(suggestionGoalForm HouseholdStrategyForCountingUnits
(unitsTotal ?obj ?place ?time ?total-units)))
(ist-Asserted SuggestionMT
(implies (ownedBy ?obj FamilyCohabitationUnit)
(suggestFor (unitsTotal ?obj ?place ?time ?total-units)
HouseholdStrategyForCountingUnits)))
(ist-Asserted SuggestionMT (isa HouseholdStrategyForCountingUnits Suggestion))
(suggestionResultStep HouseholdStrategyForCountingUnits
(evaluate ?total-units (TimesFn ?num-households ?units-per-household)))
(suggestionSubgoals HouseholdStrategyForCountingUnits
(TheList (numberOfHouseholds ?place ?time ?num-households)
(unitsPerHousehold ?obj ?units-per-household)))
(suggestionGoalForm HouseholdStrategyForCountingUnits
(unitsTotal ?obj ?place ?time ?total-units))
(implies (ownedBy ?obj FamilyCohabitationUnit)
(suggestFor (unitsTotal ?obj ?place ?time ?total-units)
HouseholdStrategyForCountingUnits))
(isa HouseholdStrategyForCountingUnits Suggestion)
More examples
(annualSales NewspaperCopy
UnitedStatesOfAmerica (YearFn 2003) ?money)
(defSuggestion PerBuyerStrategy
(annualSales ?obj ?place ?time ?money)
:subgoals ((annualExpensePerBuyer ?obj ?place ?time ?money-per-buyer)
(numberOfBuyers ?obj ?place ?time ?number))
:result-step (evaluate ?money (TimesFn ?money-per-buyer ?number)))
(defSuggestion UnitaryStrategyForCost
(annualExpensePerBuyer ?obj ?place ?time ?money-per-buyer)
:subgoals ((annualUnitsBoughtPerBuyer ?obj ?place ?time ?units)
(cost ?obj ?unit-cost))
:result-step (evaluate ?money-per-buyer (TimesFn ?units ?unit-cost)))
(defSuggestion FractionOfTotalStrategy
(numberOfBuyers ?obj ?place ?time ?number)
:subgoals ((populationDuring ?place ?any-time ?total)
(percentOfBuyers ?obj ?place ?percent))
:result-step (evaluate ?number (QuotientFn (TimesFn ?total ?percent) 100)))
(cardinality K-8SchoolTeacher ?numteachers)
(defSuggestion StudentsPerTeacherStrategy
(cardinality K-8SchoolTeacher ?numteachers)
:subgoals ((cardinality K-8SchoolStudent ?numstudents)
(studentsPerTeacher K-8School ?perteacher))
:result-step (evaluate ?numteachers (TimesFun ?numstudents ?perteacher)))
(defSuggestion UniformAgeDistributionStrategy
(cardinality K-8SchoolStudent ?numstudents)
:subgoals ((populationDuring UnitedStatesOfAmerica (YearFn 1997) ?population)
(minimumAge K-8SchoolStudent ?min)
(maximumAge K-8SchoolStudent ?max)
(lifeExpectancyForGroupInRegion UnitedStatesOfAmerica Person
(YearsDuration ?life)))
:result-step (evaluate ?numstudents (TimesFn
(QuotientFn (DifferenceFn ?max ?min) ?life) ?population))))
Current Status
• Can successfully solve 8 problems
• Strategy library contains 23 suggestions
Solved Problems - I
Proble Problem, and its predicate calculus Answer found by BotE-Solver, #Axio
m representation and comparison to a known ms
Number answer if available
1 How many popcorn kernels would it take to fill (?number . 1.343444e+7) 30
in the 1890 Maple big classroom? BotE-Solver: 13 million
(CountContained CS381ClassRoom Popcorn Correct answer: not available!
?number)
2 How much money is spent on newspapers in the (?money . 2.1884363e+10) 30
US? BotE-Solver: 21 billion
(annualSales NewspaperCopy Correct answer: 26 billion
UnitedStatesOfAmerica (YearFn 2003)
?money)
3 How many K-8 teachers are there in the US? (?numteachers . 1056454) 20
(cardinality K-8SchoolTeacher BotE-Solver: 1.05 million
?numteachers) Correct Answer: 1.9 million
4 What is the annual cost of healthcare in the US? (?money . 799428834000) 12
(annualSales HealthCare BotE-Solver: 0.8 trillion
UnitedStatesOfAmerica (YearFn 2003) Correct Answer: 1.6 trillion
?money))
Solved Problems - II
Proble Problem, and its predicate calculus Answer found by BotE- #
m representation Solver, and comparison to a Axioms
Number known answer if available
5 How many cars are bought per year in the US? (?num . 8920000) 30
(unitsBoughtPerYear Automobile BotE-Solver: 8.9 million
UnitedStatesOfAmerica (YearFn 2003) ?num) Correct Answer: 8 million
6 What is the weight of garbage thrown away by (?garbage-mass . 446000000) 10
American families each year? BotE-Solver: 446 million
(annualProduction Garbage-Generic pounds
UnitedStatesOfAmerica (YearFn 2003) Correct answer: not
(Pound-UnitOfMass ?garbage-mass)) available!
7 How many hotdogs are sold in a baseball season (?num-dogs . 1600000) 18
in Wrigley Field? BotE-Solver: 1.6 million
(unitsSold HotDogSandwich WrigleyField Correct answer: not
BaseballSeason ?num-dogs) available!
8 What is the total amount of gasoline (?oil-consumption . 20
consumption by cars in the US? 32558000000)
(annualAutomobileGasConsumption BotE-Solver: 32.6 billion
UnitedStatesOfAmerica (YearFn 2003) gallons
(Gallon-US ?oil-consumption))) Correct Answer: 35 billion
gallons
What next
• Represent about 50 problems
– Show stable re-usable representations, in strategies
and theoretical knowledge added.
– Knowledge level analysis of the kinds of knowledge
involved in BotE.
• Feel for numbers
– Build symbolic representations for numbers
• Large, expensive, upper class, etc. [CARVE, Paritosh, 2003]
– Analogical estimator: makes guesses for a numeric
parameter based on analogy.
For More
The BotE website:
www.cs.northwestern.edu/~paritosh/bote/
Library of problems, solutions, strategies.
Extra Stuff
Pseudocode
get-solution
get-solution(original-goal)
if in-play?(original-goal)
get-next-solution(original-goal)
else if the agenda is empty then quit
process-agenda
get-next-solution
get-next-solution(ao-node)
if there are cached solutions at this node
;; cur-bmarker points to the current solution
increment cur-bmarker(ao-node)
return cached solution
if goal-node?(ao-node)
if ao-node has a child suggestion that is in-play
get-next-solution(in-play-suggestion(ao-node))
;; No in-play suggestion, if this goal has a younger
;; sibling that can give us more new bindings, re-instantiate
;; this node with those bindings.
if younger-siblings(ao-node)
bindings = get-next-solution(younger-sibling(ao-node))
if bindings found
enqueue-on-agenda(re-instantiate-node(ao-node,bindings))
if suggestion-node?(ao-node)
;; seek downward
solution = get-next-solution(eldest-child,ao-node)
add-bindings(solution,ao-node)
increment cur-bmarker(ao-node)
return solution
propagate-bindings
propagate-bindings(goal-node)
if elder-sibling?(goal-node)
increment cur-bmarker(goal-node)
next-node = instantiate-node(elder-sibling(goal-node),
current-bindings(goal-node))
add-to-tree(next-node)
enqueue-on-agenda(next-node)
return
;; No elder sibling, so we can now see
;; if the parent node got solved
if parent(goal-node) exists
bindings = get-next-solution(goal-node)
while we have bindings for goal-node
result-bindings = do-result-step(parent(goal-node))
if result-bindings found
add-bindings(result-bindings,parent(goal-node)
propagate-bindings(parent(parent(goal-node)))
bindings = get-next-solution(goal-node)
process-agenda
process-agenda(ao-node)
if goal-node?(ao-node)
solutions = ask(goal-node) ;; primitive problem solving step
if solutions found
propagate-bindings(goal-node)
else suggestions = gather-suggestions(goal-node)
if suggestions found
enqueue-on-agenda(suggestions)
else ;; no suggestions found, see if we can
;; re-instiate this node
if younger-siblings(goal-node)
bindings = get-next-solution(younger-sibling(goal-no
if bindings found
enqueue-on-agenda(re-instantiate-node(ao-node,bin
else update-failed (goal-node)
if suggestion-node?(ao-node)
;; since the subgoals are ordered, add the first one to agend
enqueue-on-agenda(youngest-child(ao-node))
Examples
How much time would be saved by increasing
the speed limit from 55 to 65 mph?
Total time spent driving = time spent driving per person * number of drivers
Time spent driving per person = Total distance driven per person /Speed
Total distance driven per person = Distance per day * number of days
driving
Number of drivers: assume one driver per household = 1/3 * population =
100 mil
Total distance driven per person = 20 * 365 = 8000
Time spent per person = 8000/55 = 150 hours
Time spent if it was 65 mph = 8000/65 = 120 hours
Time saved per person = 30 hrs
Total time saved = 3 billion hours = 300,000 years.
The Microsoft Army – How much does
software for 500,000 computers cost?
Total cost on the software = cost of desktop software + server software +
upgrades
Windows XP/Office CDW prices = 650
Cost of desktop software = 500,000 * 650 = 325 mil
Cost of upgrades = 500,000 * 200 = 100 mil
Assume 1 in 100 servers = 5000 servers
Exchange server = 600
Windows 2003 server = 650
SQL Server = 1000
Cost of server software = 2250 * 5000 = 11.25 mil
Cost of upgrades = 400 * 5000 = 2 mil
Total cost = 440 million
Softmart, inc, PA got paid = 470 million
What is the annual cost of
healthcare in USA?
Lets say everyone was insured.
Total cost of healthcare has to be less than insurance premiums, for
the insurance companies to stay in business.
Cost of healthcare = average insurance premium * population
= 3,000 * 300 million ~ 1 trillion
[1.6 trillion last year. How does this work?]
How much tea is there in China?
Total tea = amount of tea consumed per day *
stockpile in time
Tea consumed per day = tea consumed per
person per cup * number of cups * population
1 billion people
5 cups a day
5 grams a cup
Lets say they stock an years supply
25 billion grams = 25 million kilos per day
In an year 365*25 million ~ 10 billion kilos
CARVE, feel for numbers
Numbers in symbolic knowledge
representation
• Consider the Great black-bucked gull
– Wingspan = Large
– Wing-span = 0.272 sq. m.
• Numbers not handled right
– Similarity: computing and making inferences
– Retrieval
– Generalization
CARVE: Symbolizing Quantity
• Don’t do it
– Sorites
– Context/Utterer sensitivity
• Dimensional partitions: Large and Small, based
on distributional properties of the quantity.
(isa Algeria
(HighValueContextualizedFn
Area AfricanCountries))
• Structural partitions: Boiling point and Poverty
line, denote changes of quality.
CARVE
Dimensional (isa Algeria
(HighValueContextualizedFn
partitioning for
Area AfricanCountries)
Ci each quantity .
.
.
Add these facts to
Quantity 1 original cases
Structural
Cj S1 S2 S3 clustering
Ci*
using SEQL
L1 L2
Analogical Estimator
(GrossDomesticProduct Brazil ?x)
• The value is known.
• Find an analogous case for which value is
known.
• Find anything in the KB which might help
me make an estimate.
Thesis and Evaluation
• More powerful and flexible back of the
envelope reasoning can be done using
these symbolic representations of quantity.
• Evaluation:
– Corpus of problems with/without
representations generated by CARVE.
Numbers in News
• The cost of Mars Pathfinder mission was
270 million.
• 50 billion tax cut.
• 62,000 square feet.
Old slides
An example
(defSuggestion VolumeFormulaForSphere
(volumeOfObject ?obj ?vol)
:test (shapeOfObject ?obj SphereShape)
:subgoals ((radius ?obj ?radius))
:result-step (evaluate ?vol
(TimesFn 4.187 ;;4/3*Pi
(ExponentFn ?radius 3))))
SOLVE
• Suggestions based problem solver
• Represent problem solving progress in an
and/or tree structure
• Incremental solution generation, get-
next-solution
• Ordered subgoals