Changes in Probabilistic
Causal Models
or: Do we stink at probability or what?
James Baker
PSY446
Participation
Mostly NU undergrads
Some NU grad students
Early data collection error / internet preteens filtered
67 data records
53 finished samples
13 of type 0
14 of type 1
13 of type 2
13 of type 3
Gametype 0 (control)
A: Display "33%" Prob: .35 Price: $3
B: Display "66%" Prob: .68 Price: $6
C: Display "97%" Prob: .95 Price: $10
No changes
Responses to gametype 0
I can get 20 right if I always choose the most expensive...
I didn't really know how the computer was operating. I just
figured that if the probability was high I would follow that
suggestion.
I felt like 'B' was more than 66 percent, so after trying the other
two, I stuck with B.
I chose to go with query B every time, because, on average, this
would return 22 correct answers. Query A would do the same.
And, I think, query C would return about 19, on average. Unless
my math is wrong (which is very possible)...consistently choosing
A or B is the best bet.
so i guess i got more 'correct' answers when i just played many
times with low probablility of winning- but again- it's a little too
late in the day for me to actually strategize.
Responses to gametype 0
I think the computer worked on the probability given,
but when you moved to a higher cost value, it
decreased the likelihood that the probability given
would work at a lower cost value. (Therefore, if you
ask at $3, then move to $6, $3 will be less likely to
work in subsequent rounds.) I mostly stayed with the
$6 option, only moving to $10 when I wanted a
correct answer and $3 when I wanted $6 to be more
likely, which seemed to work.
I spent nearly all my money on the 97% question.
i would stick with c until it stopped being correct,
then move to b for a while until it stopped working
then go back to c, etc
Results-0
> 90% A’s: 3
> 75% B’s: 4
> 70% C’s: 4
30% 95% A’s: 2
> 70% B’s: 1
> 70% C’s: 6
Mixed: 4
Among mixed: 57 A, 61 B, 77 C
Conclusion: We like round numbers?
Gametype 2
A: Display "35%" Prob: .37 Price: $3
B: Display "65%" Prob: .67 Price: $5
C: Display "95%" Prob: 1.0 Price: $7
C starts as best choice
C ramps from 1.0 to .3
Hypothesis: People will start on C, then
change to B/A when C gets worse.
Responses to gametype 2
Same logic as last time (divided $200 by the cost/query and then
multiplied by the odds of that query returning a "correct answer").
I predicted 27 correct answers for choice C, which would have
been the most.
considering that the correct answer changed (I haven't figured
out the pattern yet) when I stopped getting right answers I would
switch.
Option C maximizes expected profit in the long run. Hence, I
believe in following it should return in the best outcome in the
long-run.
I just tried the 95% to play it safe.
i didn't change my strategy at all, i just kept going for what i
thought was the most consistent, c every time.
stick with highest odds
Responses to gametype 2
I think the computer worked on the probability given, but when
you moved to a higher cost value, it decreased the likelihood that
the probability given would work at a lower cost value.
(Therefore, if you ask at $3, then move to $5, $3 will be less
likely to work in subsequent rounds.) I mostly stayed with the $5
option, since it was a better than 50% chance and it cost an
average amount.
just played C as long as I could (in writing, it seemed like the best
deal)
The correct answer was not returned at anywhere near 95%, so I
don't know how I'm supposed to guess which query to go with.
I stayed with the highest probability option
It seemed that the computer started returning wrong answers for
C more than 5% of the time, even when I would vary my
answers.
Results-2
>90% B: 1
>75% C: 10
50 80% A: 5
> 90% B: 1
> 70% C: 0
Mixed: 7
Of A’s, 2 were ALL As and 3 had other
selections in the middle section.
Conclusion: More churn with bad outcomes,
again slow to change model of A
Aggregate Results
Tendency to pick one favorite
We like round numbers?
Very slow to change mental model from
samples
More churn with bad outcomes, again slow to
change model of A
Disconnect between comments and
unconscious trends?
Confounding Factors
Display of percentages reinforcing static
mental model
Fin
James Baker
Backup
Gametype 0
20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C.
55 choices: 50 A's, 0 B's, 5 C's. .91% A, .00% B, .09% C.
20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C.
32 choices: 3 A's, 25 B's, 4 C's. .09% A, .78% B, .12% C.
30 choices: 6 A's, 15 B's, 9 C's. .20% A, .50% B, .30% C.
24 choices: 4 A's, 3 B's, 17 C's. .17% A, .12% B, .71% C.
35 choices: 5 A's, 29 B's, 1 C's. .14% A, .83% B, .03% C.
20 choices: 0 A's, 0 B's, 20 C's. .00% A, .00% B, 1.00% C.
33 choices: 0 A's, 33 B's, 0 C's. .00% A, 1.00% B, .00% C.
27 choices: 1 A's, 16 B's, 10 C's. .04% A, .59% B, .37% C.
60 choices: 55 A's, 4 B's, 1 C's. .92% A, .07% B, .02% C.
33 choices: 0 A's, 33 B's, 0 C's. .00% A, 1.00% B, .00% C.
66 choices: 66 A's, 0 B's, 0 C's. 1.00% A, .00% B, .00% C.
Backup
Gametype 1
42 choices: 3 A's, 4 B's, 35 C's. .07% A, .10% B, .83% C.
40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C.
47 choices: 7 A's, 21 B's, 19 C's. .15% A, .45% B, .40% C.
51 choices: 20 A's, 15 B's, 16 C's. .39% A, .29% B, .31% C.
49 choices: 5 A's, 35 B's, 9 C's. .10% A, .71% B, .18% C.
47 choices: 16 A's, 3 B's, 28 C's. .34% A, .06% B, .60% C.
50 choices: 14 A's, 22 B's, 14 C's. .28% A, .44% B, .28% C.
65 choices: 63 A's, 0 B's, 2 C's. .97% A, .00% B, .03% C.
40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C.
40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C.
40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C.
41 choices: 2 A's, 2 B's, 37 C's. .05% A, .05% B, .90% C.
40 choices: 0 A's, 0 B's, 40 C's. .00% A, .00% B, 1.00% C.
66 choices: 66 A's, 0 B's, 0 C's. 1.00% A, .00% B, .00% C.
Backup
Gametype 2
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
35 choices: 7 A's, 9 B's, 19 C's. .20% A, .26% B, .54% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
29 choices: 0 A's, 2 B's, 27 C's. .00% A, .07% B, .93% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
31 choices: 2 A's, 5 B's, 24 C's. .06% A, .16% B, .77% C.
40 choices: 2 A's, 37 B's, 1 C's. .05% A, .93% B, .03% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
29 choices: 1 A's, 0 B's, 28 C's. .03% A, .00% B, .97% C.
33 choices: 2 A's, 12 B's, 19 C's. .06% A, .36% B, .58% C.
Backup
Gametype 3
115 choices: 98 A's, 17 B's, 0 C's. .85% A, .15% B, .00% C.
38 choices: 12 A's, 18 B's, 8 C's. .32% A, .47% B, .21% C.
200 choices: 200 A's, 0 B's, 0 C's. 1.0% A, .00% B, .00% C.
35 choices: 2 A's, 33 B's, 0 C's. .06% A, .94% B, .00% C.
119 choices: 106 A's, 9 B's, 4 C's. .89% A, .08% B, .03% C.
84 choices: 68 A's, 7 B's, 9 C's. .81% A, .08% B, .11% C.
39 choices: 18 A's, 7 B's, 14 C's. .46% A, .18% B, .36% C.
32 choices: 4 A's, 21 B's, 7 C's. .12% A, .66% B, .22% C.
44 choices: 24 A's, 6 B's, 14 C's. .55% A, .14% B, .32% C.
200 choices: 200 A's, 0 B's, 0 C's. 1.0% A, .00% B, .00% C.
33 choices: 10 A's, 10 B's, 13 C's. .30% A, .30% B, .39% C.
28 choices: 8 A's, 2 B's, 18 C's. .29% A, .07% B, .64% C.
26 choices: 4 A's, 6 B's, 16 C's. .15% A, .23% B, .62% C.
Backup
Gametype 0
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
B,B,B,A,A,A,B,B,B,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,
C,C,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A,A,B,B,B,B,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,B,B,C,C,C,C,C,C,C,A,A,A,B,A,
B,A,A,A,B,B,B,B,B,B,B,B,B,B,B,C,A,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,
B,B,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,C,C,C,C,B,B,C,C,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,A,A,A,A,A,A,A,A,A,B,C,B,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,A,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
Backup
Gametype 1
A,B,C,C,C,C,B,A,B,C,A,C,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,C,C,C,B,C,A,C,C,B,B,A,C,C,C,B,A,A,A,C,C,C,C,C,C,C,B,B,B,A,C,B,A,C,B,B,B,B,B,B,B,B,B,B,B,
B,B,
B,A,A,A,A,A,A,A,A,A,A,A,A,A,B,B,B,B,B,B,B,C,C,C,C,C,C,C,C,A,A,A,A,A,B,B,B,B,B,C,C,C,C,C,C,C
,B,B,A,A,C,
A,A,A,B,B,B,B,C,C,C,C,C,C,C,C,C,A,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,
B,B,A
A,A,A,A,A,C,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,B,
B,B,B,B,B,B,B,B,B,B,C,C,C,C,C,C,B,B,B,A,A,A,B,B,B,B,C,C,C,C,A,A,A,A,A,A,A,A,A,C,C,B,B,B,B,B,
A,C,C,A,
C,C,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,A,A,B,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A
,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
Backup
Gametype 2
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,A,A,C,A,C,C,C,C,C,B,B,B,B,B,A,A,A,A,C,C,C,C,C,C,B,B,B,B
,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,B,C,C,C,C,C,C,C,C,B,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,B,B,A,B,B,A,C,C,B,C,C,
B,B,B,B,B,B,B,B,B,B,B,A,B,B,B,B,B,B,B,C,B,B,B,A,B,B,B,B,B,B,B,B,B,B,B,B
,B,B,B,B,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,
C,C,C,A,C,C,C,C,C,B,B,B,C,C,C,B,B,B,B,B,B,C,C,C,B,C,C,C,B,C,C,B,A,
Backup
Gametype 3
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B,B,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,
B,C,A,A,A,B,B,B,A,B,B,C,C,C,C,B,B,B,B,B,A,A,A,A,A,A,A,A,B,B,B,B,B,B,B,C,C,C,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A,
B,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,B,B,B,B,B,B,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,A,A,B,B,A,A,A,A,C,C,A,B,B,B,A,C,A,A,A,A,A,A,C,B,A,A,A,A
,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,B,A,
C,B,B,A,C,C,C,C,C,B,A,C,C,C,C,C,C,B,B,B,A,C,C,A,A,A,A,A,A,A,A,A,B,A,A,A,A,A,A,
C,B,C,C,C,C,C,C,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,A,A,A,A,
B,B,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A,A,A,A,A,C,C,C,C,C,C,C,C,C,B,B,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,
A,A,A,A,A,A,A,A,A,A,C,C,C,B,B,B,B,B,B,B,C,C,C,C,C,C,C,C,C,C,B,B,B,
C,B,B,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,C,A,A,A,A,A,A,A,A,
C,B,B,C,C,C,B,C,A,C,B,C,A,C,C,B,C,C,C,C,C,C,C,B,A,A