Performance inv inv-noexit org
gzip 3.52 3.34 3.14
bzip2 9.43 7.32 5.6
False Positives (Many inputs)
Total False+ve
gzip 244 0
bzip2 472 16
False positive ( train&test)
Total False+ve
gzip 234 8
bzip2 475 140
mcf 614 196
parser 3855 2239
twolf 8775 4019
ammp 191 27
art 154 7
equake 131 99
mesa 1140 326
Latest performance results
Performance inv
gzip 3m.55s
bzip2 5m.37s
mcf 9m.53s
False positives
mcf
243 101 41
twolf
4098 1938 47
gzip
824 105 12
bzip2
656 197 30
parser
1356 661 48
ammp
1370 749 54
art
251 15 5
equake
527 260 49
mesa
1393 207 14
-------------- after filtering char/load/ret -------------
mcf
0 64 22 34
2 66 0 0
6 66 0 0
10 66 0 0
gzip
0 225 35 15
2 229 16 6
6 230 11 4
10 231 5 2
bzip2
0 155 64 41
2 160 39 24
6 160 24 15
10 160 7 4
parser
0 162 111 68
2 164 2 1
6 183 1 0
10 183 1 0
art
0 48 8 16
2 50 0 0
6 50 0 0
10 50 0 0
Performance
sun-cluster (Niagra - single issue in-order machine)
Orginal Invariant Overhead %
art 0:09:07 0:13:18 0:04:11 45.86%
gzip 0:03:11 0:05:07 0:01:55 60.21%
bzip2 0:05:02 0:07:04 0:02:02 40.38%
mcf 0:09:34 0:10:13 0:00:39 6.84%
parser 0:17:34 0:21:01 0:03:27 19.60%
Profiling Time(s) ICache Miss #instrns
Ver1 191.6 1318835 120577838298
Ver2 238.3 2099446 155371142957
Ver3 270.5 3264656 173910392694
Ver4 306.9 4317030 198358280852
Seraph (UltraSparc IIIi superscalar machine)
Orginal Invariant Overhead %
art 0:10:16 0:11:21 0:01:05 10.59%
gzip 0:01:45 0:02:39 0:00:54 51.03%
bzip2 0:03:25 0:04:01 0:00:36 17.57%
mcf 0:15:56 0:20:26 0:04:30 28.21%
parser 0:12:29 0:13:51 0:01:22 10.99%
Profiling Time(s) ICache Miss #instrns
Ver1 105.4 4764273 120403125258
Ver4 159.5 91258394 197604107658
Apoc (x86 AMD Athlon machine)
Orginal Invariant Overhead %
gzip 0:00:55 0:01:04 0:00:08 15.13%
Completion Results (Unknown cases)
Total Same SDC Signals Others(OS-Hang+Appl-Hang+Invariant Failures)
No-exit 447 258 45 108 36
12-in 348 203 23 58 64
2-in 282 170 11 61 40
%
0
3.389831
%
3.418803
29.47368
31.92182
58.08042
45.80057
14.13613
4.545455
75.57252
28.59649
org
3m.11s
5m.3s
10m.12s
Hang+Invariant Failures)
compiled
gzip
bzip2
parser
mcf
twolf
Basic (Optimistic) 2inputs+eleminate invariants
Reduce false +ve more inputs + input sensitivity
Reduce overhead
Try better invariants
#inputs mcf gzip bzip2 parser art 80
2 34 15 41 68 16
4 0 6 24 1 0 60
8 0 4 15 0 0
12 0 2 4 0 0 40
No. of Inputs
20
0
No-exit 12-input-Exit NoExit
5 4 1 2.01%
4 5
15 9 2 17.00%
61 38
31 14 3 19.91%
58 38
30 19 4 22.60%
71 39
5 6 5 2.24%
5 5
39 31 6 18.57%
44 37
37 14 7 17.67%
42 23
447 282 1
1
mcf
gzip
bzip2
parser
art
2 4 8 12
12-Exit
3.19%
16.67%
18.44%
20.57%
3.90%
24.11%
13.12%