VIEWS: 32 PAGES: 75 CATEGORY: Education POSTED ON: 8/17/2010 Public Domain
Perspectives in Probabilistic Model Checking Joost-Pieter Katoen Software Modeling and Veriﬁcation Group afﬁliated to University of Twente, Formal Methods and Tools Seminar@Chinese Academy of Science, Beijing, June 30, 2008 c JPK Model checking • Automated model-based veriﬁcation and debugging technique – model of system = Kripke structure ≈ labeled transition system – properties expressed in temporal logic like LTL or CTL – provides counterexamples in case of property refutation • Various striking examples – Needham-Schroeder protocol, cache coherence, storm surge barrier, C code • 2008: Pioneers awarded prestigious ACM Turing Award • Today: model checking of probabilistic models c JPK 1 Probabilities help • When analysing system performance and dependability – to quantify arrivals, waiting times, time between failure, QoS, ... • When modelling uncertainty in the environment – to quantify imprecisions in system inputs – to quantify unpredictable delays, express soft deadlines, ... • When building protocols for networked embedded systems – randomized algorithms • When problems are undecidable determinitically – reachability of channel systems, ... c JPK 2 What is probabilistic model checking? up to 107 states requirements inaccuracy system P 0.01 (3deadlock) 0.8 0.2 0.6 0.4 Formalizing Modeling property speciﬁcation system model Model Checking state 1 0.678 the probability state 2 0.9797 satisﬁed state 3 0.1523 state 4 0.2123 insufﬁcient memory c JPK 3 Probabilistic models Nondeterminism Nondeterminism no yes Discrete time discrete-time Markov decision Markov chain (DTMC) process (MDP) Continuous time CTMC CTMDP c JPK 4 Breakthroughs • Zero-one probabilities for Markov decision processes (Vardi 1985) – does an LTL formula hold with probability zero? • Markov decision processes (Courcoubetis & Yannakakis 1988) – does the maximal probability for an LTL formula equal p? • Discrete-time Markov chains (Hansson & Jonsson 1990) – does the probability of a CTL formula equal p? • Markov decision processes (Bianco & de Alfaro 1995) – does the maximal probability for a CTL formula equal p? • Continuous-time Markov chains (Baier, Katoen & Hermanns 1999) – does the probability of a timed CTL formula equal p? c JPK 5 Characteristics • What kind of properties? – CTL/LTL + (time-bounded) reachability probabilities, long-run averages • What is inside? – temporal logics, model checking, numerical and OR techniques • What is its usage? – powerful tools: PRISM (4,000 downloads), MRMC, Petri net tools, Probmela – applications: distributed systems, security, biology, avionics . . . tool MRMC downloadable at: www.mrmc-tool.org c JPK 6 Discrete-time Markov chain 1 2 t s 1 u 1 2 1 2 1 2 v 1 a DTMC is a triple (S, P, L) with state space S and state-labelling L and P a stochastic matrix with P(s, s ) = one-step probability to jump from s to s c JPK 7 Markov Decision Process 1 1 2 1 2 4 t s 1 u 1 2 1 2 1 1 2 4 v 1 an MDP is a DTMC if in each state there is only one color to choose c JPK 8 Timing let’s consider random timing c JPK 9 Exponential distribution Continuous r.v. X is exponential with rate λ > 0 if its density is f (x) = λ·e−λ·x for x > 0 and 0 otherwise Cumulative distribution of X: d FX (d) = f (x) dx = 1 − e−λ·d 0 Expectation: ∞ 1 E[X] = x·f (x) dx = 0 λ c JPK 10 Exponential pdf and cdf 1.5 1 1.4 λ = 0.5 λ = 1.0 0.9 1.3 λ = 1.5 1.2 0.8 1.1 0.7 1 0.9 0.6 0.8 0.5 0.7 0.6 0.4 0.5 0.3 0.4 0.3 0.2 λ = 0.5 0.2 λ = 1.0 0.1 λ = 1.5 0.1 0 0 0 1 2 3 4 5 0 1 2 3 4 5 the higher the rate λ, the faster the cdf approaches 1 c JPK 11 Continuous-Time MDP a CTMDP is an MDP plus an exit-rate function E : S × Act → R 0 1 1 2 , 10 1 2, 4 4 , 10 t s 1 u 1, 2 2 , 25 1 2 , 25 1 1 2, 4 4 , 10 v 1, 100 note: when removing exit rates, an embedded MDP is obtained c JPK 12 Continuous-Time Markov chain a CTMC is a DTMC plus an exit-rate function E : S → R 0 1 2, 4 t s 1 u 1, 2 2 , 25 1 2 , 25 1 2, 4 v 1, 100 note: when removing exit rates, an embedded DTMC is obtained c JPK 13 Continuous-Time Markov chain a CTMC is a DTMC plus an exit-rate function E : S → R 0 1 4 2 2 t s 1 u 1 2 1 25 2 1 2 100 v 1 1 the average residence time in state s is E(s) c JPK 14 Uniform CTMCs • A CTMC is uniform if E(s) = k for all s for some k ∈ R>0 • Any CTMC can be changed into a weakly bisimilar uniform CTMC • Let k ∈ R>0 such that k maxs∈S E(s) – 1 k is at most the shortest mean residence time in CTMC C • Then uk (C) = (S, P, E, L) with E(s) = k for any s, and: E(s) P(s, s ) = P(s, s )· for s = s and P(s, s) = 1 − P(s, S\{ s }) k P = P(s,s ) s ∈S\{ s } c JPK 15 Uniform CTMCs 1 3 1 3 3 6 4 4 6 2 6 4 6 1 1 2 3 1 1 2 4 1 4 3 uniformisation with k = 6 all state transitions in uk (C) occur at an average pace of k per time unit c JPK 16 Modeling techniques for CTMCs • Stochastic Petri nets (Molloy 1977) • Markovian queueing networks (Kleinrock 1975) • Stochastic automata networks (Plateau 1985) • Stochastic activity networks (Meyer & Sanders 1985) • Stochastic process algebra (Herzog et al., Hillston 1993) • Probabilistic input/output automata (Smolka et al. 1994) • Calculi for biological systems (Priami et al., Cardelli 2002) CTMCs are one of the most prominent models in performance analysis c JPK 17 CSL: a logic for CTMCs (Aziz et al., 1996, Baier et al., 1999) • For a ∈ AP, J ⊆ [0, 1] an interval with rational bounds, and t ∈ R 0: Φ ::= a Φ∧Φ ¬Φ PJ (ϕ) ϕ ::= Φ U Φ ΦU t Φ • s0t0s1t1s2 . . . |= Φ U t Ψ if Φ holds until Ψ holds within t time units • s |= PJ (ϕ) if probability of set of ϕ-paths starting in s lies in J abbreviate P[0,0.5] (ϕ) by P 0.5 (ϕ) and P]0,1] (ϕ) by P>0 (ϕ) and so on c JPK 18 Derived operators 3Φ = true U Φ 3 t Φ = true U t Φ P p(2Φ) = P 1−p (3¬Φ) P]p,q](2 t Φ) = P[1−q,1−p[(3 t ¬Φ) c JPK 19 Timed reachability (Baier, Katoen & Hermanns, 1999) • [[ PJ (3 t Ψ) ]](s) = if and only if Pr(s |= 3 t Ψ) ∈ J • Pr(s |= 3 t Ψ) is the least solution of: – 1 if s |= Ψ – otherwise: t P(s, s , x) · Pr(s |= 3 t−x Ψ) dx 0 s ∈S • Reduction to well-studied problem allows efﬁcient, stable computation c JPK 20 Veriﬁcation times veriﬁcation time (in ms) Workstation cluster (CTMC) 10 4 Tandem queue (CTMC) Crowds protocol (DTMC) Randomised mutex (DTMC) 103 102 state space size 101 0 5⋅105 1⋅106 1.5⋅106 2⋅106 2.5⋅106 command-line tool MRMC ran on a Pentium 4, 2.66 GHz, 1 GB RAM laptop c JPK 21 Reachability probabilities Nondeterminism Nondeterminism no yes Reachability linear equation system linear programming DTMC MDP Timed reachability transient analysis greedy backward (+ uniformization) reachability CTMC uniform CTMDP c JPK 22 Challenges • Larger state spaces! – techniques to tame the state-space explosion problem • Improve diagnostic feedback! – useful, comprehensible feedback in case of property violation • Synthesis! – for which parameter values is property ensured? – for which policy is property ensured? • Hybrid systems! – evolvement often depends on discrete + continuous variables c JPK 23 Content of this talk • Probabilistic model checking – introduction, state-of-the-art, challenges • Abstraction – state-based techniques • Counterexample generation – what is a counterexample and how to obtain it? • Stochastic hybrid systems – verifying time-inhomogeneous models c JPK 24 Content of this talk • Probabilistic model checking – introduction, state-of-the-art, challenges ⇒ Abstraction – state-based techniques • Counterexample generation – what is a counterexample and how to obtain it? • Stochastic hybrid systems – verifying time-inhomogeneous models c JPK 25 Probabilistic bisimulation • . . . coincides with CSL equivalence ` ´ – s∼s ⇔ ∀Φ ∈ CSL : s |= Φ if and only if s |= Φ • . . . its coarsest quotient can be obtained in O(M · log N ) – where N = |S| and M = #non-zeros in P • . . . may be tailored to property of interest • . . . is a congruence wrt. parallel composition and hiding ⇒ . . . offers fully automated and efﬁcient abstraction • . . . but for LTL/CTL minimization effort > veriﬁcation time > c JPK 26 Probabilistic bisimulation (Kemeny & Snell, 1962), (Larsen & Skou, 1989) • Let C = (S, P, E, L) be a CTMC and R an equivalence on S • R is a strong bisimulation on S if for any (s, s ) ∈ R: L(s) = L(s ) and E(s) = E(s ) and P(s, C) = P(s , C) for all C ∈ S/R where P(s, C) = u∈C P(s, u) • s ∼ s iff ∃ a strong bisimulation R on S with (s, s ) ∈ R c JPK 27 IEEE 802.11 group communication protocol original CTMC lumped CTMC red. factor OD states transitions ver. time blocks lump + ver. time states time 4 1125 5369 121.9 71 13.5 15.9 9.00 12 37349 236313 7180 1821 642 20.5 11.2 20 231525 1590329 50133 10627 5431 21.8 9.2 28 804837 5750873 195086 35961 24716 22.4 7.9 36 2076773 15187833 5103900 91391 77694 22.7 6.6 40 3101445 22871849 7725041 135752 127489 22.9 6.1 c JPK 28 BitTorrent-like P2P protocol symmetry reduction original CTMC reduced CTMC red. factor N states ver. time states red. time ver. time states time 2 1024 5.6 528 12 2.9 1.93 0.38 3 32768 410 5984 100 59 5.48 2.58 4 1048576 22000 52360 360 820 20.0 18.3 bisimulation minimisation original CTMC lumped CTMC red. factor N states ver. time blocks lump time ver. time states time 2 1024 5.6 56 1.4 0.3 18.3 3.3 3 32768 410 252 170 1.3 130 2.4 4 1048576 22000 792 10200 4.8 1324 2.2 bisimulation may reduce a factor 66 after (manual) symmetry reduction c JPK 29 Can we abstract more? • Partition the state space into groups of abstract states – allow any partitioning, not only grouping of bisimilar states • Use a three-valued semantics – abstraction is conservative for both negative and positive veriﬁcation results – if veriﬁcation yields don’t know, validity in concrete model is unknown • Challenges: – what are abstract probabilistic models? – how to interpret PCTL/CSL on these abstract models? – how to verify abstractions? c JPK 30 Abstracting a CTMC 2 1 u1 2 3 3 3 100 2 s 5 u2 5 v 10 2 8 10 u3 10 P(Au, Av ) = { f, f , f }; note that sup F and inf F are unequal p · (1 − e−k·t ) | {z } F c JPK 31 Uniformize before abstraction! • For uniform CTMCs all exit rates are equal to k (say), and thus: pl·(1 − e−k·t) pu·(1 − e−k·t) iff pl pu – pl , pu are lower and upper bounds of time-independent transition probabilities • Recall that any CTMC C can be turned into a uniform CTMC uk (C) – in linear time while preserving CSL formulas as C ≈ u k (C) c JPK 32 What is an abstract CTMC? An abstract CTMC (ACTMC) is a quintuple C = (S, Pl , Pu, k, L) with: • Pl, Pu : S × S → [0, 1], transition probability bounds where Pl (s, S) 1 Pu(s, S) for all s ∈ S • k ∈ R>0 , the (global) exit rate for all states and • L : S × AP → { , ⊥, ? }, the labeling function The set of transition probability functions of C is { P ∈ distr(S) | Pl P Pu } c JPK 33 Abstraction ˜ For A = { A1, . . . , An } let ACTMC abstr(A, C) := (A, Pl , Pu, k, L) with: ˜ ˜ ˜l l P (Ai , Aj ) = inf P (s, Aj ) and ˜u u P (Ai , Aj ) = sup P (s, Aj ) s∈Ai s∈Ai 8 < if L(s, a) = for all s ∈ Ai ˜ and L(Ai , a) = ⊥ if L(s, a) = ⊥ for all s ∈ Ai : ? otherwise 1 4 1 [1, 1] s0 s1 4 4 3 A1 [1, 1] 4 4 1 3 3 A0 [3, 3] 4 4 4 3 4 s0 2 s2 1 [2, 3] 3 4 A2 [1, 1] 3 c JPK 34 Correctness (Katoen et al., 2007) For ACTMC C with state space S , and ACTMC abstr(A, C): s∈A implies s A for all s ∈ S, A ∈ A For states s and s of ACTMC C with s s: ∀Φ ∈ CSL : [[ Φ ]](s ) = ? implies [[ Φ ]](s) = [[ Φ ]](s ) c JPK 35 Enzyme-catalysed substrate conversion c JPK 36 A Markov chain model 2400 1 8 1 1000 1310 2301 init goal 2 3 1 6 2 1 States: enzymes 2 2 1000 1000 0220 1211 2202 substrate molecules 4 0 complex molecules 0 0 2 2 1 4 product molecules 0 4 2 1 1000 1000 0121 1112 2103 2 1 1 2 2 1 1000 1000 0022 1013 2004 1 0.001 Transitions: E + S C → E+P 1 0.001·xC e.g. (xE , xS , xC , xP ) −→ (xE + 1, xS , xC − 1, xP + 1) for xC > 0 c JPK 37 Abstraction 2400 [0, ...] 1310 2301 [0, ...] init goal States: enzymes 2 2 0220 1211 2202 [0, ...] substrate molecules 4 0 complex molecules 0 0 product molecules 0 4 0121 1112 2103 0022 1013 2004 1 0.001 Transitions: E + S C → E+P 1 0.001·xC e.g. (xE , xS , xC , xP ) −→ (xE + 1, xS , xC − 1, xP + 1) for xC > 0 c JPK 38 Experiments • Initially: enzymes 20 substrate molecules 200 • probability of converting all substrates into • vary initial substrate molecules products within time t • k = 4096 1 • ﬁxed time bound t = 20000 0,8 1 k=1024, min 0.9 k=1024, max Probability (bounds) 0.8 k=1024, diff 0,6 k=2048, min 0.7 k=2048, max Probability bounds k=2048, diff 0.6 0,4 k=4096, min 0.5 k=4096, max k=4096, diff 0.4 concrete model 0.3 0,2 0.2 0.1 0 10.000 12.000 14.000 16.000 18.000 20.000 150 200 250 300 350 400 450 500 time bound substrates c JPK 39 Results • Abstraction of enzyme-catalyzed substrate conversion – bisimulation does not yield any reduction – reduction of state space of factor 200 – reduction of veriﬁcation time with an order of magnitude • Difﬁcult analysis due to stiffness of Markov chain – standard approach needs about 6·10 7 iterations • Approximation is rather close to exact results – but approximation error cannot be estimated a priori c JPK 40 Content of this talk • Probabilistic model checking – introduction, state-of-the-art, challenges • Abstraction – state-based techniques ⇒ Counterexample generation – what is a counterexample and how to obtain it? • Stochastic hybrid systems – verifying time-inhomogeneous models c JPK 41 Counterexamples • Are of utmost importance: – diagnostic feedback, key to abstraction-reﬁnement, schedule synthesis . . . • LTL counterexamples are ﬁnite paths – 2Φ: a path ending in a ¬Φ-state – 3 Φ: a ¬Φ-path leading to a ¬Φ cycle – BFS yields shortest counterexamples • CTL counterexamples are (mostly) ﬁnite trees – universal CTL\LTL: trees or proof-like counterexample – existential CTL: witnesses, annotated counterexample • Probabilistic CTL/LTL on DTMCs: – what is a counterexample?, how to determine it?, smallest? c JPK 42 PCTL/LTL counterexamples For s |= P p(ϕ): • A counterexample C is a ﬁnite set of ﬁnite paths with Pr(C) > p – a ﬁnite path in C is called an evidence • C is a minimal counterexample if: – |C| |C | for any counterexample C • C is smallest, most indicative whenever: – C is minimal, and Pr(C) Pr(C ) for any minimal counterexample C c JPK 43 Evidences for s0 |= P 1 (a U b) 2 s0 s1 t1 0.6 0.333 evidences prob. σ1 = s0 s1 t1 0.2 σ2 = s0 s1 s2 t1 0.2 0.3 0.667 0.9 0.1 0.1 σ3 = s0 s2 t1 0.15 0.5 σ4 = s0 s1 s2 t2 0.12 u t2 σ5 = s0 s2 t2 0.09 s2 0.3 ... ... 1 0.2 1 c JPK 44 Strongest evidences (SEs) s0 s1 t1 0.6 0.333 evidences prob. σ1 = s0 s1 t1 0.2 σ2 = s0 s1 s2 t1 0.2 0.3 0.667 0.9 0.1 0.1 σ3 = s0 s2 t1 0.15 0.5 σ4 = s0 s1 s2 t2 0.12 u t2 σ5 = s0 s2 t2 0.09 s2 0.3 ... ... 1 0.2 1 c JPK 45 Counterexamples for s0 |= P 1 (a U b) 2 s0 s1 t1 0.6 0.333 evidences prob. σ1 = s0 s1 t1 0.2 0.3 0.667 0.9 σ2 = s0 s1 s2 t1 0.2 0.1 0.1 0.5 σ3 = s0 s2 t1 0.15 u t2 σ4 = s0 s1 s2 t2 0.12 s2 0.3 σ5 = s0 s2 t2 0.09 1 0.2 1 counterexample card. prob. { σ1 , . . . , σ 5 } 5 0.76 { σ1 or σ2, . . . , σ5 } 4 0.56 { σ1 , σ2 , σ4 } 3 0.52 { σ1 , σ2 , σ3 } 3 0.55 c JPK 46 Counterexamples for s0 |= P 1 (a U b) 2 s0 s1 t1 0.6 0.333 evidences prob. σ1 = s0 s1 t1 0.2 0.3 0.667 0.9 σ2 = s0 s1 s2 t1 0.2 0.1 0.1 0.5 σ3 = s0 s2 t1 0.15 u t2 σ4 = s0 s1 s2 t2 0.12 s2 0.3 σ5 = s0 s2 t2 0.09 1 0.2 1 counterexample card. prob. { σ1 , . . . , σ 5 } 5 0.76 { σ1 or σ2, . . . , σ5 } 4 0.56 minimal −→ { σ1, σ2, σ4 } 3 0.52 minimal −→ { σ1, σ2, σ3 } 3 0.55 c JPK 47 Counterexamples for s0 |= P 1 (a U b) 2 s0 s1 t1 0.6 0.333 evidences prob. σ1 = s0 s1 t1 0.2 0.3 0.667 0.9 σ2 = s0 s1 s2 t1 0.2 0.1 0.1 0.5 σ3 = s0 s2 t1 0.15 u t2 σ4 = s0 s1 s2 t2 0.12 s2 0.3 σ5 = s0 s2 t2 0.09 1 0.2 1 counterexample card. prob. { σ1 , . . . , σ 5 } 5 0.76 { σ1 or σ2, . . . , σ5 } 4 0.56 { σ1 , σ2 , σ4 } 3 0.52 smallest −→ { σ1, σ2, σ3 } 3 0.55 c JPK 48 Adapting the Markov chain s0 s1 t1 0.6 0.333 1 0.3 0.667 0.1 0.5 u t2 s2 0.3 1 0.2 1 Step 1: make all Ψ-states and all ¬Φ ∧ ¬Ψ-states absorbing c JPK 49 Adapting a bit more s0 s1 t1 0.6 0.333 1 0.3 0.667 0.1 1 0.5 1 u s2 0.3 t2 1 0.2 Step 2: insert a sink state and redirect all outgoing edges of Ψ-states to it c JPK 50 A weighted digraph s0 s1 t1 log 5 3 log 3 0 log 10 3 log 3 2 log 10 0 log 2 0 u 10 s2 3 t2 0 log 5 „ « 1 Step 3: turn it into a weighted digraph with w(s, s ) = log P(s, s ) c JPK 51 A simple derivation For ﬁnite path σ = s0 s1 s2 . . . sn: w(σ) = w(s0, s1) + w(s1, s2) + . . . + w(sn−1, sn) = log P(s1,s1) + log P(s1,s2) + . . . + log P(sn−1,sn) 0 1 1 = log P(s0,s1)·P(s1,s2)·...·P(sn−1,sn) 1 = log Pr(σ) 1 Pr(σ) Pr(σ) if and only if w(σ) w(σ) in DTMC D in digraph G(D) c JPK 52 What does this mean? (Han & Katoen, 2007) • Finding a strongest evidence is a shortest path (SP) problem – apply standard SP algorithms, or Viterbi’s algorithm ⇒ linear time complexity • Finding a shortest counterex is a k-shortest path (KSP) problem – dynamically determine k: generate C incrementally and halt when Pr(C) > p – pseudo-polynomial time complexity • This also applies to P p(ϕ) properties, as ∗ – P p (Φ U Ψ) ≡ P 1−p ((Φ ∧ ¬Ψ) W (¬Φ ∧ ¬Ψ)) ≡ P 1−p (Φ U (Ψ∗ ∨ atb)) | {z } | {z } Φ∗ Ψ∗ • These results also applicable to MDPs, CTMCs, LTL, . . . c JPK 53 Time complexity counterexample shortest path algorithm time complexity problem problem unbounded SE SP Dijkstra O(M + N · log N ) bounded h SE HSP Bellman-Ford / Viterbi O(h·M ) unbounded SC KSP Eppstein O(M + N · log N + k) bounded h SC HKSP adapted REA O (h·M + h·k· log N ) N = |S|, M = # transitions, h = hop count, k = # shortest paths including costs yields an instance of the NP-complete RSP problem c JPK 54 Representing counterexamples 1 s u 0.01 t 1 0.99 {a} A smallest counterexample for s |= P 0.9999 (3 a) contains paths su s u t, s u s u t, s u s u s u t, . . . . . . , |{z} t k times where k is the smallest integer such that 1 − 0.99 k−1 > 0.9999 Can we ﬁnd a succinct, comprehensible representation? Algorithmically? c JPK 55 A real-life case study Synchronous leader election, P (3 elected) 1.0 −→ Probability 0.8 0.6 0.4 0.2 0.0 0 10 101 102 103 104 105 106 −→ #evidences K=2 K=4 K=8 K=12 size of counterexample is double exponential in problem size c JPK 56 Use automata theory! 1 .5 (.5, v) s t s t .3 .3 (1, s) (.3, v) (.3, t) v v .7 ˆ s (.7, u) (.2, u) (1, v) .2 1 u w u w 1 (1, w) alphabet Σ consist of symbols of the form (p, s) c JPK 57 Regular expressions for counterexamples • Turn the DFA into a regular expression r (over Σ) – using the successive state elimination method – order of state removal is determined heuristically • Generate r until its probability exceeds the bound < p or p val(ε) = 1 val(r|r ) = val(r) + val(r ) val((p, s)) = p val(r.r ) = val(r) × val(r ) 1 if val(r) = 1 ∗ val(r ) = 1 1−val(r) otherwise • This yields a minimal counterexample representation c JPK 58 Content of this talk • Probabilistic model checking – introduction, state-of-the-art, challenges • Abstraction – state-based techniques • Counterexample generation – what is a counterexample and how to obtain it? ⇒ Stochastic hybrid systems – verifying time-inhomogeneous models c JPK 59 Time dependency The “rules” governing the dynamics do not change over time. But: • Embedded software unreliability – increases over time mostly due to memory leaks – decreases after a restart of the software (“software rejuvenation”) • Energy consumption – battery depletion is non-linear as extraction rate depends on energy left • Failure behaviour of hardware components – failure rates follow a bath-tub curve (child illness, constant and wear-out phase) The question is not why we should study time-dependent systems, but how! c JPK 60 Failure or hazard rates DFR CFR IFR failure rate DFR CFR IFR failure rate wearout failure period time time traditional bathtub curve the bathtub is not always ﬁlled with water c JPK 61 Time-inhomogeneous CTMCs t2 t2 t2 t2 0 1 2 3 4 3 3 3 3 0 1 0 1 0 t2 0 0 0 t2 0 0 0 0 B C B C B 3 0 t2 0 0 C B 0 2 t +3 0 0 0 C B C B C R(t)= B 0 3 0 t2 0 C E(t) = B 0 0 2 t +3 0 0 C B C B C @ 0 0 3 0 t2 A @ 0 0 0 2 t +3 0 A 0 0 0 3 0 0 0 0 0 3 A time-inhomogeneous CTMC is a quadruple (S, s 0, R(t), L) with R(s, s , t) the rate of moving from state s to s at time t c JPK 62 ICTMC semantics • Probability to leave state s in ∆t time units at time t: R ∆t − 0 E(s,t+τ ) dτ 1− e • Probability to select transition s → s at time t: ∞ Rτ − 0 E(s,t+v) dv R(s, s , t+τ ) · e dτ 0 • Probability to make transition s → s in ∆t time units at time t ∆t Rτ − 0 E(s,t+v) dv R(s, s , t+τ ) · e dτ 0 c JPK 63 CTMCs are much simpler! • Probability to leave state s in ∆t time units at time t: R ∆t − 0 E(s,t+τ ) dτ 1− e 1 − e−E(s)·∆t • Probability to select transition s → s at time t: ∞ Rτ − 0 E(s,t+v) dv R(s, s ) R(s, s , t+τ ) · e dτ 0 E(s) • Probability to make transition s → s in ∆t time units at time t ∆t Rτ R(s, s ) − 0 E(s,t+v) dv R(s, s , t+τ )·e dτ · 1 − e−E(s)·∆t 0 E(s) c JPK 64 Timed next • Let s be an ICTMC state and x the current global time • This yields a continuous state space S × R 0 • [[ PJ ( t Ψ) ]](s, x) = if and only if Pr((s, x) |= t Ψ) ∈ J t X Z Rτ − x E(s,v) dv Pr((s, x) |= Ψ) = R(s, s , τ ) · e dτ [ ]∩[0,t]⊕x [ Ψ] s ∈[ Ψ ] [ ] • How can we model check t Ψ on an inﬁnite state space? c JPK 65 Piecewise-constant ICTMCs Rτ − x E(s,v) dv Pr((s, x) |= t Ψ) = R(s, s , τ ) · e dτ s ∈[[ Ψ ]] [[ Ψ ]]∩[0,t]⊕x • Main problem: the range of the integral changes over time • Consider ICTMCs with piecewise-constant rate functions R(t) – R(s, s , τ ) and E(s, v) are constant on intervals [t k , tk+1[ R(t) 0 t1 t2 t3 t c JPK 66 Closed-form expression (Katoen & Mereacre, 2008) For any x ∈ [tk , tk+1[: Pr((s, x) |= t Ψ) = (1) (2) (3) (4) (5) (6) ak ex·b1 + ak ex·b2 + ak ex·b3 + ak ex·b4 + ak ex·b5 + ak (i) where ak and bi are constant reals c JPK 67 Solving exponential polynomials 5 (i) (6) • To solve: Pr((s, x) |= t Ψ) = p?, i.e., i=1 ak ·ex·bi + ak = p? • By Laguerre’s theorem (1883) there are at most 5 zeros N • Let f (x) = i=1 ai·ex·bi with ai, bi ∈ R and b1 < . . . < bN The number of real zeros of f is W (a 1 , . . . , aN ), and is of the same parity as W (a1 , . . . , aN ) 5 • Transform the exponential polynomial to: P (z) = i=0 ci·z ni • Solve P (z) = p using Sturm sequences and Newton’s method c JPK 68 Polynomial solver Require: polynomial P (z), interval ]a, b[ and precision = 2 −µ Ensure: z1, · · · , zm 1: 2: P (z) := derivative (P (z)) ` ´ ˆ 3: P (z) := gcd P (z), P (z) “ ” ˆ ˆ 4: P (z) := derivative P (z) “ ” 5: ˆ {F0(z), · · · , Fk (z)} := Sturm P (z), P (z) ˆ 6: {]a1 , b1 [, · · · , ]am, bm [} := Binarysearch ({F0(z), · · · , Fk (z)}, ]a, b[) 7: {z1, · · · , zm} := Newton ({]a1 , b1 [, · · · , ]am , bm[}, ) 8: return {z1, · · · , zm} c JPK 69 Time complexity The time complexity to solve P (z) = p is: O n2 log2 n(log n + s) + n log2 n |log(∆t)| + n log µ • n - degree of P (z) • s - size in bits of the coefﬁcients of P (z) in the ring of integers • ∆t - time discretization • µ - number of bits-precision for the Newton method c JPK 70 Content of this talk • Probabilistic model checking – introduction, state-of-the-art, challenges • Abstraction – state-based techniques • Counterexample generation – what is a counterexample and how to obtain it? • Stochastic hybrid systems – verifying time-inhomogeneous models c JPK 71 Other challenges • Inﬁnite state spaces – probabilistic push-down automata, regular structures • Interplay of time and nondeterminism – continuous-time MDPs • Multi-objective reachability – can we meet deadline within deadline and cost bound(s)? • Compositional reasoning – assume-guarantee reasoning c JPK 72 Probabilistic model checking • . . . . . . is a mature automated technique • . . . . . . has a broad range of applications • . . . . . . is supported by powerful software tools • . . . . . . recent signiﬁcant efﬁciency gain • . . . . . . offers many interesting challenges! more information: moves.rwth-aachen.de/˜katoen and www.mrmc-tool.org c JPK 73 Christel Baier, Boudewijn Haverkort, Holger Hermanns Tingting Han, David N. Jansen, Tim Kemna, Daniel Klink, ¨ Martin Leucker, Alexandru Mereacre, Martin Neuh ausser, ¨ Marielle Stoelinga, Marcel Oldenkamp, Verena Wolf, and Ivan Zapreev c JPK 74