Perspectives in Probabilistic Model Checking by wzq10206

VIEWS: 32 PAGES: 75

									Perspectives in Probabilistic Model Checking
                             Joost-Pieter Katoen
                  Software Modeling and Verification Group

              affiliated to University of Twente, Formal Methods and Tools




        Seminar@Chinese Academy of Science, Beijing, June 30, 2008




c JPK
                             Model checking
• Automated model-based verification and debugging technique
   – model of system = Kripke structure ≈ labeled transition system
   – properties expressed in temporal logic like LTL or CTL
   – provides counterexamples in case of property refutation

• Various striking examples
   – Needham-Schroeder protocol, cache coherence, storm surge barrier, C code

• 2008: Pioneers awarded prestigious ACM Turing Award




• Today: model checking of probabilistic models


c JPK                                                                           1
                                   Probabilities help

    • When analysing system performance and dependability
        – to quantify arrivals, waiting times, time between failure, QoS, ...

    • When modelling uncertainty in the environment
        – to quantify imprecisions in system inputs
        – to quantify unpredictable delays, express soft deadlines, ...

    • When building protocols for networked embedded systems
        – randomized algorithms

    • When problems are undecidable determinitically
        – reachability of channel systems, ...



c JPK                                                                           2
              What is probabilistic model checking?
                                                                                              up to 107 states
                        requirements        inaccuracy            system
   P 0.01 (3deadlock)                                                                                     0.8
                                                                                                             0.2

                                                                                                    0.6
                                                                                              0.4
                        Formalizing                              Modeling


                         property
                        specification                          system model



                                        Model Checking

                                                                           state 1   0.678
                                                      the probability      state 2   0.9797
                             satisfied
                                                                           state 3   0.1523
                                                                           state 4   0.2123

                                        insufficient
                                          memory




c JPK                                                                                                              3
                     Probabilistic models


                            Nondeterminism      Nondeterminism
                                  no                 yes

          Discrete time       discrete-time     Markov decision
                          Markov chain (DTMC)   process (MDP)


        Continuous time         CTMC               CTMDP




c JPK                                                             4
                              Breakthroughs

• Zero-one probabilities for Markov decision processes                       (Vardi 1985)

   – does an LTL formula hold with probability zero?

• Markov decision processes                            (Courcoubetis & Yannakakis 1988)

   – does the maximal probability for an LTL formula equal p?

• Discrete-time Markov chains                                  (Hansson & Jonsson 1990)

   – does the probability of a CTL formula equal p?

• Markov decision processes                                     (Bianco & de Alfaro 1995)

   – does the maximal probability for a CTL formula equal p?

• Continuous-time Markov chains                        (Baier, Katoen & Hermanns 1999)

   – does the probability of a timed CTL formula equal p?

c JPK                                                                                   5
                                  Characteristics

• What kind of properties?
   – CTL/LTL + (time-bounded) reachability probabilities, long-run averages

• What is inside?
   – temporal logics, model checking, numerical and OR techniques

• What is its usage?
   – powerful tools: PRISM (4,000 downloads), MRMC, Petri net tools, Probmela
   – applications: distributed systems, security, biology, avionics . . .


               tool MRMC downloadable at: www.mrmc-tool.org


c JPK                                                                           6
                     Discrete-time Markov chain

                             1
                             2

                 t                       s           1           u     1
                                                     2
                            1
                            2
                            1
                            2


                                         v     1


        a DTMC is a triple (S, P, L) with state space S and state-labelling L

 and P a stochastic matrix with P(s, s ) = one-step probability to jump from s to s



c JPK                                                                                 7
                      Markov Decision Process
                                        1
                            1           2           1
                            2                       4

                t                       s           1          u      1
                                                    2
                            1
                            2
                            1            1
                            2            4



                                        v     1



        an MDP is a DTMC if in each state there is only one color to choose




c JPK                                                                         8
                Timing




        let’s consider random timing


c JPK                                  9
                      Exponential distribution

Continuous r.v. X is exponential with rate λ > 0 if its density is

               f (x) = λ·e−λ·x   for x > 0             and 0 otherwise

Cumulative distribution of X:
                                     d
                    FX (d) =             f (x) dx = 1 − e−λ·d
                                 0


Expectation:
                                             ∞
                                                              1
                       E[X] =                    x·f (x) dx =
                                         0                    λ


c JPK                                                                    10
                      Exponential pdf and cdf


1.5                                       1
1.4                        λ = 0.5
                           λ = 1.0       0.9
1.3
                           λ = 1.5
1.2                                      0.8
1.1
                                         0.7
  1
0.9                                      0.6
0.8
                                         0.5
0.7
0.6                                      0.4
0.5
                                         0.3
0.4
0.3                                      0.2                           λ = 0.5
0.2                                                                    λ = 1.0
                                         0.1                           λ = 1.5
0.1
  0                                       0
      0   1   2      3        4      5         0   1     2      3         4      5




              the higher the rate λ, the faster the cdf approaches 1



c JPK                                                                                11
                     Continuous-Time MDP

        a CTMDP is an MDP plus an exit-rate function E : S × Act → R   0


                                    1
                          1         2 , 10      1
                          2, 4                  4 , 10

                 t                    s         1        u     1, 2
                                                2 , 25
                          1
                          2 , 25
                           1           1
                           2, 4        4 , 10



                                      v      1, 100


        note: when removing exit rates, an embedded MDP is obtained


c JPK                                                                      12
               Continuous-Time Markov chain

            a CTMC is a DTMC plus an exit-rate function E : S → R   0




                           1
                           2, 4

                  t                   s       1           u    1, 2
                                              2 , 25
                          1
                          2 , 25
                           1
                           2, 4



                                      v    1, 100


        note: when removing exit rates, an embedded DTMC is obtained


c JPK                                                                   13
           Continuous-Time Markov chain

        a CTMC is a DTMC plus an exit-rate function E : S → R     0




                        1
              4         2                                 2
              t                    s          1           u   1
                                              2
                         1        25
                         2
                        1
                        2
                                  100
                                   v     1

                                                        1
            the average residence time in state s is
                                                       E(s)


c JPK                                                                 14
                                 Uniform CTMCs

• A CTMC is uniform if E(s) = k for all s for some k ∈ R>0

• Any CTMC can be changed into a weakly bisimilar uniform CTMC

• Let k ∈ R>0 such that k            maxs∈S E(s)
   –    1
        k   is at most the shortest mean residence time in CTMC C

• Then uk (C) = (S, P, E, L) with E(s) = k for any s, and:

                        E(s)
   P(s, s ) = P(s, s )·      for s = s         and P(s, s) = 1 − P(s, S\{ s })
                         k                                                 P
                                                                    =                P(s,s )
                                                                        s ∈S\{ s }




c JPK                                                                                     15
                            Uniform CTMCs


        1           3                               1        3
 3            6     4       4              6        2   6    4     6
                                     1                                    1
                                     2                                    3

        1                                           1         2
        4           1                               4         3


                        uniformisation with k = 6




all state transitions in uk (C) occur at an average pace of k per time unit




c JPK                                                                    16
                Modeling techniques for CTMCs
• Stochastic Petri nets                                                 (Molloy 1977)


• Markovian queueing networks                                        (Kleinrock 1975)


• Stochastic automata networks                                         (Plateau 1985)


• Stochastic activity networks                               (Meyer & Sanders 1985)


• Stochastic process algebra                             (Herzog et al., Hillston 1993)


• Probabilistic input/output automata                            (Smolka et al. 1994)


• Calculi for biological systems                         (Priami et al., Cardelli 2002)


CTMCs are one of the most prominent models in performance analysis

c JPK                                                                               17
          CSL: a logic for CTMCs (Aziz et al., 1996, Baier et al., 1999)

• For a ∈ AP, J ⊆ [0, 1] an interval with rational bounds, and t ∈ R 0:


                        Φ ::= a         Φ∧Φ           ¬Φ         PJ (ϕ)

                                 ϕ ::= Φ U Φ          ΦU     t
                                                                 Φ


• s0t0s1t1s2 . . . |= Φ U    t
                                 Ψ if Φ holds until Ψ holds within t time units

• s |= PJ (ϕ) if probability of set of ϕ-paths starting in s lies in J


           abbreviate P[0,0.5] (ϕ) by P 0.5 (ϕ) and P]0,1] (ϕ) by P>0 (ϕ) and so on



c JPK                                                                                 18
             Derived operators


                       3Φ = true U Φ


                   3 t Φ = true U   t
                                        Φ


            P p(2Φ) = P         1−p (3¬Φ)



        P]p,q](2   t
                       Φ) = P[1−q,1−p[(3 t ¬Φ)




c JPK                                            19
                  Timed reachability (Baier, Katoen & Hermanns, 1999)

• [[ PJ (3   t
                 Ψ) ]](s) =         if and only if   Pr(s |= 3   t
                                                                     Ψ) ∈ J

• Pr(s |= 3       t
                      Ψ) is the least solution of:
   – 1 if s |= Ψ

   – otherwise:
                            t
                                    P(s, s , x) · Pr(s |= 3   t−x
                                                                    Ψ) dx
                           0 s ∈S


• Reduction to well-studied problem allows efficient, stable computation


c JPK                                                                         20
                                         Verification times
             verification time (in ms)


                                             Workstation cluster (CTMC)


            10
                 4                                   Tandem queue (CTMC)

                                                                           Crowds protocol (DTMC)

                                                                           Randomised mutex (DTMC)

            103




            102




                                                                                    state space size
            101
                 0




                                 5⋅105




                                             1⋅106




                                                                 1.5⋅106




                                                                                   2⋅106




                                                                                                       2.5⋅106
        command-line tool MRMC ran on a Pentium 4, 2.66 GHz, 1 GB RAM laptop


c JPK                                                                                                            21
                     Reachability probabilities


                                Nondeterminism         Nondeterminism
                                      no                    yes

             Reachability    linear equation system   linear programming
                                      DTMC                    MDP


        Timed reachability     transient analysis      greedy backward
                               (+ uniformization)         reachability
                                     CTMC              uniform CTMDP




c JPK                                                                      22
                                      Challenges
        • Larger state spaces!
          – techniques to tame the state-space explosion problem

        • Improve diagnostic feedback!
          – useful, comprehensible feedback in case of property violation

        • Synthesis!
          – for which parameter values is property ensured?
          – for which policy is property ensured?

        • Hybrid systems!
          – evolvement often depends on discrete + continuous variables


c JPK                                                                       23
                         Content of this talk

        • Probabilistic model checking
          – introduction, state-of-the-art, challenges

        • Abstraction
          – state-based techniques

        • Counterexample generation
          – what is a counterexample and how to obtain it?

        • Stochastic hybrid systems
          – verifying time-inhomogeneous models




c JPK                                                        24
                          Content of this talk

         • Probabilistic model checking
           – introduction, state-of-the-art, challenges

        ⇒ Abstraction
           – state-based techniques

         • Counterexample generation
           – what is a counterexample and how to obtain it?

         • Stochastic hybrid systems
           – verifying time-inhomogeneous models




c JPK                                                         25
                       Probabilistic bisimulation

 • . . . coincides with CSL equivalence
                `                                             ´
    – s∼s ⇔         ∀Φ ∈ CSL : s |= Φ if and only if s |= Φ

 • . . . its coarsest quotient can be obtained in O(M · log N )
    – where N = |S| and M = #non-zeros in P

 • . . . may be tailored to property of interest

 • . . . is a congruence wrt. parallel composition and hiding

⇒ . . . offers fully automated and efficient abstraction

 • . . . but for LTL/CTL minimization effort > verification time
                                              >

 c JPK                                                            26
        Probabilistic bisimulation (Kemeny & Snell, 1962), (Larsen & Skou, 1989)



• Let C = (S, P, E, L) be a CTMC and R an equivalence on S

• R is a strong bisimulation on S if for any (s, s ) ∈ R:

                   L(s) = L(s )       and       E(s) = E(s )         and
                      P(s, C) = P(s , C)         for all     C ∈ S/R
                           where P(s, C)    =      u∈C     P(s, u)


• s ∼ s iff ∃ a strong bisimulation R on S with (s, s ) ∈ R


c JPK                                                                              27
         IEEE 802.11 group communication protocol


                   original CTMC                    lumped CTMC              red. factor
   OD     states    transitions    ver. time    blocks   lump + ver. time   states   time
    4       1125          5369          121.9       71               13.5     15.9   9.00
    12     37349      236313         7180        1821              642       20.5    11.2
    20   231525      1590329        50133       10627             5431       21.8      9.2
    28   804837      5750873       195086       35961           24716        22.4      7.9
    36   2076773    15187833       5103900      91391           77694        22.7      6.6
    40   3101445    22871849       7725041      135752         127489        22.9      6.1




c JPK                                                                                        28
                         BitTorrent-like P2P protocol

                                                       symmetry reduction
              original CTMC                      reduced CTMC                 red. factor
         N     states      ver. time    states    red. time    ver. time    states   time
         2        1024            5.6     528            12           2.9     1.93    0.38
         3      32768         410        5984           100         59        5.48    2.58
         4    1048576      22000        52360           360       820       20.0     18.3


                                                    bisimulation minimisation
              original CTMC                      lumped CTMC                  red. factor
         N     states     ver. time     blocks    lump time    ver. time    states     time
         2        1024           5.6        56           1.4          0.3     18.3       3.3
         3      32768        410           252        170             1.3    130         2.4
         4    1048576     22000            792      10200             4.8   1324         2.2


        bisimulation may reduce a factor 66 after (manual) symmetry reduction


c JPK                                                                                          29
                         Can we abstract more?

• Partition the state space into groups of abstract states
   – allow any partitioning, not only grouping of bisimilar states

• Use a three-valued semantics
   – abstraction is conservative for both negative and positive verification results
   – if verification yields don’t know, validity in concrete model is unknown

• Challenges:
   – what are abstract probabilistic models?
   – how to interpret PCTL/CSL on these abstract models?
   – how to verify abstractions?



c JPK                                                                                 30
                              Abstracting a CTMC

                      2
            1        u1           2
            3                     3


                 3
                     100      2
        s        5   u2       5
                                       v

                     10
             2                     8
            10       u3           10




 P(Au, Av ) = { f, f , f }; note that sup F and inf F are unequal p · (1 − e−k·t )
              |    {z    }
                          F




c JPK                                                                            31
                  Uniformize before abstraction!


• For uniform CTMCs all exit rates are equal to k (say), and thus:

                 pl·(1 − e−k·t)       pu·(1 − e−k·t) iff     pl    pu

   – pl , pu are lower and upper bounds of time-independent transition probabilities

• Recall that any CTMC C can be turned into a uniform CTMC uk (C)
   – in linear time while preserving CSL formulas as C ≈ u k (C)




c JPK                                                                              32
                       What is an abstract CTMC?

An abstract CTMC (ACTMC) is a quintuple C = (S, Pl , Pu, k, L) with:

• Pl, Pu : S × S → [0, 1], transition probability bounds where

                        Pl (s, S)       1      Pu(s, S) for all s ∈ S


• k ∈ R>0 , the (global) exit rate for all states and

• L : S × AP → { , ⊥, ? }, the labeling function


        The set of transition probability functions of C is { P ∈ distr(S) | Pl   P   Pu }




c JPK                                                                                        33
                                             Abstraction
                                                                    ˜
For A = { A1, . . . , An } let ACTMC abstr(A, C) := (A, Pl , Pu, k, L) with:
                                                        ˜ ˜

        ˜l                  l
        P (Ai , Aj ) = inf P (s, Aj )            and   ˜u                  u
                                                       P (Ai , Aj ) = sup P (s, Aj )
                           s∈Ai                                        s∈Ai

                   8
                   <            if L(s, a) =   for all s ∈ Ai
    ˜
and L(Ai , a) =        ⊥        if L(s, a) = ⊥ for all s ∈ Ai
                   :
                       ?       otherwise

                           1
                           4                 1                [1, 1]
              s0                    s1
                                             4                 4 3     A1      [1, 1]
                                                                                4 4
                                1
                                3
                                         3              A0                  [3, 3]
                                         4                                   4 4
                                3
                                4
              s0           2
                                    s2       1                [2, 3]
                                                               3 4
                                                                       A2      [1, 1]
                           3



c JPK                                                                                   34
                    Correctness (Katoen et al., 2007)



        For ACTMC C with state space S , and ACTMC abstr(A, C):

              s∈A      implies   s   A for all s ∈ S, A ∈ A




               For states s and s of ACTMC C with s       s:

        ∀Φ ∈ CSL : [[ Φ ]](s ) = ?   implies   [[ Φ ]](s) = [[ Φ ]](s )




c JPK                                                                     35
        Enzyme-catalysed substrate conversion




c JPK                                           36
                                             A Markov chain model

        2400

     1     8
                 1
               1000
        1310          2301
                                                                                                               init   goal
     2     3          1       6
                 2                  1
                                                                               States:   enzymes                 2       2
               1000               1000
        0220          1211               2202                                            substrate molecules     4       0
                                                                                         complex molecules       0       0
                      2       2          1   4
                                                                                         product molecules       0       4
                                    2              1
                                  1000           1000
                      0121               1112           2103

                                         2   1          1    2
                                                   2               1
                                                 1000            1000
                                         0022           1013            2004


                          1         0.001
Transitions: E + S            C → E+P
                          1
                                                   0.001·xC
               e.g. (xE , xS , xC , xP )                −→       (xE + 1, xS , xC − 1, xP + 1) for xC > 0


c JPK                                                                                                                 37
                                                      Abstraction

        2400                [0, ...]



        1310        2301                   [0, ...]
                                                                                                     init   goal
                                                                     States:   enzymes                 2       2
        0220        1211          2202                  [0, ...]               substrate molecules     4       0
                                                                               complex molecules       0       0
                                                                               product molecules       0       4
                    0121          1112           2103



                                  0022           1013         2004


                       1     0.001
Transitions: E + S         C → E+P
                       1
                                            0.001·xC
               e.g. (xE , xS , xC , xP )       −→       (xE + 1, xS , xC − 1, xP + 1) for xC > 0


c JPK                                                                                                       38
                                                                   Experiments
   • Initially:    enzymes                20
                   substrate molecules   200
   • probability of converting all substrates into
                                                   • vary initial substrate molecules
     products within time t
                                                   • k = 4096
                           1                                                          • fixed time bound t = 20000

                          0,8                                                                                     1

                                                                     k=1024, min                                 0.9
                                                                     k=1024, max
   Probability (bounds)




                                                                                                                 0.8
                                                                     k=1024, diff
                          0,6
                                                                     k=2048, min                                 0.7
                                                                     k=2048, max




                                                                                            Probability bounds
                                                                     k=2048, diff                                0.6

                          0,4                                        k=4096, min                                 0.5
                                                                     k=4096, max
                                                                     k=4096, diff                                0.4
                                                                     concrete model                              0.3
                          0,2
                                                                                                                 0.2

                                                                                                                 0.1

                                                                                                                  0
                          10.000   12.000   14.000        16.000    18.000       20.000                           150   200   250   300     350   400   450   500
                                                 time bound                                                                          substrates




c JPK                                                                                                                                                               39
                                    Results

• Abstraction of enzyme-catalyzed substrate conversion
   – bisimulation does not yield any reduction
   – reduction of state space of factor 200
   – reduction of verification time with an order of magnitude

• Difficult analysis due to stiffness of Markov chain
   – standard approach needs about 6·10 7 iterations

• Approximation is rather close to exact results
   – but approximation error cannot be estimated a priori




c JPK                                                           40
                          Content of this talk

        • Probabilistic model checking
           – introduction, state-of-the-art, challenges

        • Abstraction
           – state-based techniques

        ⇒ Counterexample generation
           – what is a counterexample and how to obtain it?

        • Stochastic hybrid systems
           – verifying time-inhomogeneous models




c JPK                                                         41
                            Counterexamples
• Are of utmost importance:
   – diagnostic feedback, key to abstraction-refinement, schedule synthesis . . .

• LTL counterexamples are finite paths
   – 2Φ: a path ending in a ¬Φ-state
   – 3 Φ: a ¬Φ-path leading to a ¬Φ cycle
   – BFS yields shortest counterexamples

• CTL counterexamples are (mostly) finite trees
   – universal CTL\LTL: trees or proof-like counterexample
   – existential CTL: witnesses, annotated counterexample

• Probabilistic CTL/LTL on DTMCs:
   – what is a counterexample?, how to determine it?, smallest?


c JPK                                                                              42
                    PCTL/LTL counterexamples

For s |= P p(ϕ):

• A counterexample C is a finite set of finite paths with Pr(C) > p
   – a finite path in C is called an evidence

• C is a minimal counterexample if:
   – |C|     |C | for any counterexample C

• C is smallest, most indicative whenever:
   – C is minimal, and Pr(C)      Pr(C ) for any minimal counterexample C




c JPK                                                                       43
                            Evidences for s0 |= P 1 (a U b)
                                                                 2



            s0                    s1                   t1
                      0.6                  0.333                     evidences          prob.
                                                                     σ1 = s0 s1 t1      0.2
                                                                     σ2 = s0 s1 s2 t1   0.2
                       0.3         0.667   0.9
                0.1                                     0.1          σ3 = s0 s2 t1      0.15
                                                 0.5
                                                                     σ4 = s0 s1 s2 t2   0.12
        u                                                   t2       σ5 = s0 s2 t2      0.09
                             s2            0.3
                                                                     ...                ...
            1                     0.2                   1




c JPK                                                                                           44
                            Strongest evidences (SEs)


            s0                    s1                   t1
                      0.6                  0.333                 evidences          prob.
                                                                 σ1 = s0 s1 t1      0.2
                                                                 σ2 = s0 s1 s2 t1   0.2
                       0.3         0.667   0.9
                0.1                                     0.1      σ3 = s0 s2 t1      0.15
                                                 0.5
                                                                 σ4 = s0 s1 s2 t2   0.12
        u                                                   t2   σ5 = s0 s2 t2      0.09
                             s2            0.3
                                                                 ...                ...
            1                     0.2                   1




c JPK                                                                                       45
                  Counterexamples for s0 |= P 1 (a U b)
                                                                                 2
            s0                    s1                       t1
                      0.6                      0.333                           evidences          prob.
                                                                               σ1 = s0 s1 t1      0.2
                       0.3         0.667      0.9                              σ2 = s0 s1 s2 t1   0.2
                0.1                                         0.1
                                                     0.5                       σ3 = s0 s2 t1      0.15

        u                                                       t2
                                                                               σ4 = s0 s1 s2 t2   0.12
                             s2                0.3
                                                                               σ5 = s0 s2 t2      0.09
            1                     0.2                       1

                                       counterexample                card.   prob.
                                        { σ1 , . . . , σ 5 }         5       0.76
                              { σ1 or σ2, . . . , σ5 }               4       0.56
                                          { σ1 , σ2 , σ4 }           3       0.52
                                          { σ1 , σ2 , σ3 }           3       0.55


c JPK                                                                                                     46
                  Counterexamples for s0 |= P 1 (a U b)
                                                                           2
            s0                    s1                    t1
                      0.6                   0.333                         evidences          prob.
                                                                          σ1 = s0 s1 t1      0.2
                       0.3         0.667   0.9                            σ2 = s0 s1 s2 t1   0.2
                0.1                                      0.1
                                                  0.5                     σ3 = s0 s2 t1      0.15

        u                                                    t2
                                                                          σ4 = s0 s1 s2 t2   0.12
                             s2             0.3
                                                                          σ5 = s0 s2 t2      0.09
            1                     0.2                    1

                                        counterexample            card.   prob.
                                         { σ1 , . . . , σ 5 }     5       0.76
                                  { σ1 or σ2, . . . , σ5 }        4       0.56
                        minimal −→ { σ1, σ2, σ4 }                 3       0.52
                        minimal −→ { σ1, σ2, σ3 }                 3       0.55


c JPK                                                                                                47
                  Counterexamples for s0 |= P 1 (a U b)
                                                                          2
            s0                    s1                   t1
                      0.6                  0.333                         evidences          prob.
                                                                         σ1 = s0 s1 t1      0.2
                       0.3         0.667   0.9                           σ2 = s0 s1 s2 t1   0.2
                0.1                                     0.1
                                                 0.5                     σ3 = s0 s2 t1      0.15

        u                                                   t2
                                                                         σ4 = s0 s1 s2 t2   0.12
                             s2            0.3
                                                                         σ5 = s0 s2 t2      0.09
            1                     0.2                   1

                                        counterexample           card.    prob.
                                         { σ1 , . . . , σ 5 }    5        0.76
                                   { σ1 or σ2, . . . , σ5 }      4        0.56
                                           { σ1 , σ2 , σ4 }      3        0.52
                        smallest −→ { σ1, σ2, σ3 }               3        0.55


c JPK                                                                                               48
                 Adapting the Markov chain

                    s0                    s1                   t1
                              0.6                  0.333
                                                                         1



                               0.3         0.667
                        0.1
                                                         0.5

                u                                                   t2
                                     s2            0.3

                    1                     0.2                   1




        Step 1: make all Ψ-states and all ¬Φ ∧ ¬Ψ-states absorbing



c JPK                                                                        49
                                Adapting a bit more

                     s0                    s1                    t1
                               0.6                  0.333
                                                                      1

                                0.3         0.667
                         0.1                                                 1
                                                          0.5
                                                                      1
                 u
                                      s2            0.3
                                                                t2
                     1                     0.2




        Step 2: insert a sink state and redirect all outgoing edges of Ψ-states to it



c JPK                                                                                   50
                                  A weighted digraph

                   s0                        s1                        t1
                                log 5
                                    3                  log 3
                                                                            0

                                   log 10
                                       3       log 3
                                                   2
                       log 10                                                           0
                                                               log 2
                                                                            0
               u                                        10
                                        s2              3
                                                                   t2
                   0                         log 5


                                                                                „              «
                                                                                      1
        Step 3: turn it into a weighted digraph with w(s, s ) = log
                                                                                    P(s, s )



c JPK                                                                                              51
                          A simple derivation
        For finite path σ = s0 s1 s2 . . . sn:

        w(σ) = w(s0, s1) + w(s1, s2) + . . . + w(sn−1, sn)

              = log P(s1,s1) + log P(s1,s2) + . . . + log P(sn−1,sn)
                       0              1
                                                              1



              = log P(s0,s1)·P(s1,s2)·...·P(sn−1,sn)
                                   1



              = log Pr(σ)
                      1




          Pr(σ)      Pr(σ)     if and only if     w(σ)       w(σ)
              in DTMC D                            in digraph G(D)



c JPK                                                                  52
                What does this mean?                 (Han & Katoen, 2007)



• Finding a strongest evidence is a shortest path (SP) problem
   – apply standard SP algorithms, or Viterbi’s algorithm     ⇒ linear time complexity

• Finding a shortest counterex is a k-shortest path (KSP) problem
   – dynamically determine k: generate C incrementally and halt when Pr(C) > p
   – pseudo-polynomial time complexity

• This also applies to P p(ϕ) properties, as
                                                                            ∗
   – P p (Φ U Ψ) ≡ P    1−p ((Φ   ∧ ¬Ψ) W (¬Φ ∧ ¬Ψ)) ≡ P           1−p (Φ       U (Ψ∗ ∨ atb))
                            |     {z }    |   {z  }
                                  Φ∗            Ψ∗


• These results also applicable to MDPs, CTMCs, LTL, . . .


c JPK                                                                                      53
                              Time complexity


 counterexample     shortest path         algorithm             time complexity
    problem           problem

   unbounded   SE        SP                Dijkstra           O(M + N · log N )
   bounded h   SE       HSP         Bellman-Ford / Viterbi         O(h·M )
   unbounded   SC       KSP               Eppstein           O(M + N · log N + k)
   bounded h   SC      HKSP             adapted REA          O (h·M + h·k· log N )


         N = |S|, M = # transitions, h = hop count, k = # shortest paths


        including costs yields an instance of the NP-complete RSP problem


c JPK                                                                             54
                   Representing counterexamples
                                   1
                        s                     u        0.01
                                                                     t        1
                                 0.99
                                                                  {a}



           A smallest counterexample for s |= P       0.9999 (3 a)   contains paths

                                                                     su
                     s u t, s u s u t, s u s u s u t, . . . . . . , |{z} t
                                                                k times


           where k is the smallest integer such that 1 − 0.99 k−1 > 0.9999

        Can we find a succinct, comprehensible representation? Algorithmically?


c JPK                                                                                 55
                                 A real-life case study
                           Synchronous leader election, P (3 elected)
                         1.0
        −→ Probability

                         0.8
                         0.6
                         0.4
                         0.2
                         0.0 0
                           10    101    102 103 104        105   106
                                        −→ #evidences
               K=2                 K=4         K=8       K=12
        size of counterexample is double exponential in problem size

c JPK                                                                   56
                            Use automata theory!


    1             .5                                            (.5, v)
        s                    t                  s                                    t
             .3        .3             (1, s)          (.3, v)             (.3, t)

                  v                                                v
        .7                        ˆ
                                  s       (.7, u)   (.2, u)                 (1, v)
             .2        1
        u                    w                  u                                    w
                  1                                             (1, w)




                  alphabet Σ consist of symbols of the form (p, s)




c JPK                                                                                    57
         Regular expressions for counterexamples
• Turn the DFA into a regular expression r (over Σ)
   – using the successive state elimination method
   – order of state removal is determined heuristically

• Generate r until its probability exceeds the bound < p or       p

                     val(ε) = 1 val(r|r ) = val(r) + val(r )

                val((p, s)) = p val(r.r ) = val(r) × val(r )

                                     1            if val(r) = 1
                            ∗
                      val(r ) =          1
                                      1−val(r)    otherwise

• This yields a minimal counterexample representation

c JPK                                                                 58
                          Content of this talk

         • Probabilistic model checking
           – introduction, state-of-the-art, challenges

         • Abstraction
           – state-based techniques

         • Counterexample generation
           – what is a counterexample and how to obtain it?

        ⇒ Stochastic hybrid systems
           – verifying time-inhomogeneous models




c JPK                                                         59
                             Time dependency
The “rules” governing the dynamics do not change over time. But:

• Embedded software unreliability
   – increases over time mostly due to memory leaks
   – decreases after a restart of the software (“software rejuvenation”)

• Energy consumption
   – battery depletion is non-linear as extraction rate depends on energy left

• Failure behaviour of hardware components
   – failure rates follow a bath-tub curve (child illness, constant and wear-out phase)

    The question is not why we should study time-dependent systems, but how!


c JPK                                                                                60
                                    Failure or hazard rates


                                                                          DFR    CFR        IFR




                                                           failure rate
               DFR         CFR           IFR
failure rate




                                                 wearout
                                                 failure
                                                 period

                                                                                                      time
                                          time
                     traditional bathtub curve                                  the bathtub is not always
                                                                                     filled with water




c JPK                                                                                                        61
                        Time-inhomogeneous CTMCs

                                      t2        t2       t2        t2
                                 0         1         2         3        4
                                      3         3        3         3
        0                             1                  0                                    1
            0     t2   0    0    0                            t2     0         0     0    0
      B                               C               B                                       C
      B     3     0    t2   0    0    C               B       0     2
                                                                   t +3        0     0    0   C
      B                               C               B                                       C
R(t)= B     0     3    0    t2   0    C        E(t) = B       0      0       2
                                                                            t +3     0    0   C
      B                               C               B                                       C
      @     0     0    3    0    t2   A               @       0      0         0    2
                                                                                   t +3   0   A
            0     0    0    3    0                            0      0         0     0    3




                A time-inhomogeneous CTMC is a quadruple (S, s 0, R(t), L)
                 with R(s, s , t) the rate of moving from state s to s at time t


c JPK                                                                                             62
                             ICTMC semantics
• Probability to leave state s in ∆t time units at time t:
                                        R ∆t
                                       − 0 E(s,t+τ ) dτ
                                 1−   e

• Probability to select transition s → s at time t:
                         ∞                        Rτ
                                                 − 0 E(s,t+v) dv
                             R(s, s , t+τ ) ·   e                  dτ
                     0


• Probability to make transition s → s in ∆t time units at time t
                        ∆t                        Rτ
                                                 − 0 E(s,t+v) dv
                             R(s, s , t+τ ) ·   e                  dτ
                    0



c JPK                                                                   63
                             CTMCs are much simpler!
• Probability to leave state s in ∆t time units at time t:
                              R ∆t
                             − 0 E(s,t+τ ) dτ
                      1−    e                                       1 − e−E(s)·∆t

• Probability to select transition s → s at time t:
                     ∞                        Rτ
                                             − 0 E(s,t+v) dv                R(s, s )
                         R(s, s , t+τ ) ·   e                  dτ
                 0                                                           E(s)

• Probability to make transition s → s in ∆t time units at time t
        ∆t                      Rτ                                  R(s, s )
                               − 0 E(s,t+v) dv
             R(s, s   , t+τ )·e                    dτ                        · 1 − e−E(s)·∆t
    0                                                                E(s)


c JPK                                                                                      64
                                     Timed next


• Let s be an ICTMC state and x the current global time

• This yields a continuous state space S × R                      0


• [[ PJ (    t
                 Ψ) ]](s, x) =        if and only if          Pr((s, x) |=           t
                                                                                         Ψ) ∈ J
                          t
                                      X Z                                        Rτ
                                                                                − x E(s,v) dv
        Pr((s, x) |=          Ψ) =                            R(s, s , τ ) ·   e                dτ
                                                [ ]∩[0,t]⊕x
                                                [ Ψ]
                                     s ∈[ Ψ ]
                                        [ ]



• How can we model check                    t
                                                Ψ on an infinite state space?




c JPK                                                                                                65
                       Piecewise-constant ICTMCs

                                                                                 Rτ
                                                                                − x E(s,v) dv
 Pr((s, x) |=      t
                       Ψ) =                                   R(s, s , τ ) ·   e                dτ
                              s ∈[[ Ψ ]]    [[ Ψ ]]∩[0,t]⊕x


• Main problem: the range of the integral changes over time

• Consider ICTMCs with piecewise-constant rate functions R(t)
   – R(s, s , τ ) and E(s, v) are constant on intervals [t k , tk+1[
                          R(t)




                              0        t1      t2     t3           t


c JPK                                                                                                66
         Closed-form expression (Katoen & Mereacre, 2008)



                          For any x ∈ [tk , tk+1[:
                           Pr((s, x) |=      t
                                                  Ψ)
                                        =
         (1)        (2)           (3)       (4)        (5)      (6)
        ak ex·b1 + ak ex·b2 + ak ex·b3 + ak ex·b4 + ak ex·b5 + ak
                            (i)
                    where ak and bi are constant reals




c JPK                                                                 67
                Solving exponential polynomials

                                                      5    (i)           (6)
• To solve: Pr((s, x) |=        t
                                    Ψ) = p?, i.e.,    i=1 ak ·ex·bi   + ak     = p?

• By Laguerre’s theorem (1883) there are at most 5 zeros

                 N
• Let f (x) =    i=1 ai·ex·bi   with ai, bi ∈ R and b1 < . . . < bN

            The number of real zeros of f is W (a 1 , . . . , aN ), and is
                    of the same parity as W (a1 , . . . , aN )

                                                                 5
• Transform the exponential polynomial to: P (z) =               i=0 ci·z ni

• Solve P (z) = p using Sturm sequences and Newton’s method


c JPK                                                                             68
                                 Polynomial solver

Require: polynomial P (z), interval ]a, b[ and precision          = 2 −µ
Ensure: z1, · · · , zm
 1:
 2: P (z) := derivative (P (z))
                `             ´
    ˆ
 3: P (z) := gcd P (z), P (z)
                        “      ”
    ˆ                     ˆ
 4: P (z) := derivative P (z)
                                            “             ”
 5:                                           ˆ
      {F0(z), · · · , Fk (z)} := Sturm P (z), P (z) ˆ
 6:   {]a1 , b1 [, · · · , ]am, bm [} := Binarysearch ({F0(z), · · · , Fk (z)}, ]a, b[)
 7:   {z1, · · · , zm} := Newton ({]a1 , b1 [, · · · , ]am , bm[}, )
 8:   return {z1, · · · , zm}




c JPK                                                                                     69
                                 Time complexity


The time complexity to solve P (z) = p is:

             O n2 log2 n(log n + s) + n log2 n |log(∆t)| + n log µ

•   n - degree of P (z)
•   s - size in bits of the coefficients of P (z) in the ring of integers
•   ∆t - time discretization
•   µ - number of bits-precision for the Newton method




c JPK                                                                      70
                         Content of this talk

        • Probabilistic model checking
          – introduction, state-of-the-art, challenges

        • Abstraction
          – state-based techniques

        • Counterexample generation
          – what is a counterexample and how to obtain it?

        • Stochastic hybrid systems
          – verifying time-inhomogeneous models




c JPK                                                        71
                              Other challenges
        • Infinite state spaces
          – probabilistic push-down automata, regular structures

        • Interplay of time and nondeterminism
          – continuous-time MDPs

        • Multi-objective reachability
          – can we meet deadline within deadline and cost bound(s)?

        • Compositional reasoning
          – assume-guarantee reasoning


c JPK                                                                 72
                    Probabilistic model checking

          • . . . . . . is a mature automated technique

          • . . . . . . has a broad range of applications

          • . . . . . . is supported by powerful software tools

          • . . . . . . recent significant efficiency gain

          • . . . . . . offers many interesting challenges!


        more information: moves.rwth-aachen.de/˜katoen and www.mrmc-tool.org




c JPK                                                                          73
               Christel Baier, Boudewijn Haverkort, Holger Hermanns
              Tingting Han, David N. Jansen, Tim Kemna, Daniel Klink,
                                                              ¨
              Martin Leucker, Alexandru Mereacre, Martin Neuh ausser,
            ¨
        Marielle Stoelinga, Marcel Oldenkamp, Verena Wolf, and Ivan Zapreev




c JPK                                                                         74

								
To top