rng

Document Sample
rng Powered By Docstoc
					Random Number Generators

       Daniel Pierre
        Alice Lin
         Yuan Ji

        Random Number Generators   1
   Different types of Random Numbers

• Truly random - is defined as exhibiting ``true'' randomness,
  such as the time between “ tics” from a Geiger counter
  exposed to a radioactive element.
• Pseudorandom - is defined as having the appearance of
  randomness, but nevertheless exhibiting a specific,
  repeatable pattern.
• Quasi-random - is defined as filling the solution space
  sequentially (in fact, these sequences are not at all random
  - they are just comprehensive at a preset level of
  granularity).

                      Random Number Generators               2
True Random Number Generators
• Many true random number generators are hardware
  solutions that you plug to a computer.
• The usual method is to amplify noise generated by a
  resistor (Johnson noise) or a semi-conductor diode and
  feed this to a comparator or Schmitt trigger. Once you
  sample the output, you get a series of bits which can
  be used to generate random numbers.
• True random number generators can be used for
  research, modeling, encryption, lottery prediction and
  parapsychological testing, among many other uses.

                    Random Number Generators           3
         True Random Number
A PCI version (rw4) generating true random numbers at 160 MByte/s:




                       Random Number Generators                      4
Pseudorandom Numbers




      Random Number Generators   5
How to obtain quasi-# generator
• If we change our generator so as to maintain a
  nearly uniform density of coverage of the domain
  then we have a random number generator known
  as quasi-random number generator.
• Quasi-random numbers give up serial
  independence of subsequently generated values in
  order to obtain as uniform as possible coverage of
  the domain. This avoids clusters and voids in the
  pattern of a finite set of selected points.

                  Random Number Generators         6
Quasi-random Numbers




      Random Number Generators   7
Random Number Generators
      for Parallel Computers
Summary
Introduction
Testing and Evaluation of Random
 Number Generators
Random Number Generators
Parallel Random Number Generators
Conclusions
           Random Number Generators   8
                 Summary
                   (continue )
Random number generators use iterative
deterministic algorithms for producing a sequence of
pseudo-random numbers that approximate a truly
random sequence. Ideally the sequence should be
uniformly distributed, uncorrelated, reproduceable,
portable, easily changed by adjusting an initial seed
value, easily split into many independent
subsequences, have a large period of repetition, pass
all empirical tests for randomness, and be generated
rapidly using limited computer memory.

                  Random Number Generators          9
               Summary
                 (continue)

  The main algorithms used for sequential
random number generators are the following:
     •   Linear congruential generators
     •   Lagged Fibonacci generators
     •   Shift register generators
     •   Combined generators
                Random Number Generators    10
              Summary
                (continue)
 The main techniques used for parallelizing
random number generators in the following
             different ways:

      • Leapfrog
      • Sequence splitting
      • Independent sequences
               Random Number Generators       11
                        Summary
                          (continue)
      On a sequential computer, good generators to use are:
•   a multiplicative lagged Fibonacci generator with a lag of at
    least 127, and preferably 1279 or more;
•   a 48-bit or preferably 64-bit linear congruential generator,
    that performs well in the Spectral Test and has a prime
    modulus;
•   a 32-bit (or more) combined linear congruential generator,
    with well-chosen parameters, such as those recommended
    by L'Ecuyer;
•   if speed is really crucial, an additive lagged Fibonacci
    generator with a lag of at least 1279 and preferably much
    greater, and possibly combined with another generator, as
    in RANMAR, or using 3 or more lags rather than 2.
                        Random Number Generators              12
              Summary
                 (continue)
Recommended generators to use on a
     parallel computer are:
 • Combined linear congruential generators
   using sequence splitting;
 • Lagged Fibonacci generators using
   independent sequences, with careful
   initialization to ensure the seed tables on
   each processor are random and
   uncorrelated.
              Random Number Generators      13
            Introduction
Random number generators are widely used
for simulations in computational science and
engineering. Randomness is often present in
the formulation of the problem, for example
random noise or perturbations, and quantum
processes.
Since ``random'' numbers are in practice
computed using deterministic algorithms,
these are more accurately called pseudo-
random number generators.
               Random Number Generators    14
 Testing and Evaluation of
Random Number Generators
• Requirements for Sequential Random
  Number Generators
• Tests for Sequential Random Number
  Generators
• Requirements for Parallel Random
  Number Generators
• Tests for Parallel Random Number
  Generators Random Number Generators   15
     Requirements for Sequential Random
            Number Generators
Ideally a pseudo-random number generator
would produce a stream of numbers that
 •   are uniformly distributed,
 •   are uncorrelated,
 •   never repeats itself,
 •   satisfy any statistical test for randomness,
 •   are reproduceable (for debugging purposes),
 •   are portable (the same on any computer),
 •   can be changed by adjusting an initial ``seed'’ value,
 •   can easily be split into many independent subsequences,
 •   can be generated rapidly using limited computer memory.
                    Random Number Generators          16
  Tests for Sequential Random Number
               Generators
Many standard tests of this kind are available. In
addition to standard statistical tests, it is useful to
apply application-specific tests that are more
relevant to some of the various applications for
which random numbers are used. As with the
statistical tests, these tests generally compare the
results obtained using a pseudo-random number
generator with known exact results that would
occur if the numbers were truly random.
                    Random Number Generators        17
        Requirements for Parallel Random
                  Number Generators
• The generator should work for any number of processors.
• The sequences of random numbers generated on each processor
  should all satisfy the requirements of a good sequential
  generator,
• There should be no correlations between the sequences on
  different processors.
• The same sequence of random numbers should be produced for
  different numbers of processors, and for the special case of a
  single processor.
• The algorithm should be efficient, which in practice means
  there should be no data movement between processors. Thus,
  after the generator is initialized, each processor should generate
                          Random of the other
  its sequence independentlyNumber Generators processors.     18
    Tests for Parallel Random Number
                Generators
The many standard statistical tests for checking the
randomness properties of sequential generators can be
applied to parallel generators, by testing the random
number streams on each processor, and from all
processors combined. This is the usual approach in
testing parallel generators. However, new techniques
are necessary to test algorithms for generating random
numbers on parallel computers, for example to look
for correlations between random number streams on
different processors. Thus far, very little work has
been done in this area. Number Generators
                      Random                         19
Random Number Generators
•   Linear Congruential Generators
•   Lagged Fibonnaci Generators
•   Shift Register Generators
•   Combined Generators

            Random Number Generators   20
        Linear Congruential Generators
Probably the most commonly-used random number
generators are linear congruential generators (LCGs).
The standard C and Unix generators RAND (32-bit precision),
DRAND48 and RANF (48-bit precision) are of this type. LCGs
produce a sequence Xi of random integers using the relation
                  Xi =( a * X i-1 + c) mod M,
where a is known as the multiplier, M is the modulus, and c is an
additive constant that may be set to zero. The parameters
(a,c,M) must be chosen carefully to ensure a large period and good
uniformity and randomness properties. The maximum period is M
for suitably chosen parameters (M-1 if c=0). Standard random
number generators return a floating point number xi in the interval
[0,1), which can be obtained by dividing Xi by M.
                          Random Number Generators              21
             Lagged Fibonnaci Generators
Lagged Fibonacci generators (LFGs) are becoming increasingly
popular, since they offer a simple method for obtaining very long
periods, and they can be very fast. The standard C and Unix
generator RANDOM is of this type. The sequence is defined by
X i = X i-p X i-q which we will denote by F(               ), where p
and q are the lags, p > q, and        is any binary arithmetic operation,
such as addition or subtraction modulo M, multiplication modulo
M, or the bitwise exclusive OR function (XOR). The arithmetic
operations are done modulo any large integer value M, or modulo 1
if the X's are represented as floating point numbers in the interval
[0,1), as can be done if the operation is addition or subtraction.
Multiplication must be done on the set of odd integers.The modulus
function is not required when using XOR. This method requires
storing the p previous values in the sequence in an array called a lag
                            Random Number Generators                 22
table.
Shift Register Generators
Shift register (or Tausworthe)
generators are generally used in a
form where they can be considered as
a special case of a lagged Fibonacci
generator using XOR. XOR gives by
far the worst randomness properties
of any operation for an LFG, so these
generators are not recommended.
          Random Number Generators      23
 Combined Generators
Combining two different generators
has been shown (both theoretically
and empirically) to produce an
improved quality generator in many
circumstances.

          Random Number Generators   24
      Parallel Random Number
             Generators
   Three basic ways:
• Leapfrog - The sequence is partitioned in turn among
  the processors like a deck of cards dealt to card
  players.
• Sequence splitting - The sequence is partitioned by
  splitting it into non-overlapping contiguous sections.
• Independent sequences - For some generators, the
  initial seeds can be chosen in such a way as to
  produce long period independent subsequences on
  each processor.
                   Random Number Generators         25
The most common parallel
random number generators
  •   The Leapfrog Method
  •   Sequence Splitting
  •   Independent Sequences
  •   Other Methods
           Random Number Generators   26
            The Leapfrog Method
Ideally we would like a parallel random number generator
to produce the same sequence of random numbers for
different numbers of processors. A simple way to achieve
this goal is for processor P of an N processor machine to
generate the sub-sequence
so that the sequence is spread across processors in the
same way as a deck of cards is dealt in turn to players in a
card game.
This is known as the leapfrog technique, since each
processor leapfrogs by N in the sequence.


                     Random Number Generators             27
                Sequence Splitting
Another method for parallelizing random number
generators is to split the sequence into non-overlapping
contiguous sections, each generated by a different
processor. For example, one could divide the period of the
generator by the number of processors, and jump ahead in
the sequence by this amount for each processor.
Alternatively, the length of each section of numbers could
be chosen much larger than could possibly be used by any
processor. If the length of the sections is L, then processor
P would generate the sequence

                       Random Number Generators          28
        Independent Sequences
This method is similar to sequence splitting, in that
each processor generates a different, contiguous
section of the sequence. However in this case the
starting point in the sequence is chosen at random for
each processor, rather than computed in advance using
a regular increment. This has the advantage of
avoiding (or at least reducing) possible long-range
correlations, but only if the seed tables on each
processor are random and independent.
                   Random Number Generators         29
             Other Methods
The cellular automata generator is a generalization
of the shift register generator, based on cellular
automata rules. A parallel version called
CMF_RANDOM is provided by Thinking
Machines. Both the sequential and parallel versions
of this generator have passed many of the standard
statistical tests, however CMF_RANDOM failed a
Monte Carlo Ising model test and therefore cannot
be recommended. This generator is also much
slower than those provided in the CMSSL, so it is
not often used for large-scale simulations.
                  Random Number Generators      30
                Conclusions
Developing a good, efficient algorithm for
generating pseudo-random numbers on a computer
is a very difficult problem, especially for parallel
computers. The theoretical understanding of random
number generators is rather limited, and no amount
of empirical testing can ever determine how well a
given generator will perform for a new application.
The improvement in computer performance
continues unabated, of course, and it is crucial that
the implementation and testing of random number
generators keeps pace with these changes.
                  Random Number Generators         31
Some Test Methods

  Let’s see some test methods for testing random number generators
  •Chi-Square Test
  •Cramer-von Mises Test
  •Gap Test
  •Maximum Test




                               Random Number Generators              32
Chi-Square Test
  Let X1…XN be a sample drawn from a population with
  unknown cumulative distribution function Fx(x). We wish to
  test the null hypothesis H0
  When H0 is true:      Fx(x)=F0(x), for all x
  where F0(x) is a complete specified cumulative distribution
  function.
  On the contrary,
  when H1 is true,      Fx(x)F0(x), for all x
  Assume that the N observations have been grouped into k
  mutually exclusive categories. Let’s denote the observed
  number of trial outcomes and the expected number for the jth
  category by N j and Np 0    j




                                  Random Number Generators       33
Chi-Square Test continued
  We have the following test criterion

                     k      ( N j  Np 0 ) 2             k
                 Y                                     N         N
                                       j
                                               , where          j
                     j 1        Np 0
                                    j                    j 1




    When H0 is true, Y tends to be small and
    When H0 is false, Y tends to be large.
    For large samples, Y’s distribution is approximately chi-
    square with k-1 degrees of freedom. When H0 is true, we
    expect

                            P(Y  1 )  
                                    2
                                                                         (1)

      Where  is the significant level, it is about 0.05 or 0.1
      12 could be got from the table of chi-square
      distribution.



                                          Random Number Generators             34
Chi-Square Test continued
 When testing for uniformity we simply divide the interval [0,1] into
 k nonoverlapping subintervals of length 1/k so that Np 0  N / k
                                                         j



  In this case, we have

                             k
                        k                  N 2
                     Y
                        N
                             (N
                            j 1
                                   j   
                                           k
                                             )   (2)



    (1) and (2) could be used for testing random number generators




                                   Random Number Generators             35
Cramer-von Mises Test
  Similar to Chi-Square test, this test also need a given sample X1…XN from
  cumulative distribution function Fx(x)=F0(x). We wish to test the null
  hypothesis:
  When H0 is true:       Fx(x)=F0(x), for at least one value of x
  when H1 is true,       Fx(x)F0(x), for at least one value of x
  We denote the order statistic by X(1)…X(N) and consider the following test
  statistic:
                                           2i  1
                         N                          2
                    1
                Y       [ F0 ( X (i )         )]
                   12 N i 1                2N

   The ordinate of F0(x) is found at each value in the random sample
   X(i), and from this is subtracted the quantity (2i-1)/2N, which is the
   average just before and just after the jump at X(i)—that is, the
   average of (i-1)/N and i/N. The difference is squared, so that
   positive differences do not cancel the negative ones, and the
   results are added together. The quantities of Y are tabulated by
   using an asymptotic distribution function of Y. This test, with
   significance level , rejects H0 if and only if Y>y1-, where y1-
   can be found from the appropriate table.
                                 Random Number Generators                      36
Gap Test
   The gap test is concerned with the randomness of the
   digits in a sequence of numbers. Let U1…UN be such a
   sequence. We say that any subsequence Uj,Uj+1,…Uj+r of
   r+1 numbers represents a gap of length r if Uj and Uj+r lie
   between  and  and (0 <  1) but Uj+i, i=1…r-1, does
   not. For a “true” sequence of random numbers the
   probability of obtaining a gap of length r is

                         P(r )  (0.9) r (0.1)

   A chi-square test based on the comparison of the expected
   and actual numbers of gaps of length r maybe used.




                                 Random Number Generators        37
Maximum Test
 Let Y jk  max{U ( j 1) k 1 ,...,U jk }, j  1,..., N be a sequence of N k-tuples, It has
 been proved that, if the sequence U1,…Unk is from U(0,1), then Y1k ,...,YNk
 is also from U(0,1). To check whether or not U1,…Unk is a “true” sequence
 of random numbers, we can apply the chi-square test to the sequence
             {Y jk , j  1,.., N }


  In fact, more computer time may be spent testing random numbers
  than generating them.




                                         Random Number Generators                              38
Summary and Recommendations



       All random generators should be tested for their thoroughly quality before
       being used. At least two levels of quality in random number generators
       should be implemented to assess whether the answers are independent of
       the random numbers. This a matter requiring judgment and is aided by
       experience.




                         Random Number Generators                                   39