Docstoc

madden (PDF)

Document Sample
madden (PDF) Powered By Docstoc
					Scientific Software Development in an
        Academic Environment:
  the UK Car-Parrinello Consortium


               Paul Madden
    Chemistry Dept., University of Edinburgh



             “Ab initio” condensed matter simulations
              (R. Car and M. Parrinello, 1985)
MD



     H5O2+ in H2O




     Axel Kohlmeyer (Bochum)
AIMD
 Calculate the intermolecular
 forces from the electronic
 wavefunction which is
 updated “on-the-fly”.




        Axel Kohlmeyer (Bochum)
A Kohn-Sham orbital in water




                               M. Sprik, Cambridge
     Leading-edge scientific software
 development issues (for academic group)
• Timescale long compared to Ph.D.
  (sustainability of “resources”)
• Development co-exists with application
  (shortcuts => consolidation)
• Optimisation is different to development
• Dissemination (of software) is too demanding
          (National) HPC Issues

• How to ensure resource is properly used
      “capability” job concept
       expertise & back-up (code optimisation.)
      “tensioning” allocations
       fluctuations in personnel
 How to ensure benefit is shared by larger
  community (capability => exclusivity)
        e.g. by availability and maintenance
        of generally useable code
            UKCP (since ~1989)

• 13 Groups (in different universities)
• To use national HPC facilities for AIMD
• Funding from EPSRC (1 postdoc + 30,000,000
  processor hours over 3 years)
• Internal allocation of these resources
• Multi-code (CPMD, SIESTA, vasp, CASTEP….), but
  collaborative development on CASTEP
    CASTEP (M.C. Payne + developers)
• AIMD – for Materials (pseudopotentials, B-Z
  sampling, metals……)
• Marketed by Accelrys (easy use, gui etc)
• Accelrys – CLRC – UKCP agreement makes
  CASTEP free to UK academics
• CCLRC/EPSRC gives 1.5 FTEs for
  CASTEP/HPC development
• UKCP gives new functionality to Accelrys (linear
  response phonons & polarizability; MLWFS – dipoles, quadrupoles,
  Born charges; finite displacement phonons; harmonic free energy…..)
  incorporated via the developers group.
                     HPCx
• 50 IBM p690+ Regatta nodes = 1600 procs.
• 6.6 TFlops + 1.28TB memory
• “Capability discounts” etc.– e.g. if T(512)/T
  (1024)>1.7 => 30% discount

• Processors organised in groups of 32 (LPAR) v.
  good local comms.
• CASTEP scaled badly (fft => latency in All-to-All
  communication)
       Communication speedup

• All-to-All MPI          LPAR 1   LPAR 2
  communication
  between all processor   1    5   9    13
  pairs.                  2    6   10   14
• Instead, gather data    3    7   11   15
  within each LPAR
                          4    8   12   16
  (e.g. 1, 9, ..) and
  transfer 1  9 etc,
  then scatter within
  LPAR.
• M. Plummer (CCLRC)
              Castep 2004 HPCx
               performance gain
                  Al2O3 120 atom cell, 5 k-points

           8000
           7000
           6000
Job time




           5000
           4000
           3000                                     Jan-03
           2000                                     Current 'Best'
           1000
              0
                   80     160     240    320

                   Total number of processors
              Castep 2003 HPCx
               performance gain
                   Al2O3 270 atom cell, 2 k-points


           20000

           15000
Job Time




           10000
                                                     Jan-03
            5000
                                                     Current 'Best'

               0
                    128        256       512

                    Total number of processors
                Phil Lindan – U. of Kent,
        Hydration of (110) surface of TiO2 (Rutile)



AIMD

 300K

81 H2O

72 ions

20 ps
Linear Scaling Techniques (in system size)

                                  αβ         
ρ ( r , r ′)     =    ∑    ∑ φαR ( r ) K φ βR ( r ′)
                      αβ    R
     R      = lattice vectors

        α                         β
    K        = density kernel

   {φ }
     αR
          = non-orthogonal generalised Wannier
               functions (NGWFs)
    P.D. Haynes, C.-K. Skylaris, O. Diéguez,
         A.A. Mostofi and M.C. Payne
                             ONETEP and CASTEP comparison:
Numerical comparisons        •Same basis set
                             •Same plane wave kinetic energy cut-off
with CASTEP                  •Same simulation cells
                             •Same norm-conserving pseudopotentials
                             •Same functional (LDA was used here)
                             •Different CPU and MEMORY scaling
                             (CASTEP is cubic, ONETEP is linear!)

                             Energy difference between neutral
                             /di-ion (zwiterionic) form of
                             peptides


                        CA              eV
                                       1.2               kcal/mol
                    STEP               1
                                       1.2                 28.0
           LY       NETEP              0                   27.7
               PE   CASTEP           1.07                 24.7
           P        ONETEP           1.08                  24.9
True linear-scaling: Cost per iteration is linear in number of atoms and number of
iterations required for convergence is small and independent of number of atoms

                            GLY                          GLY50


                                                                                                              GLY100




                                            SCF convergence
                                                                                           GLY200
                          1.0E-01                                GLY (10 atoms)

                                                                 GLY50 (353 atoms)
                          1.0E-02                                GLY100 (703 atoms)

                                                                 GLY200 (1403 atoms)
 Energy(Ha) gain / atom




                          1.0E-03
                                                                 NTB (256 atoms)

                                                                 NTB_DEF (516 atoms)
                          1.0E-04


                          1.0E-05


                          1.0E-06                                                            NTB

                          1.0E-07                                                                   NTB_DEF
                                    0   5         10            15       20           25
                                                SCF iteration
  “Unleashing the power of the Bluegene”

• At present, AIMD-style applications have
  scaling problems beyond ~ O(102) procs.
• Long (simulation) runs on a fragment of
  the machine are not efficient => new
  strategies
• Long runs are necessary for: large
  systems (embedding, coarse graining) or
  rare events.
 Rare events and barrier crossing




                           A        B
                       B
kBT
           A
             Ion pair dissociation in water


                                        Cl-

                                  Na+




  r*




                                  ) 0 n )
                                  k (n r (
                                  (Ý−tn
                                   =r θr
                                   r io [−
                                    δ
                                    ) 0r *
                                     [) io
                                  tio (*]
       Cl-
Na+
Order parameters and reaction coordinates




       F(q)
                           Auto ionization of water




                                      2H2O -> OH-     + H3O+

            t=0




                           Time scale, hours

                                                       t=150 fs




P. Geissler, D. Chandler
Transition Path Sampling (D. Chandler)




                     B
   kBT
            A
Random to sample configuration space

    T(
   − ln
   k P
    B  )
       x                                           →
                                                  x x
                                                    '
                                      Accept or reject according to value of

                                                   P (x')     Monte Carlo
                                                   P (x )
                         kT
                          B

                              x
Random walk in trajectory space
                                           =
                                          ( x t
                                          t 1, )
                                          ) , ...,
                                            x
                                          x(2 x
                                   '(
                                  x t)
  ( =
 hx 1                 x(t)                    x→
                                              (
                                              t) x
                                                 ()
                                                 't
 A )

                                  B
             A                                     ρ (h
                                                   x x(
                                                   ' A) '
                                                    )' x
                                                   (h B  )
                                                    0 0 t
     Accept or reject according to value of
                                                   ρ( B
                                                   x xx
                                                    ) 0(
                                                   (hh
                                                    0A  )
                                                      ) t
Linear scaling – Can sample the trajectories independently (sim ult a ne ously)
Nuclear Quantum Effects in Barrier Crossing




                          Tunnelling,
                          especially of light
                        B atoms, like H
      kBT
               A
Nuclear Quantum Effects in Barrier Crossing




                              B
        kBT
                   A

Each particle becomes a “polymer”
ring, with each bead connected by
harmonic springs (realization of
Feynmann’s path integrals).
 In a many-particle system (A,B,C….), the real
interactions occur between beads at the same
                imaginary time




 bead Ai interacts with Bi, Ci,…… via the
intermolecular interactions, and locally with
Ai-1 & Ai+1 via a harmonic spring
     Corresponds to N simulations running
simultaneously with infrequent, local interactions




   A1,B1….     A2,B2…        A3,B3….




   AN,BN….
     Corresponds to N simulations running
simultaneously with infrequent, local interactions




   A1,B1….     A2,B2…        A3,B3….




   AN,BN….

Can fill the Bluegene with weakly interacting
100 processor tasks!
      Leading-edge scientific software
  development issues (for academic group)
 • Timescale long compared to Ph.D.
   (sustainability of “resources”)
 • Development co-exists with application
   (shortcuts => consolidation)
 • Optimisation is different to development
 • Dissemination is too demanding


Cooperation between individuals and organisations
                 necessary

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:10/22/2011
language:English
pages:30