Reviews of Accelerator Science and Technology by uek60010


More Info
									       Accelerator Modeling:
Present capabilities, future prospects,
and applications to the HEP Program
     (with emphasis on SciDAC)
              Robert D. Ryne
    Lawrence Berkeley National Laboratory

             with contributions from
      Kwok Ko (SLAC) and Warren Mori (UCLA)

     Presented to the HEPAP AARD Subpanel
               December 21, 2005
 SciDAC Accelerator Science & Technology
         (AST) Project: Overview
• Goals:
   — Develop new generation of parallel accelerator modeling codes to solve
     the most challenging and important problems in 21st century accel S&T
   — Apply the codes to improve existing machines, design future facilities,
     help develop advanced accelerator concepts
• Sponsored by DOE/SC HEP in collaboration with ASCR
• Primary customer: DOE/SC, primarily its HEP, also NP programs
   — codes have also been applied to BES projects
• Funding: $1.8M/yr (HEP), $0.8M/yr (ASCR/SAPP)
   — Strong leveraging from SciDAC ISICs
• Duration: Currently in 5th (final) year
• Participants:
   — Universities: UCLA, USC, UC Davis, RPI, Stanford
   — Industry: Tech-X Corp.
 SciDAC Accelerator Science & Technology
      (AST) Project: Overview cont.
• Management:
   — K. Ko and R. Ryne, co-PIs
   — Senior mgmt team: K. Ko, R. Ryne, W. Mori, E. Ng
• Oversight and reviews by DOE/HEP program mgrs
   — Vicky White
   — Irwin Gaines
   — Craig Tull (present)
• The project must
   — advance HEP programs (R. Staffin)
   — through synergistic collaboration w/ ASCR that advances the state-of-
     the-art in advanced scientific computing (M. Strayer)
       SciDAC AST Overview: Focus Areas
• Organized into 3 focus areas:
   — Beam Dynamics (BD), R. Ryne
   — Electromagnetics (EM), K. Ko
   — Advanced Accelerators (AA), W. Mori
• All supported by SciDAC Integrated Software Infrastructure Centers (ISICs)
  and ASCR Scientific Application Partnership Program (SAPP)
• Most funding goes to BD and EM; AA is very highly leveraged
        Why do we need SciDAC???

• Why can’t our community do code development
  just ourselves as we have done in the past?
• Why can’t it be done just as an activity tied to
  accelerator projects?
• Why can’t our community follow “business as
                Computational Issues
• Large scale:
   —simulations approaching a billion particles, mesh points
   —Huge data sets
   —Advanced data mgmt & visualization
• Extremely complex 3D geometry (EM codes)
• Complicated hardware with multiple levels of memory
  heirarchy, > 100K processors
• Parallel issues
   —Load balancing
   —Parallel sparse linear solvers
   —parallel Poisson solvers
   —particle/field managers
    Close collaboration w/ ASCR researchers
           (ISICs, SAPP) is essential

• A hallmark of the SciDAC project is that it built upon
  collaboration between applications/computational
  scientists with mathematicians, computer scientists,
  parallel performance experts, visualization specialists,
  and other IT experts.
• The AST project collaborates with several ISICs:
   —TOPS (Terascale Optimal PDE Solvers)
   —APDEC (Applied Partial Differential Equations Center)
   —TSTT (Terascale Simulation Tools & Technologies)
   —PERC (Performance Evaluation Research Center)
     Overview of the 3 focus areas

• Beam Dynamics (BD)
• Electromagnetic Modeling (EM)
• Advanced Accelerators (AA)
     Overview of the 3 focus areas

• Beam Dynamics (BD)
• Electromagnetic Modeling (EM)
• Advanced Accelerators (AA)
        SciDAC Codes: Beam Dynamics

• Set of parallel, 3D multi-physics codes for modeling beam
  dynamics in linacs, rings, and colliders
   —IMPACT suite: includes 2 PIC codes (s-based, t-based);
    mainly for electron and ion linacs
   —BeamBeam3D: strong-weak, strong-strong, multi-slice,
    multi-bunch, multi-IP, head-on, crossing-angle, long-range
   —MaryLie/IMPACT: hybrid app combines MaryLie+IMPACT
   —Synergia: multi-language, extensible, framework; hybrid
    app involves portions of IMPACT+MXYZPTLK
   —Langevin3D: particle code for solving Fokker-Planck
    equation from first principles
       IMPACT suite becoming widely used;
> 300 email contacts in FY05, > 100 already in FY06

          •RAL           •SLAC
          •PSI           •LBNL
          •GSI           •LANL
          •KEK           •Tech-X
SciDAC code development involves large, multidisciplinary teams.
               Example: MaryLie/IMPACT code
Development, reuse, and synthesis of code components.
        Examples: Synergia, e-cloud capability
New algorithms and methodologies are key. Examples: (1) high aspect ratio
        Poisson solver; (2) self-consistent Langevin/Fokker-Planck

           Electric field error vs. distance            Self-Consistent Diffusion Coefficients vs. velocity

                                                                            Spitzer approximation

   Error in the computed electric field of a Gaussian
distribution of charge (x=1mm and y=500mm). Even            First-ever 3D self-consistent
  using a grid size of 64x8192, the standard method        Langevin/Fokker-Planck simulation
   (blue curve) is less accurate than the Integrated
       Green Function method (purple) on 64x64.
     SciDAC beam dynamics applications
     benefit DOE/SC programs, esp. HEP
• Beam-Beam simulation of Tevatron, PEP-II, LHC, RHIC
• ILC damping rings (space-charge, wigglers)
• FNAL Booster losses
• CERN PS benchmark study
• RIA driver linac modeling
• SNS linac modeling
• LCLS photoinjector modeling
• CERN SPL (proposed proton driver) design
• J-PARC commissioning
• Publications:
   — 23 refereed papers since 2001 (including 5 Phys Rev Lett., 10
     PRST-AB, 4 NIM-A, 2 J. Comp. Phys., Computer Physics
     Comm.), numerous conf proceedings papers
• USPAS course on computational methods in beam dynamics
Examples: Collider modeling using BeamBeam3D

     LHC beam-beam
     nx1=nx2= ny1=ny2=0.31,

First-ever 1M particle, 1M turn strong-   PEP-II luminosity calculation shows importance of multi-
strong b-b simulation (J. Qiang, LBNL)    slice modeling (J. Qiang, Y. Cai, SLAC; K. Ohmi, KEK)

  Parameter studies of antiproton                   Code scalability depends strongly on
  lifetime in Tevatron                              parallelization methodology (J. Qiang)
ILC damping ring modeling using ML/I
                   Results of MaryLie/IMPACT
                   simulations of an ILC ―dog-bone‖
                   damping ring (DR) design showing
                   space-charge induced emittance
                   growth using different space-charge
                   models. Space charge is important for
                   the ILC DR in spite of the high energy
                   because of the combination of small
                   emittance and large (16 km)
                   circumference. Top (nonlinear space
                   charge model): the beam exhibits
                   small emittance growth. Bottom
                   (linear space charge model): the
                   beam exhibits exponential growth due
                   to a synchro-betatron resonance. The
                   instability is a numerical artifact
                   caused by the simplified (linear)
                   space-charge model. (M. Venturini,
   FNAL booster modeling using Synergia

FNAL booster simulation results using Synergia showing the merging of 5
microbunches. SciDAC team members are working closely with experimentalists
at the booster to help understand and improve machine performance.
(P. Spentzouris and J. Amundson, FNAL; J. Qiang and R. Ryne, LBNL)
            Beam Dynamics under SciDAC 2
                   (HEP program)
• Support/maintain/extend successful codes developed
  under SciDAC 1 (BD, EM, AA)
• Develop new capabilities to meet HEP priorities: LHC, ILC,
  Tevatron, PEP-II, FNAL main injector, booster, proton driver
   — Self-consistent 3D simulation of: e-cloud, e-cooling, IBS, CSR
   — Start-to-end modeling with all relevant physical effects
• Enable parallel, multi-particle beam dynamics design &
• Performance and scalability optimization on platforms up
  to the petascale (available by the end of the decade)
• Couple parallel beam dynamics codes to commissioning,
  operations, and beam experiments
     Overview of the 3 focus areas

• Beam Dynamics (BD)
• Electromagnetic Modeling (EM)
• Advanced Accelerators (AA)
     SciDAC AST – Electromagnetics
Under SciDAC AST, the Advanced Computations Dept. @
SLAC is in charge of the Electromagnetics component to:

 Develop a comprehensive suite of parallel electromagnetic
 codes for the design and analysis of accelerators,
 (Ron’s talk)
 Apply new simulation capability to accelerator projects
 across SC including those in HEP, NP and BES,
 (Ron’s talk)
 Advance computational science to enable terascale
 computing through ISICs/SAPP collaborations.
 (this talk)
   ACD’s ISICs/SAPP Collaborations
ACD is working with the TOPS, TSTT, PERC ISICs as
well as SAPP researchers on 6 computational science
projects involving 3 national labs and 6 universities.
 Parallel Meshing – TSTT (Sandia, U Wisconsin/PhD thesis)
 Adaptive Mesh Refinement – TSTT (RPI)
 Eigensolvers – TOPS (LBNL), SAPP (Stanford/PhD thesis,
                                    UC Davis)
 Shape Optimization – TOPS (UT Austin, Columbia, LBNL),
                       TSTT (Sandia, U Wisconsin)
 Visualization – SAPP (UC Davis/PhD thesis)
 Parallel Performance – PERC (LBNL, LLNL)
Parallel Meshing & Adaptive Mesh Refinement
  Parallel meshing is needed for generating LARGE meshes to
  model multiple cavities in the ILC superstructure & cryomodule

   Processor:    1                              2                         3                                4

  Adaptive Mesh Refinement improves accuracy & convergence
  of frequency and wall loss calculations
                                                        RFQ - Frequency Convergence                      RFQ - Q Convergence
                                             55.2                                             6100
                                             55.1                                             6050
                          Frequency in MHz

                RIA RFQ                      54.9            Frequency
                                                                                                         Wall loss Q

                                             54.7                                             5900
                                             54.6                                             5850
                                             54.3                                             5750
                                                    0   1000000 2000000 3000000 4000000              0   1000000 2000000 3000000   4000000
                                                                                                            Number of Unknowns
                                                          Number of Unknowns
     Eigensolvers & Shape Optimization
Complex eigensolver for treating external coupling is essential
for computing HOM damping in ILC cavities.

            Lossless                   Lossy             Periodical     External
                                       Material          Structure      Coupling

   ISIL w/            ESIL             Implicit Restarted        SOAR   Self-Consistent
 refinement                                 Arnoldi                          Loop

Shape Optimization to replace manual, iterative process in
designing cavities with specific goals subject to constraints.

            Omega3P                               optimization            geometric
            Sensitivity                                                     model

                                                   Omega3P                 meshing
     (only for discrete sensitivity)
  Visualization & Parallel Performance
Visualization is
critical to mode
analysis in
complex 3D
cavities, e.g.
mode rotation
Parallel Performance studies are needed to maximize code
efficiency and optimize use of computing resources.
   Solve & Postprocess Breakdown   Communication Pattern
     Proposed Projects for SciDAC 2
SLAC will develop the NEXT level of simulation tools for
NEXT generation SC accelerators (ILC, LHC, RIA, SNS)
by continuing to advance Computational Science in
collaborations with the ISICs/SAPP component of SciDAC
 Parallel adaptive h-p-q refinement where h is mesh size,
  p is order of FE basis and q is order of geometry model
 Parallel shape optimization (goals w/ constraints) and
  prediction (cavity deformations from HOM measurements)
 Parallel particle simulation on unstructured grids for
  accurate device modeling (RF guns, klystrons)
 Integrated electromagnetics/thermal/mechanical modeling
  for complete design and engineering of cavities
 Parallel, interactive visualization cluster for mode analysis
  and particle simulations
     Overview of the 3 focus areas

• Beam Dynamics (BD)
• Electromagnetic Modeling (EM)
• Advanced Accelerators (AA)
Recent advances in modeling advanced accelerators:
      plasma based acceleration and e-clouds

W.B.Mori , C.Huang, W.Lu, M.Zhou, M.Tzoufras, F.S.Tsung, V.K.Decyk (UCLA)
     D.Bruhwiler, J. Cary, P. Messner, D.A.Dimtrov, C. Neiter (Tech-X)
                  T. Katsouleas, S.Deng, A.Ghalam (USC)
                         E.Esarey, C.Geddes (LBL)
                 J.H.Cooley, T.M.Antonsen (U. Maryland)
          Accomplishments and highlights:
               Code development

• Four independent high-fidelity particle based codes
   — OSIRIS: Fully explicit PIC
   — VORPAL: Fully explicit PIC + ponderomotive guiding center
   — QuickPIC: quasi-static PIC + ponderomotive guiding center
   — UPIC:    Framework for rapid construction of new
              codes--QuickPIC is based on UPIC: FFT based

• Each code or Framework is fully parallelized. They each have
  dynamic load balancing and particle sorting. Each production
  code has ionization packages for more realism. Effort was made
  to make codes scale to 1000+ processors.
• Highly leveraged
             Full PIC: OSIRIS and Vorpal
• Successfully applied to various
  LWFA and PWFA problems

                                      Colliding laser pulses

   Self-ionized particle beam wake

                                           Particle beams

                                       Scale well to 1,000’s of
  3D LWFA simulation                        processors
                      Quasi-static PIC:
Code features:
• Based on UPIC parallel object-oriented plasma
  simulation Framework.
Model features:                                            afterburner
• Highly efficient quasi-static model for beam drivers
• Ponderomotive guiding center + envelope model for
  laser drivers.
• Can be 100+ times faster than conventional PIC with
  no loss in accuracy.                                       hosing
• ADK model for field ionization.
Applications:                                            E164X
• Simulations for PWFA experiments,
• Study of electron cloud effect in LHC.
• Plasma afterburner design
Recent highlights: LWFA simulations
using full PIC

   Phys. Rev. Lett. by Tsung et al. (September 2004) where a peak energy
    of 0.8 GeV and a mono-energetic beam with an central energy of 280
    MeV were reported in full scale 3D PIC simulations.
   3 Nature papers (September 2004)
    where     mono-energetic     electron
    beams with      energy near 100 MeV
    were measured. Supporting PIC
    simulations were presented.
   SciDAC members were collaborators
    on two of these Nature publications
    and SciDAC codes OSIRIS and Vorpal
    were used.

            Vorpal result on cover
Modeling self-ionized PWFA experiment with QuickPIC

                        Em ax ~ 4GeV
                        (initial energy chirp                  Located in the FFTB


                                                                   e-     Laser Pulse Li Plasma
                                                                                                                Streak Camera
                                                                                                               (1ps resolution) •Cdt
                                                                           (193 nm) ne- 6 · 10 15 cm -3
                                                                                        L- 30 cm                                         X-Ray
                                                                   N=1-2 · 10 10
                                                                   z =0.1 mm                                              Cerenkov
                                                                                 Optical Transition        Spectrometer
                                                                   E=30 GeV                                                Radiator
                                                                                     Radiators                                           Dump
E164X experiment                                                                                              25 m
                                                                                                               25 m
                                                                                                          Not to scale!

                             Emax ~ 5- 0.5( - tronradiation)  4.5GeV
                                          8                                      12

                                          6                                      10

                                                                                      Beam current (KA)
                                        4                                      8
                               E (GeV)

                                          2                                      6

                                          0                                      4

                                          -2                                     2

                                          -4                                      0
                                               0   100   200    300     400    500

                             QuickPIC simulation
         Afterburner simulation:
      0.5 TeV ~ 1 TeV in 28 meters


               s = 28.19 m

 Simulation done with QuickPIC in 5000 node-hours
Full PIC run would have taken 5,000,000 node-hours!
          Vision for the future: SciDAC 2
    High fidelity modeling of .1 to 1TeV plasma
                 accelerator stages
•   Physics Goals:
     — A) Modeling 1to 10 GeV plasma acc stages: Predicting and designing near term experiments.
     — B) Extend plasma accelerator stages to 250 GeV - 1TeV range: understand physics & scaling laws
     — C) Use plasma codes to definitively model e-cloud physics:
           • 30 minutes of beam circulation time on LHC
           • ILC damping ring
•   Software goals:
     — A) Add pipelining into QuickPIC: Allow QuickPIC to scale to 1000’s of processors.
     — B) Add self-trapped particles into QuickPIC and ponderomotive guiding center Vorpal packages.
     — C) Improve numerical dispersion* in OSIRIS and VORPAL.
     — D) Scale OSIRIS, VORPAL, QuickPIC to 10,000+ processors.
     — E) Merge reduced models and full models
     — F) Add circular and elliptical pipes* into QuickPIC and UPIC for e-cloud.
     — G) Add mesh refinement into* QuickPIC, OSIRIS, VORPAL,and UPIC.
     — H) Develop better data analysis and visualization tools for complicated phase space data**

     •   *Working with APDEC ISIC
     •   **Working with visualization center
                      In Conclusion…

• Q: What is the scope of our research in regard to HEP
  short/medium/long-range applications?
• A: It is mainly short/medium.
   —Capabilities have been developed, codes applied to:
      • Short: PEP-II, Tevatron, FNAL Booster, LHC
      • Medium: ILC
      • Long: Exploration of advanced accelerator concepts
         – these activities are highly leveraged, represent 10% of the
           SciDAC AST budget
                      Final remarks

• Future HEP facilities will cost ~$0.5B to ~$10B
   —High end modeling is crucial to
      • Optimize designs
      • Reduce cost
      • Reduce risk
   —Given the magnitude of the investment in the facility, the
    $1.8M investment in SciDAC is tiny, but the tools are
• Laser/plasma systems are extraordinarily complex
   —High fidelity modeling, used in concert with theory &
    experiment, is essential to understand the physics and
    help realize the promise of advanced accelerator
                      Acronyms used
• SciDAC: Scientific Discovery through Advanced Computing
• AST: SciDAC Accelerator Science & Technology project
• ASCR: Office of Advanced Scientific Computing Research
• BD: Beam dynamics activities of SciDAC AST
• EM:Electromagnetics activities of SciDAC AST
• AA: Advanced Accelerator activities of SciDAC AST
• ISIC: SciDAC Integrated Software Infrastructure Center
   — TOPS: Terascale Optimal PDE Solvers center
   — TSTT: Terascale Simulation Tools and Technologies center
   — APDEC: Applied Partial Differential Equations center
   — PERC: Performance Evaluation Research Center
• SAPP: Scientific Application Partnership Program (ASCR-supported
  researchers affiliated w/ specific SciDAC projects)

To top