Load Sharing Facility (LSF) Commands
bsub < script bjobs bjobs -u all bhist -l [JobId] bresume [JobId] bkill [JobId] bstat bqueues Debug Default Background : : : : Submit LSF job : Show status of user jobs : Show status of all jobs : Show historical job information : Resumes a suspended job : Delete a job : Show status of LSF jobs : Displays information about queues Queues 32 CPUs, 1 hour, 32 GB limit up to 1/2 compute nodes, 1 week limit, CPH limited * 32 CPUs, 120 hours not charged against allocation; will not run if other jobs are waiting
AFRL MSRC Information
Available Online at: http://www.afrl.hpc.mil • Kerberos, SecurID, and Secure Shell • Sample Batch Submission Scripts • Quick Reference Guide to the AFRL MSRC HPC Environment • User Tips Quick Reference Guide • Introduction to the AFRL MSRC • AFRL MSRC User’s Guide • HP XC User Guide • SGI Altix 3700 User’s Guide • SGI Altix 4700 User’s Guide • Archival Server User’s Guide • Checkpoint/Restart User Guide Please visit our website for information about: • Hardware Resources • Software Resources • Message of the Day • Frequently Asked Questions (FAQ) • Policies and Procedures • Video Clips • The latest version of this Guide! .......and much more
CCAC Help Desk Hours of Operation 0800 – 2300 Eastern Time Toll Free: 1-877-CCAC-039 (2 2 2 2) help@ccac.hpc.mil Commercial: 937-255-0679 Fax: 937-656-9538 hpc-accounts@ccac.hpc.mil After hours, please call 1-888-677-2272 or send an email to the Help Desk at help@ccac.hpc.mil. 0607-07 ASC 04-1651, 06-0240
Air Force Research Laboratory Major Shared Resource Center (AFRL MSRC)
Batch Job Submission and Execution System eagle jobs: User must submit jobs from eagle-0. LSF will determine on which Altix system the job will run. falcon jobs: User must login to one of the falcon nodes [1-4]. hpc11 jobs: User must submit jobs from hpc11-0. LSF will determine on which hpc11 *CPU Hours (CPH) CPH = Number_of_CPU x Wall_Time CPH Limit ≈ 1/2 * compute nodes * 48 hours For more information visit: http://www.afrl.hpc.mil/overall/faq/batchfaqs.php Kerberos Commands kinit : obtain Kerberos ticket kpassword [principal] : change Kerberos password kdestroy : destroy Kerberos tickets
Quick Reference Guide to the AFRL MSRC High Performance Computing Environments
2435 Fifth Street Bldg 676 Wright-Patterson AFB OH 45433-7802
SGI Altix 4700 Complex HAWK Quick Reference
(hawk-0) 32 (1.6 GHz) Intel Itanium 2 Interactive CPUs and 64 GB of memory (hawk-1) 500 (1.6 GHz) Intel Itanium 2 (hawk-2) CPUs and 2 TB of memory Batch (3.5 GB/processor user accessible) (hawk-3 16 nodes with 500 (1.6 GHz) thru hawk-18) Intel Itanium 2 CPUs and Batch 1 TB of memory each (1.75 GB/processor user accessible) 9 MB secondary cache on each processor HPC systems share four 440 TB disk workspaces SUSE10 Linux operating system Libraries Intel MKL: includes ScaLAPACK, sparse solver, LAPACK, BLAS, cblas, FFTs, DFTs, VSL, VML SGI SCSL: includes BLAS 1, 2, and 3, and LAPACK Editors ed, emacs, ex, gedit, joe, vi, vim, xemacs Debuggers ddd, gdb, idb Parallel Run Command pam [-n # of Cores] -mpi -auto_place [program]1 mpirun -np [# of Cores] [program] [options]2 dplace -x2 -c [Core range] [program]3 mpirun -np [# of Cores] omplace -nt [#thread/rank] [program] [options]4 MPI 2 hybrid MPI and SMP with 500 processors or less 3 OpenMP or SHMEM (example CPU range: 0-15 for 16 processors) 4 hybrid MPI/OpenMP with 500 PE or less
1
SGI Altix 3700 Complex EAGLE Quick Reference
(eagle-0) Interactive (eagle-1) Batch (eagle-2 thru 4) Batch 16 (1.6 GHz) Intel Itanium 2 CPUs and 32 GB of memory 500 (1.6 GHz) Intel Itanium 2 CPUs and 2 TB of memory (3.5 GB/processor user accessible) 3 nodes with 500 (1.6 GHz) Intel Itanium 2 CPUs and 500 GB of memory each (875 MB/processor user accessible)
HP XC Cluster FALCON Quick Reference
(falcon) Interactive 4 Nodes, each with: 4 (2.4 GHz) AMD Opteron CPUs 1 MB Cache 16 GB RAM (15 GB user accessible) (falcon) Batch 1024 XC Nodes, each with: 2 (2.8 GHz) AMD Opteron CPUs 1 MB Cache 4 GB RAM (MPI - 2.25 - 2.5 GB user accessible) (SMP - 3 GB user accessible) Infiniband interconnect 97 TB shared disk space HP SFS (Lustre) scalable file system Libraries ACML: includes BLAS 1, 2, and 3, FFT, and LAPACK Editors emacs, xemacs, vi, vim gdb, pgdbg, valgrind, totalview Debuggers
8 MB secondary cache on each processor HPC systems share two 43 TB disk workspaces Libraries Intel MKL: includes ScaLAPACK, sparse solver, LAPACK, BLAS, cblas, FFTs, DFTs, VSL, VML SGI SCSL: includes BLAS 1, 2, and 3, and LAPACK Editors ed, emacs, ex, gedit, joe, vi, vim, xemacs Debuggers ddd, gdb, idb Parallel Run Command pam -mpi -a eagle -auto_place [program]1 mpirun -np [# of CPUs] [program] [options]2 dplace -x2 -c[CPU range] [program]3 MPI 2 hybrid MPI and SMP with 500 processors or less 3 OpenMP or SHMEM (example CPU range: 0-15 for 16 processors)
1
mpirun -srun -n [# of CPUs] [program] [args...]
Compilers Compiler Commands
Parallel Run Command
Compilers Compiler Commands ifort : Intel FORTRAN 77 and 90 icc : Intel C icpc : Intel C/C++ GNU Compiler Commands gcc, c++, g++, g77: C, C++, C++, FORTRAN 77
Compilers Compiler Commands ifort : Intel FORTRAN 77 and 90 icc : Intel C icpc : Intel C/C++ GNU Compiler Commands gcc, c++, g++, g77 : C, C++, C++, FORTRAN 77
g77 pgf77, pgf90, pgf95 gcc pgcc g++ pgCC
: GNU f77 : PGI f77, f90, f95 : GNU C : PGI C : GNU C++ : PGI C++