Get documents about "Introduction to Fortran 9095 Programming for HPC
Document Sample
Introduction to Fortran 90/95 Programming for HPC
Understanding Modern Fortran -
Developing and Maintaining Code
Gil Speyer and Dan Stanzione
Arizona State University
Fulton High Performance Computing
Lars Koesterke
Texas Advanced Computing Center
March 16th, 2009
Speyer@asu.edu, dstanzi@asu.edu
High Performance Computing Initiative
Who This Talk is For
● This is not really an HPC talk - this is a
prerequisite on scientific programming to prepare
the user for HPC
● This is for FORTRAN beginners
– A little programming experience is useful, but not
necessarily in FORTRAN
● Those who already know FORTRAN may wish to
skip this morning and come back for the true HPC
talks (OpenMP, MPI, etc).
– If you know FORTRAN, and you stay, don’t say we
didn’t warn you :).
High Performance Computing Initiative
Why Talk About FORTRAN
● FORTRAN is a 55 year old programming language… why
are we talking about it in advanced computing?
– Still the *most trusted* language for numerical work
– Huge legacy base of scientific codes
– Nearly 70% of modern HPC workload is FORTRAN code.
– HPC codes last a *long* time, with many generations of changes,
and cost a fortune to build.
● over 35 years, anything not FORTRAN or C is just a fad!!!
– FORTRAN has evolved to stay relevant!
High Performance Computing Initiative
Objectives
● Familiarize the attendee with concepts of both
traditional and modern FORTRAN.
● Allow the attendee to read, understand, and
modify existing FORTRAN codes.
● Provide a base of knowledge to allow the
attendee to learn advanced FORTRAN on their
own.
● Provide an introduction to large scale, long term
code management.
High Performance Computing Initiative
Overview
● History of Fortran : Before and After F77
● A First Look
● Basics
● Before F77 : You don’t want to learn this (or do you?)
● F77 : Features
● F77 - F90 : Old meets New
● F90 : Features
● F90 : Beyond this Talk
High Performance Computing Initiative
Overview II
● Understand the Compiler
● Optimization : an Afterthought?
● What is Debugged?
● Rules
● More Rules
● Software development : What are you up
against?
High Performance Computing Initiative
Recommended Reading
● Michael Metcalf and John Reid -
– FORTAN 90/95 explained (Fortran 95/2003 explained)
● Stephen J. Chapman -
– FORTRAN 90/95 for Scientists and Engineers
● Adams, Brainerd, Martin, Smith and Wagener -
– Fortran 90 Handbook
– The Fortran 2003 Handbook Complete ANSI / ISO Reference
High Performance Computing Initiative
A Little History
● Original FORTRAN (FORmula TRANslator) - 1954
● Fortran I-IV : 1956, 57, 58, 62
● Fortran66 (1966)
● Fortran77 (1977 - still around)
● Fortran90 (First “modern” FORTRAN)
● Fortran95/2003 (minor update plus HPC additions)
FORTRAN is the parent of all modern “high level”
programming languages, and still the one most concerned
with numerical computation, and achieving high
performance
High Performance Computing Initiative
History of FORTRAN
Language Quirks -
Know Where They Come From
● FORTRAN dates from the days of punch cards
and paper tapes.
– Punching in extra spaces was hard
– Spaces matter
● Lots of legacy code is therefore tough to read.
● Lots of other languages have their own
anachronisms
– E.g. semicolons at end of C lines.
High Performance Computing Initiative
History of FORTRAN
● Fortran 77 The big Bottleneck
– Fortran did not evolve after the F77 Standard
– Fortran 90 compilers became available by the
mid-90s
● Pivotal Flaw of the Fortran 77 Standard
– Standard specified what is allowed, but not what is
not allowed!
– Code would run differently on different platforms
– Vendor specific extensions
– A Big (continuing) Mess!
High Performance Computing Initiative
History of FORTRAN
Fortran90+ : The “new” Fortran
● Modern, efficient, and appropriate for
Number Crunching and High Performance Computing
● Upgrades every few years : 90, 95, 2003, 2008, ...
– Major Upgrade every other Release : 90, 2003, ...
– Easy switch : F90 is fully compatible with F77!
● And yes, it used to be FORTRAN, but now Fortran is
officially acceptable.
High Performance Computing Initiative
General Remarks
Styles in this Document
● Fortran Statements :
– EXAMPLE STATEMENT (F77)
– EXAMPLE STATEMENT (F90+)
Variables that are not specified explicitly have
●
the Type Integer (first letter of name I-N) or
Real (first letter A-H,O-Z)
The statement IMPLICIT NONE is recommended
●
for beginners (forces all variables to be explicitly declared).
High Performance Computing Initiative
General Remarks II
● Fortran traditionally had a “fixed format”, corresponding
to punch card layout, that gave meaning base on which
column characters appeared in:
– 72 characters per line
– columns 1-5 : Statement Label
– column 6: Continuation character
● Any character in column 6 (but usually “c”) means this line is a
continuation of the previous statement
– Comments : 1st Character a : c, * or !, or after ! in a line (except in
column 6)
High Performance Computing Initiative
General Remarks III
● Modern Fortran compilers also support “free
format”:
– Statement starts at Pos. 1
– Continuation Line : & last in previous line
– Comments start with an !
High Performance Computing Initiative
The Basics
Variables : Integer, Real, Logical, Character, Complex:
REAL X, Y, Z REAL :: X, Y, Z
INTEGER I, J, K, M INTEGER :: I, J, K, M
LOGICAL B LOGICAL :: B
CHARACTER*(8) N CHARACTER(LEN=8) :: N
Parameters:
PARAMETER (M=10)
INTEGER, PARAMETER :: M = 10
Arrays:
DIMENSION X(M)
REAL, DIMENSION(M) :: X
High Performance Computing Initiative
The Basics II
Literal Constants:
Integer : 1, 0, -999, +10
Real : 1., 0., 1.E10
Names:
Valid : A, A_Thing, REAL
Invalid : 1A, A Thing, $sign
Logical Expressions & Relational Operators:
.NOT. .AND. .OR. .GE. .GT. .EQ. .NE. .LT. .
LE. >= > == /= < <=
High Performance Computing Initiative
The Basics III
● For Loop (with While and Repeat)
DO 10 I=10, 1, -1
...
10 CONTINUE
DO I=10, 1, -1
...
IF (I .EQ. 3) CYCLE
...
ENDDO
• Forall – optimize execution when order is not key
FORALL(i=1:n, j=1:n, y(i,j)/=0.) x(j,i) = 1.0/y(i,j)
High Performance Computing Initiative
The Basics IV
Data Statements:
REAL, DIMENSION(10) :: X
DATA X /1., 2., 3., 5., 7., 11., 13., 17./
Different Kinds of Variables : The sloppy way
REAL*4 X INTEGER*4 I ! 4 bytes
REAL(KIND=4) X INTEGER(KIND=4) I ! 4 bytes
REAL*8 Y INTEGER*8 J ! 8 bytes
REAL(KIND=8) Y INTEGER(KIND=8) J ! 8 bytes
!*** 9 significant decimals; exponent range -99 - +99,
PARAMETER :: LONG = SELECT_REAL_KIND(9, 99)
!*** Constants
1.7_long
!*** Variables
REAL(KIND=LONG) :: X
High Performance Computing Initiative
The Basics V
Main Program: Subroutines:
PROGRAM DO_THIS SUBROUTINE DO_THIS_TOO(A, B, M)
REAL, DIMENSION(M) :: A
! Declare Variables ...
INTEGER, PARAMETER :: N = 10 RETURN
REAL :: A, B, Y END
REAL, DIMENSION(N) :: X
• Function
! Call a Subroutine
CALL DO_THIS_TOO(X, Y, 5) REAL FUNCTION MY_FNC(Y, N, I)
REAL, DIMENSION(N) :: Y
! Use a Function ...
A = MY_FNC(X, N, 3)
MY_FNC = Y(I)
RETURN
! Use an Array
END
B = X(3)
END
High Performance Computing Initiative
Old Stuff : You don’t want to use this!
It’s obsolete,
but you’ll find it in Legacy Code
• Encode / Decode (Before Strings were introduced)
• Computed Goto
• Equivalence (Variables share Position in Memory)
• Non-Integer Indices
• Alternate Return
• Entry Statement (Subroutine can have multiple entry points)
• Statement Functions
High Performance Computing Initiative
Files & Formats in Fortran
Open/Close a File Read, Write and Format:
OPEN (UNIT=1, FILE=’my_file.txt’)
WRITE (1, ‘(A)’) ‘Hello world!’
WRITE (1, 99) ‘Hello world again!’
99 FORMAT (A)
YEAR = 2008 (declared as an INTEGER
HEIGHT = 8840.7
WRITE (1,’(A,I4)’) ‘This is year’, YEAR
WRITE (1,’(A,F6.1,A)’) ‘Height of Mt. Everest:‘, HEIGHT, ‘KM’
CLOSE (1)
I4 : Integer with 4 Digits
F6.1 : Real with 6 Digits and 1 Digit after the decimal point
• Format Specifier for Strings (A), Integers (I), Reals (E,F,G),
Logicals (L), etc.
• The number specifies the number of digits
High Performance Computing Initiative
F77 F90 : Old vs. New I
Memory allocation : Static / Dynamic
!*** Static
!*** ------
INTEGER, PARAMETER :: N = 10
REAL, DIMENSION(N) :: Y
!*** Dynamic
!*** -------
REAL, ALLOCATABLE, DIMENSION(:,:) :: X ! Declaration
...
ALLOCATE (X(10:20), STAT=ISA) ! Executable Statement
ALLOCATE (Y(N,M), STAT=ISA)
!*** Check Error-Status (ISA <> 0 is an Error!)
IF (ISA .NE. 0) THEN
STOP ‘An Error has occurred in Subroutine ...’
ENDIF
High Performance Computing Initiative
F77 F90 : Old vs. New II
Global Memory : Common Block / Module
!*** Global Variables : Common Block
!*** Add to Subroutines that use DATA
COMMON /DATA/ X, Y, Z
!*** Modules : Declaration
MODULE DATA90
REAL, PARAMETER :: C = 2.997925E5
REAL, DIMENSION(10,20) :: X
END MODULE DATA90
!*** Modules : Use
SUBROUTINE DO_THIS(...)
USE DATA90
!*** If module contains subroutine
MODULE vecmath
CONTAINS
FUNCTION vecadd(a,b)
...
END FUNCTION
...
END MODULE vecmath
High Performance Computing Initiative
F77 F90 : Old vs. New II
• Global Memory : Modules and scope
REAL FUNCTION minimum(a, b, func)
! returns the min. value of the function func(x)in (a,b)
REAL, INTENT(in) :: a, b
INTERFACE
REAL FUNCTION func(x)
REAL, INTENT(IN) :: x
END FUNCTION func
END INTERFACE
REAL f,x
:
f = func(x) ! invocation of the user function.
:
END FUNCTION minimum
High Performance Computing Initiative
Memory
- A Variable is an Address in Memory
- An Array is the first Address of a Chunk of Memory
!*** Reserving Space : 200 Elements in Memory
DIMENSION X(10,20), Y(200)
!*** Referencing : Element (5,6) is Element # 5 + 6x10 in Memory
X(5,6) = 5.
!*** Old and very confusing : EQUIVALENCE
!*** X and Y share the same Memory location
EQUIVALENCE (X(1,1), Y(1))
Now : X(5,6) is the same location in Memory as Y(65)
Even more confusing :
EQUIVALENCE (X(1,1), Y(3))
Now : X(5,6) is Y(63)
High Performance Computing Initiative
F90 Features I
• Dynamic Memory Allocation
• Automatic, Deferred Shape Arrays
• Using the Heap and the Stack
• Array Syntax
• Modules
• Derived Datatypes
• Exit and Cycle in For-Loop
• Intent In/Out
• Inquiry Functions
High Performance Computing Initiative
F90 Features II
Array Syntax:
REAL, DIMENSION(10) :: X, Y
REAL, DIMENSION(10,20) :: Z
X = Y
X(1:10) = Y(1:10)
X(1:5) = Y(2:6)
X = Z(1:10,3)
Intent In / Out:
SUBROUTINE DO_THIS(X, N)
INTEGER, INTENT(IN) :: N
REAL, INTENT(OUT), DIMENSION(N) :: X
High Performance Computing Initiative
F90 Features III
Inquiry functions:
REAL*4 :: X
DIGITS(X) ! Number of significant digits
EPSILON(X) ! X + ε <> X
HUGE(X) ! Largest Value
MAXEXPONENT(X) ! Maximum Exponent
TINY(X) ! Smallest positive number
...etc.
High Performance Computing Initiative
F90 Features IV : Derived Datatypes (I)
TYPE PERSON
CHARACTER(LEN=10) :: NAME
REAL :: AGE
CHARACTER*6 :: UT_EID
END TYPE PERSON
TYPE(PERSON) :: YOU
TYPE(PERSON), DIMENSION(10) :: WE
CALL DO_THIS (NAME, AGE, UT_EID, N) ! Hand down all Elements
SUBROUTINE DO_THIS (NAME, AGE, UT_EID, N)
CHARACTER(LEN=10), DIMENSION(N) :: NAME
REAL, DIMENSION(N) :: AGE
...
CALL DO_THIS (WE) ! Hand down a Derived Type
SUBROUTINE DO_THIS (WE)
USE MY_DEFINITIONS ! Put the definitions in Module
TYPE(PERSON), ...
High Performance Computing Initiative
F90 Features IV : Derived Datatypes (II)
! Define Derived Datatype
TYPE PERSON
CHARACTER(LEN=10) :: NAME
INTEGER :: AGE
CHARACTER*6 :: UT_EID
END TYPE PERSON
! Declare Variables of TYPE(PERSON)
TYPE(PERSON) :: YOU
TYPE(PERSON), DIMENSION(10) :: WE
! Use Variables
YOU%NAME = ‘John Doe’
YOU%AGE = 34
YOU%UT_EID = ‘jd3456’
WE(1) = YOU
WE(2)%NAME = ‘Jane Doe’
WE(2)%AGE = YOU%AGE
WE(2)%UT_EID = ‘jd3457'
High Performance Computing Initiative
F90 Features V : Modules
file: my_types.f90
------------------ Compilation:
MODULE MY_TYPES ifort -c my_types.f90
TYPE PERSON --> my_types.o
CHARACTER(LEN=10) :: NAME my_types.mod
...
END TYPE PERSON
END MODULE
A second file is created (.mod)
Subroutines that use the Module need this file!
file: print_person.f90
----------------------
SUBROUTINE PRINT_PERSON(A_PERSON)
USE MY_TYPES
TYPE(PERSON) :: A_PERSON
WRITE (*,’(A,A)’) ‘Name of the Person : ‘, A_PERSON%NAME
RETURN; END
Compilation:
ifort -c print_person.f90 (this needs my_types.mod)
--> print_person.o
High Performance Computing Initiative
Fortran 90+ : Beyond this Talk
• Interfaces
• Object Oriented Programming* *Use with care!
• Pointers, Linked Lists*
• Interoperability with C
• Floating Point Exception Handling
• Recursion
• OpenMP & HPF
• Asynchronous Output, Stream Attribute, Flush
• Parameters from the Command Line
• Getting Environment Variables
High Performance Computing Initiative
New to Fortran 90?
Consider these Features First
• Dynamic Memory Allocation
‣ Array sizes determined at Run-time
‣ Avoid Recompilation
• Derived Datatypes
‣ Assemble Variables into Logical Groups
‣ Avoid Long Parameter Lists
• Modules
‣ Declaration of Derived Types
‣ Global Arrays (if you choose to use them)
High Performance Computing Initiative
Understand the Compiler
• Compilers
• Intel : available on the TACC clusters (Lonestar & Ranger)
• Portland Group (Ranger)
• g95 : Free compiler by Andy Vaught (g95.org), very impressive!
• Optimization (Intel compiler)
• Optimization level : -O<n> : with n=0,1,2,3
• Architecture : -x<L> : with L=S, T, W, ...
• Interprocedural Optimization : -ip, -ipo
High Performance Computing Initiative
Compilers
• Enable floating point exceptions : -fpe0
• Understand what is done at compile time and at run time
• Optimization changes results (slightly)
• Select a default : -i4, -i8, -r4, -r8, -d8
• Be aware of internal type conversions
INTEGER :: I
REAL*4 :: X4
REAL*8 :: X8, X
X = I * 2. * X4 * X8
X = REAL(I) * 2. * X4 * X8
!Better, but still 2 or even 3 type conversions necessary
REAL(I) : Integer to Real
X4 : REAL*4 to REAL*8
(2. : REAL*4 to REAL*8)
High Performance Computing Initiative
Optimization : an Afterthought?
Optimization by the Compiler
• Vectorization, Inlining, Loop-unrolling, Copy propagation, etc.
Analyze / Modify / Rewrite parts of the code
Changing the Memory Layout
Fetching Data from Main Memory is very costly
Data is transferred in chunks from the Memory to the Cache
Make most of the Data in the Cache!
• Data is fetched in bulk and stored in the fast Cache
• Access the Data in Stride-one fashion First Index runs fast!
Order in Memory:
REAL, DIMENSION(1000,1000) :: A
A(1,1), A(2,1), A(3,1),...,
DO I=1, 1000 A(1000,1), A(1,2),...
DO J=1, 1000
X = A(J,I) ! Stride 1 first Index runs fastest
Y = A(I,J) ! Bad use of the Cache
High Performance Computing Initiative
What is Debugged?
Different Kinds of Problems:
I) Code gives the wrong Result
II) Floating Point Exceptions
III) Segmentation Faults
III) Overwriting Memory; Problem is intermittent, depends on
Optimization Level, and changes with added WRITE
Statements; Debugger doesn’t really help!
II) Debugger helps; Problem can depend on the
Optimization level, but is generally much easier to fix than III)
I) Check Results thoroughly!
It’s not enough for the code to run until the end!
If FPE’s are not fatal, any code can run (kind of)
High Performance Computing Initiative
How to Debug?
Go through the code step-by-step with the debugger?
Sometimes that might be necessary
Think! Look at the code, and add Debug Statements
Do not question the Compiler; there are no Compiler Bugs!
• Add formatted WRITE Statements, otherwise you
end up debugging the Debug output
• Consider adding “permanent” Debug statements
Revisiting the Subroutine later? : just flip the switch
INTEGER, PARAMETER :: IVERB = 2
IF (IVERB .GE. 1) WRITE (0,’(A)’) ‘This is Subr. DO_THIS’
IF (IVERB .GE. 2) THEN
WRITE (0,’(A,I2,1X,F12.5)’) ‘Parameters are : ‘, I, X
ENDIF
High Performance Computing Initiative
Obvious Rules
• Add Comments
• Don’t use obsolete and obscure constructs
• Use F90 constructs rather than their F77 counterparts
• Don’t use Vendor specific extensions (Easy with F90+)
• Indent DO-Loops and IF statements
• Use GOTO’s rarely
... and not so obvious, perhaps
Make your output readable!
• Always format your output
• Normal Output
• Debug Output
• Always add Table Headers
High Performance Computing Initiative
Software Development
What are you up against?
• Sufficient speed to write code
• Meet User’s (or your own) needs
• Efficiency
• Stability
• Upgrades
• Documentation
High Performance Computing Initiative
