The Business Case for Agile Languages
Stephen Ferg Bureau of Labor Statistics ferg_s@bls.gov www.ferg.org
1
In the beginning...
System programming languages
Assembler, C Fortran, Cobol, Algol PL/1, Pascal, Basic, C++, Java
Command languages
JCL TSO CLists, CMS "execs" Batch files
2
"Little Languages" appear...
awk sed
3
Scripting languages evolve
UNIX shell scripting Rexx Tcl, TK
Perl Python PHP Ruby
Some have higher-level object-oriented features that make them powerful application development languages in their own right.
4
Agile programming languages
"Scripting" languages aren't just for scripting any more. We really should stop calling them "scripting" languages. "Agile" languages would be more accurate.
5
What is an agile language?
Interpreted
requires a run-time interpreter or virtual machine
Untyped or dynamically typed
No data declarations
No compilation step
6
The hallmark of agile languages...
Vastly increased productivity!
7
Productivity ::= the number of machine instructions that a programmer can produce per year.
A programmer can write roughly the same number of lines of code per year regardless of language.
• Assembly Language One line = one machine instruction • System programming languages One line = 3 to 7 machine instructions • Scripting/agile languages One line = 100s to 1000s of instructions
"Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html
8
Language Levels and Productivity
1000 Scripting Languages
VB
Instructions/Statement
100
Python, Perl, Ruby, TCL Java
10
C++ C Assembler
System Languages
Degree of Typing Strong
9
1
None
From "Scripting: Higher Level Programming for the 21st Century" by John K. Ousterhout. This version prepared by Dana Moore and updated by Stephen Ferg
The results indicate that, for the given programming problem, 'scripting languages' ( Perl, Python, Rexx, Tcl ) are more productive than conventional languages."
University of Karlsruhe (Germany) Technical Report, March 2000
10
A good example... Python
Developed in 1991 by Guido van Rossum
Best of breed Powerful / flexible Easy-to-learn / use Open source
A fan of
Monty Python's Flying Circus
11
Developer Reports
I find that I'm able to program about three times faster in Python than I could in Java, and three times faster in Java than I could in C.
Andy Hertzfeld
When a 20,000 line project went to approximately 3,000 lines overnight, and came out being more flexible and robust ... I realized I was on to something really good.
-- Matthew "Glyph" Lefkowitz
...the lines of Python code were 10% of the equivalent C++ code.
-- Greg Stein, eShop
12
Productivity = Web speed
May 4, 2004
When burglars brought down the Internet link to Ziff-Davis' Manhattan offices, open-source software - and Sean Gallagher's personal Web server - kept eWEEK.com's stories flowing.
For about six hours yesterday, the staff of eWEEK.com was treated to a lesson in emergency improvisation. But thanks to 123 lines of Python code, running on a $7-amonth personal Web site on an Apache Web server somewhere in California, you probably didn't notice.
13
Fast to write... but maintainable?
... the real punchline of the story is this: weeks and months after writing my Python program, I could still read the code and grok what it was doing without serious mental effort.
Eric S. Raymond, author of The Cathedral and the
Bazaar
14
"Programming is fun again!"
Over and over on comp.lang.python :
"Now that I've discovered Python, I enjoy programming again!"
15
The ability of agile languages to increase productivity and maintainability leads to... increasing use of scripting/agile languages.
16
Scripting languages represent a different set of tradeoffs than system programming languages. They give up execution speed and strong typing but provide significantly higher programmer productivity and software reuse. This tradeoff makes increasing sense as computers become faster and cheaper compared to programmers. John Ousterhout, developer of Tcl I think there is a trend ... Dynamically typed languages such as Python, Ruby, and even Smalltalk will be mainstream industrial languages in the coming years.
Robert C. Martin, author of Agile Software Development
17
Sounds good, but ...
18
Is open-source safe ?
19
Can you trust open-source software?
Python is an "open-source" language. It has no vendor. Does that mean we'll have support problems? What about... Vendor longevity? Consulting & training support? Books and reference materials? Tools? IDEs, debuggers, screen-painters?
20
Is anybody using open-source software? Is open-source software used in the Federal government?
21
Who is using open-source software?
"LAMP"
Linux Apache MySql PHP | Perl | Python
Apache has overwhelmingly dominated the Web server market since 1996.
PHP is the most popular Apache module, running on almost 10 million domains (over a million IP addresses). ... and then there's ...
22
DoD
In 2002, a Mitre study found 115 FOSS (free and open-source) products in use in the U.S. Dept. of Defense.
http://egovos.org/pdf/dodfoss.pdf
23
... and IBM
In September 2003, IBM began promoting Linux with a series of television ads depicting a young boy receiving lessons from famous innovators and teachers. The boy represents the next generation of humanity, learning from teachers who – like the open-source community – freely share their accumulated expertise.
24
Why would a Federal agency use open-source software?
From Government Computer News November 20, 2000 ...
The experience of the NASA Acquisition Internet Service (NAIS) development team ...
25
Our proprietary Web development application that promised interoperability with another vendor’s database software failed to interoperate... Then we discovered Perl and have been using it for the last five years for all NAIS applications.
Recently, price restructuring for a commercial DBMS threatened to consume most of the NAIS budget. We decided to convert to MySQL. Our tests showed MySQL could perform NAIS functions faster. Cost of the optional technical support was about 1 percent of that for the commercial product.
Technical support for MySQL has been excellent when we needed it, plus there are hundreds of Web sites that offer free help and support for such open-source products. We plan to evaluate the Apache HTTP Server to correct limitations of the commercial Web server we currently use. 26
Who is using Python? What are they doing with it?
Industrial Light & Magic, maker of the Star Wars films, uses Python extensively in the computer graphics production process.
Disney Feature Length Animation uses Python for its animation production applications.
27
Google, a leading internet search engine, is powered by Python. Yahoo uses Python for its groups site, and in its Inktomi search engine. The Philips (formerly IBM) Fishkill semiconductor manufacturing facility uses Linux and Python for factory tool control.
28
NASA uses Python in several large projects, including a CAD/CAM system and a graphical workflow modeler used in planning space shuttle missions. The National Institutes of Health and Case Western Reserve University are building cutting-edge genetic analysis software with Python. The National Weather Service uses Python to prepare weather forecasts.
29
Lawrence Livermore National Laboratories is basing a new numerical engineering environment on Python. The Theoretical Physics Division at Los Alamos National Laboratory uses Python to control large-scale simulations on massively parallel supercomputers.
30
US Navy uses Python & Zope for a web based workflow system US Dept. of Agriculture uses Python & Zope for massive collaboration US Dept. of Labor uses Python & Zope for the Workforce Connections learning management system
31
What if Guido got run over by a bus?
... or "What about vendor longevity?"
32
Common Misconceptions
Open-source software has no vendor
The vendor of an open-source product is the (single) person who developed it.
Anybody can change (and maybe mess
up) an open-source product.
33
The Python Software Foundation
A non-profit organization
Holds Python's intellectual property rights
Produces the core Python distribution: the Python language, standard libraries, documentation, source code, etc. Manages contributions to the Python codebase
34
The PSF will outlive Guido. There is no guarantee of any commercial vendor's long-term survival.
Oh, by the way... you do have software escrows for all of your commercial software don't you?
35
Is OSS as good as commercial software?
Vendors and products vary widely in both the commercial and open-source arena.
The fact that a piece of software is commercial is no guarantee of its quality.
The best open-source software is as good as the best commercial software.
36
The Bottom Line
Each product and vendor should be evaluated on its own merits, regardless of whether it is commercial or open-source.
Some open-source software is in the same league as the best software anywhere, commercial or not.
37
What about market share? Open-source: mindshare How alive is the user community?
International Python Conference
IPC - in USA since 1994 In conjunction with OSCON since 2002
EuroPython conference
in Europe since 2002
38
Newsgroup Activity
comp.lang.* December 2002
Statistics compiled by Aaron K. Johnson.
java c++ c perl python basic ruby lisp tcl pascal smalltalk fortran cobol
26953 19913 13874 10486 9647 7909 6466 6132 5256 4229 2398 2355 1845
39
What are its capabilities, features?
Frank Stajano
Python is famous for its "batteries included" philosophy
40
The Python Standard Library
GUI strings regular expressions database connectivity HTTP, CGI, HTML, XML numeric processing debugger object persistence
41
What about execution speed ?
Slower than a systems language, but ...
Modern hardware processors generally make language speed a non-issue Most applications are limited by speed of database or network connection, not programming language
Remember when they said that relational databases would never catch on? "Too slow" they said...
42
How will we ever survive without type-checking?
I found that type issues simply never arose. My unit tests kept my code on the straight and narrow. I simply didn't need static type checking.
Robert C. Martin
43
Where will I find support?
comp.lang.python -- Outstanding!!
Many online books and tutorials
Consulting and contracting firms that specialize in open-source training and development
44
45
The Bottom Line...
"Use the Best Tool for the Job: Put Both a Scripting and Systems Language in Your Toolbox"
- Bill Venners
http://www.artima.com/commentary/langtool.html 46
How Do I Start?
Investigate scripting and agile languages
Navigate your standards process, if necessary
Provide training, learning materials Start experimenting ....
Scripts, internal/administration utilities One-time, throw-away programs Test scaffolding Prototyping
47
A prediction: you will move “The Tipping Point”
It is possible to write programs to automate every task. But you don’t.
Python makes it easy enough
48