When running Linux on IBM System z
Velocity Software Inc. / Hill Gang – July 14th
L @ l it ft
• Today’s “legacy system”! Times have changed.
h i ff i i ll i
• How changes in IT are affecting your installation.
– IT Trends & Directions
– TCO Considerations
• Measure, Track, and Report - Why bother?
• Results – What should you expect?
• Tools of the trade: The good, the bad, & the sad
• This session is based on actual customer experience &
history as it relates to migrations from x86 (commodity)
server “farms” to Linux on z.
The i d bl th t d l th
• Th issues and problems they encountered along the way.
• Their efforts to resolve those issues and problems.
adapted overcame .
• How they adapted, overcame, and . . survived ! !
Definition: “Legacy System”
“A legacy system is an old computer system
or application program that continues to be
used because the user (typically an
organization) does not want to replace or
g y g
“Legacy can be a burden or heritage
Karl-Heinz Strassemeyer (IBM Corp)
IT in 2010 has a new set of issues.
Unfortunately, the “legacy systems” of the 90s are
creating more issues to deal with than the
mainframe of the 70s.
A nightmare to manage & administer.
Limited to horizontal capacity increases. (see below)
Increasingly expensive to maintain.
Today’s legacy systems are the byproduct of the
“add another rack” IT generation of the 90 s . . .
add rack 90’s
Challenges to the IT Professional . .
Space, Heat, Power, “Green” Efforts
Maintenance, Upgrades, Charge Back, Capacity
Planning, Performance DR Security
Planning Performance, DR, Data Security, and more
Internal IT Pressures
Migrations, Consolidations, Compliance Issues,
g , , p ,
Maximizing Resources, Flexible Capacity
Industry Strategies & Directions
Cloud / Storm / SUN-down / Sky-BLUE/ Cloudburst
What everyone wants from IT . .
Secure, high speed access to
data any time, from any
device, and from any where.
Even from a dive boat in
Turks Caicos working with
Neale on breakfast for this
event . .
What IT Execs are thinking:
Response from 100 IT executives at companies using Linux operating
system on IBM mainframes.
93% of respondents projected that their use of IBM's IFL (Integrated
Facility for Linux) would increase or remain steady.
The two main reasons:
1. Advantage of computing capacity available on their mainframe's
Linux on th mainframe would b more cost-effective th other
the i f ld be t
cost- ff ti than th
What is the greatest problem with your
primary data center ?
“Power and Cooling will be a top 3 issue with all CIOs in the next 6
to 12 months.” Michael Bell – Gartner Group
Spending Installed Base
(US$B) (M Units)
Power and cooling costs x8
Server mgmt and admin costs x4 45
New server spending 40
Source: IDC, Virtualization 2.0: The Next Phase in Customer Adoption,
Doc #204904, Dec 2006
IBM’s Global CEO Study
• Over 1500 CEOs were interviewed across all geos.
• Most intensive study conducted in this space.
• More than 80% see an increase in complexity as their
number one IT issue. Fewer than 50% say they are ready.
• Challenges differ from region to region:
Government R l i
– US / G Regulations
– Japan / Power shift to emerging markets
– China / “Thinking Global” as opposed to closed society.
• Conclusion: “No matter what the industry, those that can
manage and react to complexity will have the advantage
over those that can t.”
IBM’s Mainframe Cloud Agenda
• Focus on Mainframe Strengths
– Extend System z’s massive virtualization capabilities
• Leverage System z efficiencies
• Offer System z (mainframe) customers value beyond that
i th l tf l ti dd i
seen via other platforms / solutions and doing so
transparently to the broader community. (90 Day Free
Trials on IFLs)
“Keeping your head above the clouds STG Tech Conference 09
IBM’s Cloud Website: http://www.ibm.com/ibm/cloud/
System z for Cloud
As cost- 2009
“As the IT world prepares for cost-cutting in 2009, server
virtualization seems like one of the few technologies bound for
continued growth and success. . . . The IBM System z may be the
b enterprise-class virtualization platform available today . . .”
best enterprise- l i li i l f il bl d ”
Enterprise Strategy Group – January 2009 Report “IBM System z: The Enterprise Server
IBM’s Project “BIG Green”
• Consolidation of several thousand Linux servers to 30 IBM
z10s. (may end up being fewer than 20..)
• 80% Reduction in Energygy
• 85% Reduction in Floor Space
– 1,600 for z10s vs 11,600 square feet for Distributed
• Categories as a % of Gross Savings:
Internal IT Study for TCO
• Determine consolidation ratio Server Migration : x86 vs. zLinux
for given workloads (Linux,
Windows, etc.) 0
• Estimate annual cost to operate
virtualized servers for 3-5 15
Compare this to stand-alone
hi d l 10 x86/other
provisioning or public Cloud Virtual/IFL
offerings like Amazon
• N< PPR*PUR*CPF supports 0
that the more servers you can 1st 2nd 3rd 4th
consolidate to virtual, the lower Qtr Qtr Qtr Qtr
your cost. Consolidation Math
N= # of virtual servers / Proc Perf Ratio / Proc Util Ratio / Cores per Frame
Response Time Comparison
sponse Tim in Secon
10 20 30 40 50 60 70 80 90
Number f Virtual M hi
N b of Vi t l Machines
10 20 30 40 50 60 70 80 90
Number f Virtual M hi
N b of Vi t l Machines
But keep in mind, not everything was meant to run on z . .
Cost Per Image for Linux Servers
Cost Per Worklo Image in $k
250 Public Virt.
200 Private Vi t
P i t Virt.
150 zLinux Add
Detailed Cost Breakdown - 5 yr TCO
Platform 100 x3250 100 Amazon VMware & Five 7 IFLs on z10
Large EC2 x3950 – 8ways
Hardware $5 000 000
$5,000,000 $2 800 000
$1 080 000 $2 030 000
Software $21,490,000 $20,840,000 $2,380,000 $1,520,000
Facilities and $11,020,000 $4,020,000 $2,760,000 $1,210,000
Administration (admin only)
Total Cost for $37,510,000 $27,740,000 $6,220,000 $4,780,000
Cost per image $375,100 $277,400 $62,200 $47,600
IBM SWG White Paper / April 2009 & IBM Internal Study of 4000 xSeries Distributed Servers
System z Rising
• More than 1800 customers 2000
are using System z for 1800
• In 09, IBM added 54 new
“mainframe” customers 1000
for the purpose of Linux 800
• Customer experience and 600
results support the current
l h 400
strategy to migrate Linux
applications to System z. 2002 2004 2006 2008
The number of Linux IFLs sold/year
doubled between 2006 and 2008.
Unique Value of the IFLs
• Run at full MIPS even on
sub-capacity models 1.72 391%
4.4 1157% IFL Value Increase
• If/when you upgrade, the
1100% over G5
IFLs move up in power at 12
55% over z9BC
no additional charge. 8
• IFL’s have increased in 4 IFL Capacity
2 %IFL Increase
power by roughly 55% 0
over each previous
generation or 1100% since IFL Value Increase
Note: IBM delivered more IFL MIPS in 1Q10 than in 2009.
Horizontal vs. Vertical Scale (x86 vs. z)
What Horizontal vs. Vertical Scale means to
you (x86 vs. z)
• How about adding a second engine to a car at the flip of a
switch . . . without stopping.
• Increasing horse power from 180 HP to 450 HP at the flip
of a switch . . . without stopping.
• Or being able to do both . . . on the fly . . . without
• System z allows you to increase your capacity either
horizontally or vertically and it’s the only platform to
achieve this flexibility.
But don’t get yourself in a “hole” . .
Some applications run
better on z than others.
’ d d
It’s a good idea to
migrating to Linux on z.
What applications make a best fit?
WebSphere MQ Series
W bS h S i
CICS Transaction Gateway , IMS Connect for Java
WebSphere and JAVA applications development h / d dh
High I/O Bandwidth,
WebSphere Application Server (WAS), Portal Security and DR Requirements
Domino Simplifying the Support Model
Network I f
N t t t FTP, NFS t
k Infrastructure, FTP NFS, DNS etc.., Shortening end-to-end Path Reqs
Oracle Database Reduction in Network Traffic
Applications requiring top end disaster recovery Collocation of Applications
ComServer and Communications Controller for Consolidation of diverse loads
Virtualization and Security Services
Communicate Pro (VoIP) “Selecting Projects for System z”
Bill Reeder, IBM Corp.
What applications make a good fit?
UDB (DB2) Evaluate Server Choices
Informix, (IDS) Correct Application Availability
Apache web serving Supporting Applications
S ti A li ti
Political Issues within the Org.
SAMBA Potential Porting Issues
TIM/TAM (LDAP Services)
Existing Linux Workloads
What applications are difficult to move?
ISV and IBM applications that have not yet ported their
application to run on Linux on System z
Applications that by design run at VERY High sustained
utilization which I will define here as >95%.
Stand-Alone single applications as the only Linux for System z
Applications that are to internally sensitive to try and move
This URL is a link to the IBM software running and supported to
run under Linux on System z.
http // 1 ibm com/ser ers/eser er/ series/os/lin /soft are html
What workloads are moving to IFLs?
IBM Corp. 2010
Why bother to Measure, Track,
and Report ?
Len’s Travel Tips and other ways to reduce TCO . .
Len’s Travel Tips (continued)
The question becomes . . . . . .
When you hit the next “bump” on the road to better
do you keep buying new tires, or do you get the front end
aligned ? ? ?
Responsible IT professionals should always consider
• Performance Management to ensure service levels
• Capacity Planning to ensure future needs are met.
• Operational Alerts that detect issues such as
looping, exceeding disk capacity, etc., and doing
so for hundreds/thousands of servers concurrently.
• Charge back and accounting information to allow
your business to charge for resources consumed.
Real Life Situations
• Financial in NY Area - Crit Sit / 2 months / SRM parms
wrong – SLES8 server wouldn’t initialize
• State of Montana - Crit Sit / 1 month / Linux Server
Looping, 1 IFL offline. POC Failed.
• University Crit Sit - Response Time issues.
• Financial in DFW Area - System spikes every 5 minutes .
. Wrong choice of monitors . .
• Industrial – Decided to buy 20G of Storage rather than use
best practices and set the correct parms..
• Industrial #2 – “We keep adding IFLs.” (now leaving the
platform . .)
Large Insurance Company
• Pain Points (history) –
– Too many servers
– Space Power Cooling
– Inability to Allocate Resources Where Needed
– New Apps - Time/Expense of Basic Trials and Testing
• Today – (strategy deployed in less than 4 months)
– +700 Servers on 2 Boxes / Dynamic Capacity Options
– 100,000 Active Users
– Zero Production Outages Since 2005
$3M/Year in Savings
– $3M/Y i S i
Some installations are smaller:
• One CEC (system)
• Two IFLs (one production, one test)
• 20 Linux Servers replaced, more planned
• Proof of Concept – 90 Days (free trial – including zVPS)
• Savings – space, power, admin, etc.
• "My TSM debugging could not have been done without Velocity.
IBM’s DB2 Level 2 support was wondering where I got some of the
info ;-)” …. and that level of detail…
• Using IFL.
“Using any other monitor would cost me at least 5% of my IFL That
means one (1) IFL for every 20 that I install just to monitor my
• “They gave me their monitor for free.”
– What about the drag on your system?
– What b t d di ti th Tb t f t f th ft ?
Wh t about dedicating more than a Tbyte of storage for the software?
– What did it cost you in time to install? (Did it install?)
– What does it give you for data, health checks, support, etc…?
– Believe me, IT WASN’T “Free”.
B li “F ”
Or, one or more of the following:
• “Time to add another IFL.”
• “It’s just the way Linux runs on System z.”
• “You might consider using System p instead.”
• “You might consider leaving that appl. on the x86.”
• “This was your idea. You might consider a new job.”
• “I told you this would never work on a mainframe.”
• “DRT…. Dead Right There ”
However, accomplishing the basics does
create certain challenges . . As y move
forward, what should you expect?
It’s hardly likely that you would trust your
y p f g
system’s performance to an astrologist . . .
• A single suite of products that can deliver whatever is
necessary to manage performance on z/VM and zLinux.
– Keeps cost to a minimum.
– Simplifies service, upgrades, education, etc.
Single d i t f
– Si l vendor interface
• A vendor with a history for delivering best-of-breed
products and support.
• A cross industry install base with references in all major
Velocity Software Inc.
• Founded in 1988 by former IBMers.
• Velocity products are used by hundreds of installations
worldwide; many are IBM showcase customers.
• IBM uses Velocity’s products internally and makes frequent
references to our products in their Redbooks.
• We have a working relationship with IBM System z labs in
Endicott(z/VM), Poughkeepsie(zOS), and Boeblingen(zVSE).
• Velocity has participated in every VM Early Support Program
since 1988 and has been a PartnerWorld member since 89. 89
• Velocity presents research at many conferences including:
– SHARE, GSE, CMG, WAVV, and Local Linux User Groups
Velocity Software Delivers :
• Total Data Accuracy
– When necessary, identifies, prorates and corrects inaccurate data.
• Complete Data Collection
– Variety of Platforms (NT, Linux, SUN, HP, etc)
– Standard Interfaces (SNMP and NETSNMP)
Concurrently across 100s or 1000s of servers.
– C l 100 1000 f
• A 100% Data Capture Ratio
y y p
– Best in the industry . . . Why measure incomplete data . . ?
• Minimum impact on processors by monitoring agents.
– Velocity’s target is .1% or less of ONE processor
Not All Performance and Systems Management Tools are Created Equal . . . .
z/VM – Linux Requirement Velocity Software Product All Other Vendors
Support All Linux YES NO
Charge Back Accounting YES NO
Capacity Planning Info. YES NO
Low Cost Operation YES NO
Low O h d A
L Overhead Agents YES NO
Performance Serv. (zTUNE) YES NO
Performance Education YES NO
Included in IBM Redbooks YES NO
Standard Open Linux Agent YES NO
Network Perf. Monitoring YES NO
Accounting – VM Level YES NO
Accounting – Linux Process YES NO
Accounting – Linux Applic. YES NO
MXG and MICS Interfaces YES NO
“Trying to determine performance issues without
Velocity’s suite, is like asking a doctor to give you an
y , g g y
examination while wearing your clothes.”
And, the results are pretty much the same.
Quote from an IBM D.E. at WAVV
• zVPS is Velocity’s Performance Suite. It is a
collection of several components including:
– zPRO (optional)
– zTUNE (optional)
zVPS Overview (cont)
• Provides real time metrics on all facets of server
performance including z/VM, Linux, and other
distributed (SUN Windows, etc.)
di ib d servers (SUN, Wi d )
• Full function capacity planning and interfaces to
planning tools such as MXG and MICS
• Complete and accurate charge back accounting
information for Linux applications and z/VM.
• Operational alerts for up to 1000s of virtual
servers and immediate detection of performance
IBM System z
zVPS can monitor z/VM
and measure the
activity of Linux on Distributed Servers (x86)
System z in addition Linux (IFL) &
to distributed servers
like SUN Solaris, zVPS Linux
Linux, HP, Windows,
Net: You can determine the level of use for other
any server in your network.
Data can be used for Trend Analyses
IFL CPU Utilization – 12 Months
Jan Mar May Jul Sep Nov
Hmm, CPU spikes every 10 minutes
Which user class is causing the problem?
In this case it’s one of the Linux Servers.
Drill further -> guest Suselnx2
fy ff gp ( )
Identify the offending process (“smallstr”)
Unique Capabilities of Velocity Software
Cloud Enablement Tool for Systems Administrators
Cloning f Li S S l i VSE
• Cl i for Linux Servers, Solaris, VSE, CMS
users and more.
• Provides administrators the ability to define
servers based on “golden images”.
• Manages z/VM security (RACF)
• Provides an easy-to-use, intuitive front-end to
Dirmaint via a Tab approach and syntax checking
for defining directory entries.
• Documentation / Command search capabilities
• Configuration analysis to ensure best possible
configuration for performance.
• Dozens of Automated “health checks” for both
Linux and z/VM.
• Analyses of your Capacity and Performance data.
• Recommendations to improve p performance and to
optimize resources whenever it’s requested.
Velocity works as an extension of the local team.
• We are committed to providing the best z/VM performance
• Our zVPS Product Suite is a complete set of Performance and
Systems Management tools designed exclusively for z/VM
and Linux on IBM System z
• We provide excellent product support services, and education.
• Our install base is both global and diverse.
l i S f i h i d l d i hi by
• Velocity Software is the recognized leader in this space b
most of IBM’s z/VM – Linux “showcase” accounts.
Proven Performance from Velocity Software
The next step is yours . . .