IT – Enterprise Applications
ITEA APPLICATIONS SERVICES
What is Production Support?
Production Support includes a variety of activities, both reactive and proactive, that keep ITEA-supported
enterprise applications operational, accurate, and effective as currently designed and implemented. These
1. Resolving, or helping clients or other ITaP units to resolve, incidents caused by procedure malfunctions,
program logic errors, technical environment malfunctions, process set-up errors, user errors, and other
issues. The objective is to get application appropriately functioning, accurate, and back on schedule as
soon as possible. This includes “night-calls” (and other “off-hour” support for production services), off-
hour support for clients as established in an SLA or other support agreement, and M-F daytime support
This is reactive support.
2. Providing day-to-day “operational” consulting support for clients and ITI operation/production, especially
to help assure quality system operations or to avoid mistakes and malfunctions. (This does not include
“what if” or “what would it take” research questions raised by clients, which are considered “Pre Work”
on a potential RFS.) This “operational” consulting is also reactive support, and may include “off-hour” as
well as M-F daytime activities.
3. Resolving, or helping clients or other ITaP units to resolve, underlying problems or root-causes of
incidents to avoid further incidents. These may be reactive issues, requiring immediate action if impact
is critical and immediate; or follow-up action that are planned, scheduled, and resourced as an
“Application Fix” through an RFS. (i.e. For a variety of reasons the issue may not require immediate
action.) Again, situations requiring reactive, immediate action may include “off-hour” as well as M-F
4. Providing support for critical processing or other planned events. These may be seasonal or recurring
(e.g. semester start, semester end, fiscal year end, calendar year end, etc.), or ad hoc (e.g. special
compressed schedules, extra runs, out-of-cycle runs, etc.). These are proactive support activities that
are planned, scheduled, and resourced.
5. Providing support to help assure applications remain stable and operational when impacted by externally
driven IT issues. These issues may include technical infrastructure changes (e.g. operating system or
other infrastructure software/hardware upgrades), technical production changes (e.g. Autosys to UC4
change), or IT process/system security changes (e.g. emerging mandates from IT S&P). Again, these
are proactive support activities that are planned, scheduled, and resourced.
6. Providing ongoing monitoring or testing of production activities to initiate corrective action as needed and
to help assure processes are in “good health”. This includes reviewing logs, validating set-ups, running
process previews, etc. These activities should be planned and assigned.
There may be other “mandated” changes driven by government mandates, University mandates, or changes to
interfacing systems. Although these changes may be considered “in support of a production application”, and
may not enhance or add value to the application, these are business initiated changes and will arrive via an RFS
for planning, scheduling, and resourcing. These changes are not considered “Production Support” for this
How does Production Support operate?
Production Support will operate in two different modes for Reactive and Proactive support.
Reactive production support activities include items 1 and 2 above, plus any situations for item 3 above that
require immediate action. M-F daytime support and Off-hours support (a.k.a. “night calls”) will utilize two
different models or approaches.
74b92649-6932-4032-9045-fecb52354354.doc Page 1 of 4 12/19/2011
M-F daytime support - Each competency center (CC) will have one or more email groups receiving, logging,
assessing, and tracking incidents for the CC’s supported applications. The group will also resolve any and all
incidents as possible within the groups technical ability, system knowledge, and resources. These group email
address will be used for ongoing dialogue on an issue to keep the whole group informed. Clients for each CC
will be directed to utilize the appropriate email group(s) to report and discuss issues. The email groups by CC’s
Business Services (BS) Competency: (4 staff to be assigned to all groups, re-evaluate after 1 month)
ITEA Prod Support - Business Services
ITEA Prod Support - Physical Facilities
Student Competency: (4 staff to be assigned, rotate monthly)
ITEA Prod Support - Student
UD/PUID Competency: (2 1/2 staff to be assigned, variable but probably rotate after 1 month)
ITEA Prod Support - UD/PUID
Staffing (levels and assignment), rotation frequency, issue re-assignment procedures, and detailed escalation
rules for a CC will be determined by the CC ASM, although each email group will consistently log, track, and
report issues. These email groups will include only part of the CC staff, technical knowledge, and system
knowledge, so escalation rules will provide the means to further involve CC resources to resolve issues.
Generally, the Production Support email group will have other CC staff available for consulting to resolve issues,
but at times further involvement from other CC staff may be needed to effectively accomplish the needed work.
This may be due to the volume of issues, complexity of issues, or system or technical knowledge required. The
Production Support manager will help facilitate involvement of others according to the escalation rules
established by the CC ASM. Ultimately, the CC ASM will have authority over the CC resources, escalation
rules, and the balance of production support verses other activities. The Production Support manager will work
with the email groups to assure receipt, logging, acknowledgement, re-assignment, tracking, and reporting of
issues, and to further facilitate resolution of issues.
Off-hour or “night call” support - Night calls vary by CC (i.e. in terms of urgency, volume, underlying
application situations or SLAs, etc.). These calls typically relate to nightly processing for critical batch windows,
pre-arranged weekend or other “off-hours” work by the client, or “off-hours” services delivered to a group of
constituents (i.e. students, alumni, public, upper management, etc.), etc. Due to the nature of the processing
and service, it’s important that the calls are handled quickly and appropriately. In some cases due to volume,
it’s important that the work-load is appropriately balanced across the CC staff. In all cases it is important that
there is backup to assure coverage. Each CC will determine its specific approach for staffing off-hour support.
The basic approaches include an “on-call” rotation chart, or use of the existing “depth chart” to indicate “on-call”
staff. (Each CC will maintain a “depth chart” showing application knowledge and experience.) A rotation chart
will document who is on-call, and who are the backups for each application or application area over a period of
Business Services Competency:
BS CC will utilize their depth chart to publish who is on-call, and who are the backups for each
Student Competency: (4 staff to be assigned, rotate after 1 month)
Student CC will utilize their depth chart to determine an on-call rotation with backup on-call
UD/PUID Competency: (2 1/2 staff to be assigned, variable but probably rotate after 1 month)
UD/PUID CC will utilize their depth chart to determine an on-call rotation with backup on-call
The Competency Center ASMs will be responsible for maintaining CC depth charts and establishing rotation-
chart assignments. The Production Support manager will monitor to make sure the information is current,
maintained in a central repository, published with ITI, and utilized to facilitate support.
Proactive support (items 4, 5, 6, and part of 3 above) will be assignments for any staff in the CC, and will be
planned and scheduled.
74b92649-6932-4032-9045-fecb52354354.doc Page 2 of 4 12/19/2011
How do clients contact Production Support?
E-mail and the email groups will be the primary way clients will report issues. In order to promote the use of
email, it will be important to acknowledge the receipt of the email, provide an indication as to the next step, and
when the client should expect to hear more. These email groups will be published and re-enforced with clients.
To help assure as much cross-coverage and backup on issues as possible, the email group address should be
consistently included in all ongoing dialogues.
From time-to-time clients may have the need to talk directly to a person rather than interact via email. These
may be situations of extreme urgency or follow-up on an issue. This has been referred to as calling the “911”
line or calling the “bat phone”. Initially, the Production Support manager’s phone with the CC ASM’s phone as
back-up will be published with the clients as the direct-line outlet. Once we know more about possible situations
requiring and usage of the “911” line, we may elect to replace or augment with a rotating cell phone for further
How do we interact with ITI?
E-mail and the email groups will also be published and utilized with other ITaP units to help assure ITEA
coverage with these groups. Again, the email group address should be consistently included in ongoing
dialogues to help assure as much cross-coverage and backup on issues as possible.
To help assure coordination and planning, Production Support manager will be the ITEA-AS point-of-contact
(POC) for ITI initiatives requiring ITEA support (e.g. zOS upgrade, Oracle upgrades, etc.), although these
initiatives will be handled as “requests” on an ITEA RFS. This work will be planned, scheduled, and resourced
Although only one RFS from ITI is needed, an RFS may generate multiple Tasks for the applications affected.
What are the standard operating procedures for PS.
How are problems recorded? - When received, issues are to be logged in the Task system as “Problem
Investigation” or “Emergency Fix”. Once logged, the issue will be tracked, and the solution or resolution will be
recorded in the Task record. Once resolved, the Task will be closed with an appropriate completion date If the
item is not a production process or application “incident” (e.g. it is a request for service that should be an RFS,
question that is Pre-Work, etc.), the item will be forwarded to the CC ASM (w/ cc: to PS ASM)..
How are problems reassigned? - Although all PS group members will receive PS group emails, there are two
models for assignment of arriving issues to help assure coverage. In the first model, each PS group member is
assigned one or more areas and is primary staff for handling issues for those areas. In the second model, a
“point person” or lead will reassign issues to a specific PS group member. In both models, the Production
Support ASM is also on the email list and monitors to help prevent fall-through and assure coverage of all
issues. The CC ASM will determine which model best fits the clients’ and CC needs.
Area Area Area Area
1 2 3 4
Area Area Area
2 3 4
Production Support ASM
Production Support ASM
74b92649-6932-4032-9045-fecb52354354.doc Page 3 of 4 12/19/2011
How are problems escalated? - The PS group members are the primary resource for resolving issues, but
volume of issues, technical or system knowledge required, or complexity or impact of the issue may require
involvement from additional CC members. Other members of the CC (or across CC’s) are available for
consulting to help resolve issue. To provide general guidelines, other CC staff may freely provide up to 1-2 hrs
consulting and other support. If more support is required for an issue, the Production Support ASM will facilitate
further involvement of CC staff through the CC ASM.
How are problems resolved? - Once a problem or issue is resolved, the Task will be completed with an
appropriate completion date. Additionally, the client will be notified to confirm the issue is completed.
How is problem status communicated? - Logging and completing Tasks in the Task system will help capture,
track, and report status of issue. The Task “solution” field will be used to note action, approaches, and
solutions. This will also become part of the record for future change control and future problem solving. Again,
the client is to be kept informed as to the status and resolution of their issues.
How are projects transitioned to PS? - Transitioning projects into the CC applications portfolio is an issue for
the entire CC. Both the CC ASM and the PS ASM will work with the Project Manager to help assure only stable,
well-documented, and maintainable applications are folded into the CC’s supported application suite. Issues
including SLA, outstanding RFSs, client expectations, tool & skill requirements, and vendor contact are to be
resolved as appropriate. The CC ASM will maintain the depth-chart and arrange necessary training, and the PS
ASM will maintain turnover template document and will validate appropriate updating/storage of documentation.
Generally the project team will implement, stabilize, and support the project product though its initial production
cycles (i.e. general guidelines for “shakedown” are 2-6 weeks). The basic template form (draft) to guide
MI_EXCHANGE\ Application Services Production Support\Production Turnover\ Basic Production Turnover Outline.doc
Who is primary contact with application vendors? - The CC ASM or designee will be the Purdue POC for
the vendor (i.e. release announcements, technical issues, plans, schedules, etc.). The PS ASM or CC ASM
designee will be the technical focal point or conduit for vendor support on application problems. (In some
cases, clients have retained the role for contacting vendor support, and PS ASM or designee with work with the
client to pursue vendor support as appropriate.)
How are night calls handled? - Again, each CC will determine its specific approach for staffing off-hour
support. Each CC will maintain a “depth chart” showing application knowledge and experience to support
staffing a rotation or provide on-call staff. If appropriate for the CC (and at the CC ASM discretion), a rotation
chart will document who is on-call, and who are the backups for each application or application area over a
period of time. Otherwise, the depth chart will indicate who is on-call, and who is on back-up. The CC ASMs
will be responsible for maintaining CC depth charts and establishing rotation-chart assignments. The
Production Support manager will monitor to make sure the information is current, work with the identified staff to
provide coverage, maintain in a central repository, publish on-call lists with ITI, and utilize the lists to facilitate
How are PS staff assigned other work? - The primary and overriding assignment for PS staff is “reactive”
production support responsibilities. PS staff may be assigned other duties including proactive production
support assignments, RFS work, Pre-Work, estimating, assessing, etc.; but “reactive” production support issues
will take precedence. The CC ASM will make assignments and balance duties, with the consultation of the PS
ASM. Ultimately, the CC ASM is responsible for the overall allocation of resources and the services provide
each client within his or her CC.
74b92649-6932-4032-9045-fecb52354354.doc Page 4 of 4 12/19/2011