Preparing to read your feedback report . . .
Your feedback report contains Baldrige Examiners’ observations that are based on their
understanding of your organization. They have provided comments on your organization’s
strengths and opportunities for improvement relative to the Education Criteria for
Performance Excellence. The feedback is nonprescriptive. It will tell you where Examiners
think you have strengths to celebrate and where they think improvement opportunities exist.
The feedback will not say specifically how you should address these opportunities. The
specifics will depend on what you decide is most important to your organization.
Applicant organizations read and use feedback comments (both strengths and opportunities
for improvement) in different ways. We’ve gathered some tips and practices from prior
applicants for you to consider:
Take a deep breath and approach your Baldrige feedback with an open mind. You applied
to get the feedback. Read it, take time to digest it, and read it again.
Celebrate your strengths. You have worked hard and should congratulate yourselves.
Use your strength comments to understand what the Examiners observed you do well and
build upon them. Continue to evaluate and improve the things you do well.
You know your organization better than the Examiners know it. There might be relevant
information that was not communicated to them or that they did not fully understand.
Therefore, not all of their comments may be equally accurate.
Although we strive for ―perfection,‖ we do not achieve it in every comment. If Examiners
have misread your application or misunderstood your organization on a particular point,
don’t discount the whole feedback report. Consider the other comments and focus on the
most important ones.
Prioritize your opportunities for improvement. You can’t do everything all at once.
Think about what’s most important for your organization at this time and decide which
things to work on first.
You may decide to address all, some, or none of the opportunities in a particular Item.
It depends on how important you think that Item or comment is to your organization.
Use the feedback as input to your strategic planning process. Focus on the strengths and
opportunities for improvement that have an impact on your strategic goals and objectives.
Malcolm Baldrige National Quality Award—Feedback Report 1
KEY THEMES
The Site Visit Team found the descriptor for scoring band 6 to be the most accurate overall
for Northwest Missouri State University (Northwest). For an explanation of the scoring
bands, please refer to Figure 6, ―2005 Scoring Band Descriptors.‖
An organization in band 6 typically demonstrates refined approaches responsive to the
multiple requirements of the Items. These approaches are characterized by the use of key
measures, good deployment, evidence of innovation, and very good results in most areas.
Organizational integration, learning, and sharing are key management tools. Results address
many customer/stakeholder, market, process, and action plan requirements. The organization
is an industry leader in some areas.
a. The most important strengths or outstanding practices (of potential value to other
organizations) are as follows:
Northwest has deployed ―The Culture of Quality,‖ a vision that was initially
articulated by President Dean Hubbard in 1984. This vision of an educational
institution based on quality principles is embodied in the mission, vision, and
cultural core values of the organization and is lived by the employees of this
rural university. The President’s constancy of purpose in fully deploying this
improvement-focused culture through all levels of the organization demonstrates
visionary leadership. The Culture of Quality permeates the organization through
both faculty and staff departments, including an extensive number of student
employees, and Northwest is internationally recognized for its unique approach
to educational institution excellence.
An effective set of listening and learning strategies are in place across Northwest.
Students, faculty, and staff concerns and compliments are systematically addressed
and responded to through the Culture of Quality Comment Card System that is
administered by the Office of the President. Each comment card is read and responded
to by the President. In addition, listening and learning is accomplished at the strategic
level through the initial steps of the Seven Step Planning Process. Multiple surveys
are used to gather input from multiple stakeholders to support data-driven decision
making, including an alumni survey to assess the impact of a Northwest education.
Northwest has fully embraced the Baldrige Education Criteria for Performance
Excellence as a model for excellence. The Baldrige Criteria are used for multiple
purposes at the organization: a framework for excellence, input into the strategic
planning process and a method to identify specific improvement opportunities.
This use of the Baldrige Criteria as a strategic and tactical tool takes full advantage
of the Baldrige assessment process.
Malcolm Baldrige National Quality Award—Feedback Report 2
The Northwest Quality System Model (NQSM) is an integrated quality model that
systematically encompasses all elements of the university’s planning and continuous
improvement needs. The NQSM improvement model has four phases: (1) strategic
context, (2) strategy identification and refinement, (3) Seven Step Planning Process
(SSPP) development and deployment, and (4) Baldrige self-assessment. This systems
perspective ensures the broad collection, informed analysis, and focused use of
pertinent student, stakeholder, and market information and data. This model provides
the basis for integrated activities throughout the organization.
Systematic deployment of Northwest’s mission, vision, and values is directed
by data-driven approaches including the NQSM, SSPP, broad use of Baldrige
assessments, and four Decision Drivers (enrollment, student success, satisfaction,
and financial) (Figure 2.1-2). Improvement priorities are set through the yearly
Strategic Planning Retreat, bi-monthly Strategic Planning Council (SPC) meetings,
and the annual report process. The four-year long-term planning horizon is addressed
through the Decision Drivers, Strategic Initiatives, and Strategic Key Quality
Indicators (KQIs), which extend beyond four years. Tactical KQIs address planning
horizons of two–five years, and shorter-term planning horizons are addressed through
departmental SSPPs. Participants include faculty, staff, the administration, students,
key partners and suppliers, and the Board of Regents. Refinements have been made
to reduce the number of steps in the process, include additional stakeholders, and
expand the breadth of data used as input.
Key learning-centered processes are Northwest’s (1) key instruction processes
(Figure 6.1-2): curriculum, teaching/advising, and learning environment and
(2) key student services (Figure 6.1-3): enrollment, library, student affairs,
information management, academic assistance, and the Center for Information
Technology in Education (CITE). Every process undergoes an annual SSPP by
department that defines and validates Tactical KQIs, sets goals and objectives,
formulates an assessment strategy, defines action plans and deployment strategies,
sets baselines, tracks trends and comparisons, searches for better practices, and sets
targets and stretch goals. Cost analysis activities are conducted. Tactical KQIs are
used for the identification of educational and student service processes to address
students’ educational, developmental, and well-being needs, as well as to maximize
student success. The SSPP is deployed throughout all departments and units of the
organization. The SSPP is accomplished by identifying requirements, designing
processes to meet those requirements, determining appropriate measures, and
implementing the processes. The SSPP process itself has been through cycles of
improvement since its introduction in 1996.
Northwest has empowered employees through a decentralized decision-making
structure. One example of this empowerment is the deployment of budget
responsibility at the department level; this deployment strongly supports the concept
of academic freedom. To support its data-based decision making, the university uses
Malcolm Baldrige National Quality Award—Feedback Report 3
an integrated series of color-coded Dashboards to review organization-wide
performance. These Dashboards indicate performance to plan and comparisons
to past performance and future goals and provide significant drill-down capability.
This system of Dashboards is fully deployed to all departments and supports decision
making by identifying areas that need additional focus. Key performance measures
for all departments are aligned with the four Decision Drivers and also with the
President’s Dashboard. In addition to the Dashboard, formative data are used by
departments/units to analyze daily operations and to monitor work in progress, and
summative data are used to assess longer-term progress. Senior leader reviews include
the weekly President’s Cabinet meeting using the President’s Dashboard, twice
monthly Dean’s Council meetings using the Provost’s Dashboard, annual department
reviews with the President and Provost, and the semi-monthly Strategic Planning
Council meetings.
Northwest has generated many innovative ideas, processes, and technologies to
improve organizational effectiveness and to create breakthrough improvements.
Breakthrough improvements include Missouri Academy, the Beginning Teacher
Assessment Program, and Career Pathing for student employees. A recent innovation
includes the American Dream Grant Program that provides grants to low-income
students that are typically first-generation college students. This aligns with the
Northwest vision to be an ―internationally recognized innovator in continuously
improving higher education.‖
b. The most significant opportunities, concerns, or vulnerabilities are as follows:
Senior leaders use the NQSM (Figure P-7), Strategic Planning Process, SSPP
(Figure 6.1-1), and systematic Baldrige assessments to create a sustainable
organization. However, the use of innovation to make meaningful change to
improve processes is not fully deployed into daily work. For example, delivering
learning-centered processes is central to the vision, mission, and values of being
a technology leader, yet the faculty has only recently been encouraged to move static
classroom content, such as lectures, to allow more time in class for active learning.
Without systematically driving organizational innovation, the organization may not
achieve its focus on integrating technology or responding to changing organizational
needs and challenges in the operating environment.
The SSPP is an integrated planning and improvement system that helps link activities
in the organization. Although very effective as a planning tool, the SSPP is less
systematic in its approach to process improvement. During the site visit, it was
found that departments within Northwest use a wide variety of strategies to address
process improvement. For example, some departments reported using the Northwest
Innovation Process, another used ―talking about the problem,‖ and another reported
using the SSPP on all improvement opportunities. Without an integrated and
Malcolm Baldrige National Quality Award—Feedback Report 4
systematic approach to improvement within and across departments, the university
may generate confusion when cross-functional groups address opportunities,
which may not gain the greatest benefit from the cross-functional activities.
Individual departments within the university are empowered through decentralized
decision making to determine appropriate training and development for faculty and
staff; however, there is not a method to ensure that Northwest is able to appropriately
align training and development with the Tactical and Strategic KQIs or with the
department’s SSPP.
With much of the future growth anticipated to come from a more diverse student
population, such as students from the Kansas City metropolitan area, nationally,
and internationally, approaches to better understand diversity issues and opportunities
and to integrate them into strategic short- and long-term plans could assist Northwest
in addressing the strategic challenge of declining regional population.
c. Considering Northwest’s key business/organization factors, the most significant
strengths, opportunities, vulnerabilities, and/or gaps (related to data, comparisons,
linkages) found in its response to Results Items are as follows:
Despite the recent state financial crises, Northwest has been able to successfully
address budget losses by using its reserves, implementing a tuition surcharge,
and reorganizing workloads. As a result, no positions have been eliminated and no
programs cut. In addition, Northwest has dedicated more than $175 million toward
physical plant improvements, renovations, maintenance, and construction. As a
result of effective financial management, the organization has maintained steadily
increasing financial reserves and an A3 bond rating (the highest rating available
for this type of organization).
While American College Testing (ACT) Alumni Outcome Survey results were
generally better than comparative cohorts, many of the alumni measures show
flat to declining trends, suggesting that learning-centered processes may need to
be strengthened.
Northwest’s Graduation Rate (Figure 7.1-6) has increased from approximately 43%
in 1999 to approximately 62% in 2005. This is an increase of approximately 44%
and compares favorably to the best competitor’s (of five other moderately selective
regional universities) graduation rate of approximately 55%. Northwest was site
visited by the American Association of State Colleges and Universities (AASCU)
to study institutions whose graduation rates are significantly higher than expected.
Northwest has demonstrated consistent improvement in the area of student learning
(general education) over the last ten years, outperforming its best state competitor.
Northwest uses the Educational Testing Service (ETS) Academic Profile as an
Malcolm Baldrige National Quality Award—Feedback Report 5
institution-wide measure of student general education performance, and results
show a positive trend from the 1996–1997 school years to the 2004–2005 school
year (Figure 7.1-1). The portion of students that exceed the national average
increased from the 40th percentile in 1996–1997 to the 70th percentile in 2004–2005.
Also, levels from the past two school years exceeded the levels of the best state
competitor for moderately selective schools in Missouri. Further, nationally normed
major field examinations (Figure 7.1-4) also are used to determine student learning
results. From the 1996–1997 school year through the 2004–2005 school year,
Northwest’s results show a positive trend from the 42nd percentile to the 67th
percentile, respectively. Additionally, the 2004–2005 results exceed all Missouri
competitors. Student success is a key Decision Driver and instructional effectiveness
is a key student and parent requirement.
Several measures of faculty and staff satisfaction and performance, cited as important
to Northwest, are not evident, including measures of work system performance and
effectiveness and results for faculty and staff learning. Further, worker compensation
claims trend negatively and are not as favorable as the eight competitors shown.
Lack of metrics and performance in key faculty and staff results suggests that
Northwest’s approach to achieve its core value of ―caring about each other‖
and ―we are a learning organization‖ may not be systematically deployed.
Malcolm Baldrige National Quality Award—Feedback Report 6
DETAILS OF STRENGTHS AND OPPORTUNITIES FOR IMPROVEMENT
Category 1 Leadership
1.1 Senior Leadership
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
President Dean Hubbard’s visionary leadership is recognized by faculty, staff, board
members, and other executives at Northwest as an important driver of the Culture of
Quality, which has evolved over the past 18 years and has been through multiple cycles
of improvement. Senior executives are versed in the Culture and work to deploy the tenets
of the approach throughout the organization by gaining the commitment of faculty and staff
to ensure that quality is the basis of planning and decisions.
The Culture of Quality is deployed through three key leadership teams: the President’s
Cabinet, the Dean’s Council, and the Strategic Planning Council (SPC). Two-way
communications and agility are ensured through a Leadership Forum, which brings
executive team members together with other important management groups, such as
the Academic Chairs’ Council, Student Administrative Services (SAS), Faculty Senate,
Support Staff Council, and Student Senate. The overlapping membership on the various
teams ensures that plans are deployed, knowledge is shared, and feedback and
communication flow in both directions.
The Culture of Quality provides the infrastructure and resources for sustainability
as Northwest faces the impending retirement of approximately 75% of senior
leaders in the next five years. Board members, senior leaders, and faculty leaders
(deans and academic chairs) concur that the Culture of Quality, with related processes
and the NQSM, has become the sustaining strength in the fabric of Northwest and
does not rely on a single figure to maintain the progress made through these approaches.
For example, the search for additional funding sources during the financial challenges
of 2002–2003 led to the discovery of opportunities such as the Plant Biologics Program,
which was quickly incorporated into the university structure using the SSPP. The ability
to formulate strategy and use innovative approaches to adapt to the market environment
promotes sustainability.
Innovation is encouraged through a grant program that was established in 1987.
Faculty members submit grant applications to the Provost and college Deans, who then
allocate funds for the proposed projects. Innovations that emanate from these projects,
as well as those that arise spontaneously from within the organization, are further
developed through the SSPP to move an idea from the concept phase to implementation.
Malcolm Baldrige National Quality Award—Feedback Report 7
Northwest’s Strategic Initiatives (SIs) originate from the NQSM. Senior leaders serve
as KQI and SI champions and play a key role in identifying actions needed to deploy,
communicate, and measure progress toward completing the Sis, and they use the
Dashboard System to track progress. KQIs are determined for student and stakeholder
groups as part of the SSPP. KQI validation by faculty, staff, and key suppliers helps
ensure that value for all stakeholders is balanced.
Senior leaders have deployed comment cards as a unique way of facilitating
two-way communication throughout campus. This relatively inexpensive method of
communication has become a key method of connecting all stakeholders on campus with
the President and senior leaders. The approach has become an opportunity for the senior
staff to better understand campus issues in a timely fashion and to make rapid changes
to accommodate the needs of all stakeholder groups.
Senior leaders communicate with faculty and staff through a variety of communication
methods, including presidential addresses, SPC retreats, leadership forums (Figure 1.1-1),
and Dashboards and annual reports (Figure 4.2-1). The overlapping membership
on leadership and administrative teams facilitates communication. To encourage
empowerment, job descriptions are written so that each begins with a description of the
support needed to sustain continuous improvement. Senior leaders take an active role in
faculty and staff recognition programs, including participation in award events and in
the development of reward and recognition systems.
OPPORTUNITIES FOR IMPROVEMENT
Efforts at implementing a mentorship program have had varying results for the
organization and its leaders. Consequently, Northwest lacks a systematic succession
planning program for senior leaders, but it has instituted leadership training presentations
for the Leadership Forum to begin developing future leaders. There is no method in place
to measure the effectiveness of this new program, therefore university leaders may find
it difficult to determine if the program has a predictable effect on leadership development.
Although Northwest uses the SSPP to move innovative ideas from concept to
implementation, the organization’s faculty, staff, and key partners are not consistently
informed about opportunities for developing and implementing new ideas. The innovation
grant program, designed to promote the development of innovations by faculty, staff,
students, and partners, has not been systematically deployed to provide an opportunity for
university leadership to improve the involvement of key stakeholders in helping to realize
Northwest’s core value: ―We are a learning organization, continually improving our
university and ourselves.‖
Malcolm Baldrige National Quality Award—Feedback Report 8
Although Northwest’s leadership teams value the relationships with stakeholder groups,
such as feeder high schools, receiving colleges and universities, community colleges,
employers, and the community, the university has not developed a systematic approach
to communicating its vision, values, and relevant data to these important stakeholders.
Balancing value for these organizations may assist Northwest in attaining its core value
of being a leader in its field.
Malcolm Baldrige National Quality Award—Feedback Report 9
1.2 Governance and Social Responsibilities
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest’s President is responsible to the Board of Regents for the actions of senior
leaders and for overall operations. The board meets nine times a year to establish
and review policies, receive reports, and approve budgets, plans, and new positions.
In addition, it meets twice per year with an independent auditor to review audit results
(to ensure fiscal accountability) and holds an annual retreat with the President’s Cabinet
to review strategic plans and performances (to protect stakeholder interests). To further
protect stakeholder interests, the board also may conduct Dashboard reviews.
A 360-degree review process is used for the President and senior leaders. The President
performs the evaluations for senior leaders and makes recommendations to the board
regarding opportunities for improvement. With board concurrence, the President works
with senior staff members to set goals for the next year. The board works with the
President to develop his goals for the upcoming year, incorporating information from
the 360-degree feedback, faculty and staff surveys, Dashboard data, and board members’
knowledge of the community. The board uses an instrument to perform a self review
once each year to identify opportunities for improvement of board functions.
Adverse impacts of Northwest’s programs, offerings, services, and operations are
proactively addressed through the combination of focused studies (i.e., economic
impact, needs for teachers, and in-service and graduate training opportunities),
committees with regional representation to assist in planning and policy reviews,
compliance with regulatory and accreditation standards, integration of community
leaders into the SPC’s Environmental Scan, and monthly President’s Cabinet meetings
with community leaders (―Eggs and Issues‖). Measures and targets have been established
for each social responsibility process (Figure 1.2-1).
Northwest’s outreach programs encompass a number of venues, one of which is the
Eggs and Issues Program. This bi-monthly meeting draws community leaders together
with senior administrative and academic leaders at the university to deal with issues
and to find ways of reaching solutions with a minimum of bureaucratic interference.
The commitment of the President to these meetings, as well as the attendance of nearly
all senior university staff members, provides a positive message regarding Northwest’s
commitment to the community, as well as providing a key listening and learning post
for this important stakeholder.
Malcolm Baldrige National Quality Award—Feedback Report 10
Northwest supports the local community through membership in a wide range of
organizations (e.g., Nodaway County Economic Development Council, North Missouri
Education Consortium, Kansas City Regional Access Consortium for Higher Education
(KC REACH), and Excellence in Missouri Foundation). Northwest operates the
McKemy Center for Lifelong Learners and provides consulting services for small
businesses through the Small Business Development Center. Senior leaders and faculty
serve the education community on several U.S. Department of Education committees,
participate in three national consortia, and host visits and share information with other
educational institutions.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest has a Compliance Program to detect and manage the most serious
ethical issues, such as violations of National Collegiate Athletics Association (NCAA)
rules, Americans with Disabilities Act (ADA) infractions, academic dishonesty, and
sexual harassment, these approaches are not consistently deployed throughout the system.
The process for dealing with ethical lapses also is not consistently deployed. Since no
incentive or requirement exists to report other types of ethical issues, the magnitude
of these issues is not measured.
Although agriculture, business, and education were targeted as key communities for
support by Northwest during the annual planning retreat, the university has not created
programs to address the needs of the agricultural community. Since Northwest is located
in a rural area, commitment to improving the condition of the agricultural industry may
provide an opportunity to deal with a key requirement of its community.
Malcolm Baldrige National Quality Award—Feedback Report 11
Category 2 Strategic Planning
2.1 Strategy Development
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest uses the four-stage NQSM (Figure P-7) to conduct annual strategic
planning and to align short-term annual action plans (Tactical KQIs) with long-term
university-wide objectives (Strategic KQIs). The Strategic Planning Council (SPC)
is responsible for implementing and making changes to the NQSM with the assistance
of participants representing the entire university, key partners, students, and the
community. A refined process with fewer steps results in cascading strategic components
(Decision Drivers, Strategic KQIs, SIs, Tactical KQIs, and departmental SSPPs) has been
deployed to all work units throughout the university. Phase four of the NQSM includes a
systematic evaluation and improvement cycle based on a systematic Baldrige assessment.
Strategic planning input is collected and analyzed from students (via the Student
Senate), faculty, staff, administrators, members of the Board of Regents, alumni,
corporate partners, key suppliers, and stakeholders from the community (called the
―Voices of Access‖). Northwest’s strategic planning process identifies potential external
blind spots through its annual Environmental Scan Process and SPC Retreat. Northwest
has a longer-term planning horizon of four years (to coincide with the Coordinating
Board of Higher Education [CBHE] review cycle), a medium term (two years), and a
shorter-range (one year) time horizon. The medium-term planning horizon is addressed
through the four plans associated with each SI, and the short-term horizon is addressed
through each department’s Tactical KQIs.
Northwest uses the SPC retreat to review and assess Environmental Scan information
and to develop SIs. Trend information is collected and analyzed to support decision
making during the retreat. Environmental Scans are completed annually to review
demographic, economic, technological, political/legal, social, and competitive data, and
Baldrige assessments are used as the internal analysis. Northwest addresses longer-term
organizational sustainability through the longer-term threats and opportunities identified
by the Environmental Scan. The university’s ability to execute its strategic plan is
addressed in the SPC’s review of performance toward accomplishment of the SIs.
The SPC Retreat Process is reviewed via annual surveys of SPC participants. As a
result of the last survey, the structure of the SPC Retreat was changed to focus on the
six strategic planning teams.
Malcolm Baldrige National Quality Award—Feedback Report 12
Northwest’s Strategic Initiatives (SIs) (Figure 2.1-2) include key measures and
2006 targets for each SI. The SIs are aligned with Northwest’s four Decision Drivers.
Based on a review of organizational performance results, the SIs (Figure 2.1-1) and
corresponding Tactical KQIs have been institutionalized into appropriate departments
and processes. New SIs were developed during the last SPC Retreat, and the four
interlocking plans are being developed by the SI Champions.
The enrollment and financial viability strategic challenges are addressed through
university-wide Decision Drivers and corresponding Strategic KQIs (Figure 2.1-1)
and SIs (Figure 2.1-2). During the annual Environmental Scan, shorter- and longer-term
challenges and opportunities are considered when developing SIs. In keeping with its
value of ―focus on students and stakeholders,‖ Strategic KQIs are validated, SIs are
developed during the SPC Retreat, and the differing needs of students, faculty, staff,
administration, and community representatives are prioritized and balanced to provide
a maximum value to each of these groups. Short- and longer-term challenges and
opportunities also are balanced during the SPC Retreat.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest uses feedback from performance excellence assessments as
its internal analysis for strategic planning, no additional information or analyses
were included in the internal analysis during the last three strategic planning cycles.
The performance excellence assessments do not address potential internal blind spots.
Failure to identify potential blind spots may limit the effectiveness of the university’s
strategy and potentially hinder its ability to address the needs of Northwest students.
Although Northwest uses Strategic KQIs and has a Campus Safety Crisis Management
Plan, the university does not collect and analyze data and information for strategic
planning to assess organizational continuity in emergencies.
Northwest uses its annual Environmental Scan to address opportunities and threats.
Over the last three strategic planning cycles, the development and use of trends and
―Megatrends‖ by the six strategic planning teams have been inconsistently deployed.
Lack of a consistent Environmental Scan across all strategic planning teams may result
in challenges and threats that are not addressed, thereby limiting the effectiveness of
strategic planning.
Malcolm Baldrige National Quality Award—Feedback Report 13
2.2 Strategy Deployment
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest allocates resources to Strategic Initiatives and departments through a series
of budget-focused President’s Cabinet meetings. A cash reserve is held for contingencies.
Through the annual SSPP (Figure 6.1-1), each department develops and deploys Tactical
KQIs (action plans). During Phase 3 of the NQSM, Tactical KQIs are linked to Strategic
KQIs. Each department has a budget and allocates resources as part of its SSPP to ensure
accomplishment of its Tactical KQIs. During the annual report process, each department
reviews its progress in achieving its Tactical KQIs, including key Tactical KQI changes,
with the Provost and President to ensure that changes are sustained.
Northwest uses its annual report process to review departmental action plans within
each department’s SSPP. Each department has the authority to modify its Tactical
KQIs or create new Tactical KQIs within the framework of the department’s SSPP
if circumstances require a shift and rapid execution of new plans. Tactical KQIs, the
SSPP, and the annual report process have been deployed to all departments.
The Tactical KQIs in each department’s SSPP are considered the short-term action
plan. The four interlocking plans (deployment, measurement, communication, human
resources) for each SI are considered the longer-term action plans. Key changes have
resulted in modifications that include (1) reductions in state funding that led to the
development of activity-based costing in the SSPP, (2) the need for innovative teacher
education programs that has led to collaboration with area high schools to improve
teacher preparation, and (3) the need to serve non-traditional, place-bound students
that has led to distance education needs and subsequent applications of technology
to higher education.
Four interlocking plans (human resources, communication, deployment, and
measurement) are developed by the SI Champion for each Strategic Initiative.
During the annual SSPP, additional human resource plans are derived by each
department to support achievement of its Tactical KQIs.
Additional performance measures for tracking action plan progress are contained in
departmental SSPPs. Northwest’s overall measurement system is structured around
its four Decision Drivers, including all online Dashboards, Strategic KQIs, and SI
measurements. This structure reinforces organizational alignment and has been
deployed to all departments and key on-campus suppliers.
Malcolm Baldrige National Quality Award—Feedback Report 14
OPPORTUNITIES FOR IMPROVEMENT
Although Tactical KQIs (action plans) may be modified and deployed at the department
level, there is no systematic approach to establish new cross-department SIs outside of
the SPC Retreat, particularly if circumstances require a shift in plans or rapid execution
of new plans. Failure to modify and deploy cross-department SIs may hinder Northwest’s
responsiveness to changing conditions, thereby limiting overall performance.
Although the SSPP and annual review have been deployed to all departments,
alignment of departmental Tactical KQIs to Strategic KQIs shows a deployment
gap in approximately one-third of the departments. This may result in misallocation
of departmental resources to activities that are not strategically aligned with overall
university direction.
Although past performance and short- and longer-term performance projections are
included in the measurement plan for each SI, Northwest does not compare its projected
performance with competitor performance projections, key benchmarks, or goals. Lack of
comparisons to competitors’ projected performance or benchmarks may limit Northwest’s
ability to identify projected performance gaps. This deficiency may hinder Northwest’s
ability to develop effective strategies to meet competitive threats while responding to its
strategic challenges of enrollment and financial viability.
Malcolm Baldrige National Quality Award—Feedback Report 15
Category 3 Student, Stakeholder, and Market Focus
3.1 Student, Stakeholder, and Market Knowledge
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest identifies student and market segments during the Environmental Scan
conducted as part of its NQSM process, using data gathered from census reports, analysis
from the ACT Enrollment Information Service and College Board survey, focus groups,
and other national surveys of prospective college students. After a segment is identified,
the special accommodations to meet its needs are specified (Figure 3.1-1), plans are
developed to meet those needs, and assessment strategies are designed to measure
satisfaction and effectiveness.
To determine student and stakeholder requirements, Northwest uses a broad array of
information sources, including national surveys and data sources (Figure 3.1-3), locally
developed surveys (Figure 3.1-4), department surveys, comment cards, focus groups,
advisory groups, senior capstone courses, student meetings, and student representation
on university boards and committees, as well as monitoring of facility utilization, course
and program selection patterns, grades, satisfaction, and student movement measures
(i.e., persistence, voluntary departures, and transfers). President’s Cabinet members are
responsible for analyzing and synthesizing information from these surveys into coherent
patterns, and they ultimately report these patterns to the SPC where they are integrated,
aligned, and prioritized into the annually updated SSPPs and KQIs.
The information gathered through Northwest’s listening and learning approaches is
systematically segmented and used throughout the institution, including at the individual
department level, to plan and modify programs and offerings as part of the strategic
planning process.
Northwest keeps listening and learning methods current through a systematic review
process coordinated by the university’s Office of Assessment, Information, and
Analysis. This office ensures that listening and learning methods align with KQIs,
are relevant to segments, produce timely and actionable data, and generally address
the organization’s needs. This office also benchmarks programs and processes within
and outside higher education.
OPPORTUNITIES FOR IMPROVEMENT
Northwest utilizes satisfaction survey processes such as the Noel-Levitz Student
Satisfaction Survey to listen and learn about changes in student and stakeholder
expectations; however, the university’s approach to using this data is not fully deployed.
Malcolm Baldrige National Quality Award—Feedback Report 16
For example, as the students mature in the organization, the listening and learning
approaches are not translating the data into actionable feedback, as indicated by gaps
in performance in the Juniors’ Service Excellence Category on the Noel-Levitz survey
(―I seldom get the run around when seeking information on this campus‖). As a result, as
students progress through the university, an increase in student dissatisfaction is apparent.
While Northwest has identified a number of activities, such as the American Dream
Program, Missouri Academy, and alumni relationships, to target feeder schools, there
is no overall approach to systematically analyze demographic and population trends to
identify emerging markets of opportunity and key requirements to more effectively target
desired student segments.
Malcolm Baldrige National Quality Award—Feedback Report 17
3.2 Student and Stakeholder Relationships and Satisfaction
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Various methods are used to build relationships with students and stakeholders including
potential students, newly accepted students, new students, current students, and alumni.
Relationships with potential students include multiple methods to build awareness of
the organization and its capabilities in a manner that focuses on the potential student.
Current students sign a Covenant for Learning in which student and university
responsibilities are articulated and each party ―pledges‖ to do what is necessary to ensure
student success. Freshman Seminar is a far-reaching orientation approach that prepares
new freshman students for the college experience. Individualized counseling ensures
that new student’s schedules are appropriately aligned and that ACT/academic history
contributes to appropriate schedules and future success. Learning for all students is
enhanced through services such as the Writing Lab and the Talent Development Center.
These and other approaches are intended to attract, satisfy, and provide positive referrals
while being aligned with multiple student stakeholder requirements (Figure P-4).
Service KQIs, which were developed in 1991 and are reviewed annually, set the contact
standards and relationship requirements for the organization. A wide variety of measures
and indicators help ensure progress at both the organization and department level.
Most common methods of access include nationally normed and locally developed
surveys, focus groups, e-mail, Web pages, telephone conversations, and the complaint
management system. Identified requirements include student and stakeholder access
points: Internet, phone, mail, in-person/word-of-mouth, and an agent for international
students. Approaches such as the Comment Card System, nationally normed and
locally developed surveys, advisory committees, and focus groups assist in this
information gathering.
Northwest manages student and stakeholder complaints primarily through its Comment
Card Process, a systematic approach that involves the President, President’s Cabinet
members, department heads, and stakeholder groups. The process includes specific
response requirements. For example, the Comment Card Process features a closed-loop
reporting back to the President and is fully deployed throughout the university.
The President reviews every comment card. The President’s staff aggregates and
analyzes complaint data annually and reports findings to the Cabinet for potential
additional corrective actions.
Northwest keeps its approaches current by using the annual SSPP review process to
identify and recommend improvements. The annual review uses a variety of inputs
including student and stakeholder satisfaction, Baldrige feedback, student and stakeholder
Malcolm Baldrige National Quality Award—Feedback Report 18
focus groups, and comments cards. Improvement actions also are initiated based on
the Cabinet member review of student and stakeholder satisfaction performance on
the Dashboard.
Northwest determines student and stakeholder satisfaction and dissatisfaction through the
Noel-Levitz Student Satisfaction Inventory and methods listed in Figures 3.1-3 and 3.1-4.
Student and stakeholder satisfaction data are analyzed in aggregate and by strategic and
department/unit segment to provide actionable information. Dissatisfaction is determined
through a variety of indicators including student attrition, comment card complaints,
student exit interviews, and graduate school admission results, as well as through focus
and advisory groups.
Northwest uses student and stakeholder satisfaction and dissatisfaction information
to measure performance (Dashboard), identify improvement needs, refine existing
improvement activities, and identify inputs to strategic planning. This information
also is reviewed at the monthly SPC meetings and annual strategic planning retreat.
A recent improvement from the analysis of satisfaction/dissatisfaction data was the
Residential Facilities Master Plan to provide different housing options for identified
student segments.
Satisfaction data are compared with national norms and a group of 41 comparable
education institutions that have been identified by the organization. The information
is used to understand preferences, identify key satisfiers relative to performance, set
targets, identify opportunities for improvement, and identify benchmarking opportunities.
Northwest keeps approaches to building relationships and providing student and
stakeholder access current with educational service needs and directions through the
strategic planning process. The Annual Review Process ensures that Northwest is asking
the right questions to determine satisfaction, provide feedback, and respond to student
and stakeholder needs and expectations. A recent improvement was the replacement of
the Student Development Task and Lifestyle Assessment (SDTLA) with the National
Survey of Student Engagement (NSSE) to secure more actionable and comparative data.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest utilizes the NSSE to measure the students’ analysis, synthesis,
evaluation, and application abilities, no systematic approach to use this feedback to
improve student satisfaction and learning throughout all university program offerings
is utilized. This may adversely impact Northwest’s effort to shift its focus to higher
order cognition.
Malcolm Baldrige National Quality Award—Feedback Report 19
Category 4 Measurement, Analysis, and Knowledge Management
4.1 Measurement, Analysis, and Review of Organizational Performance
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
The SSPP serves as the template for determining requirements, selecting summative and
formative performance measures, aligning Tactical KQIs with Strategic KQIs, identifying
best practices, and setting performance goals within departments and appropriate level
units. A matrix is used to show alignment of the data/information system with related
process requirements. An online Dashboard is utilized across the university to track
progress on Strategic and Tactical KQIs. These Dashboards are deployed at the
department level and provide extensive drill-down capability. Dashboards are shared
across departments to provide organizational knowledge and opportunities for learning.
This system supports fact-based decision making across the university.
To support its fact-based decision making, Northwest utilizes multiple performance
measures including both summative and formative measures. Within the strategic plan,
key institutional measures (Figure 2.1-2) are aligned with Decision Drivers and SIs
and are incorporated into a Dashboard System deployed from the departmental through
the institutional level. The Dashboard indicators are frequently tied to aggregate
measurements and can be drilled down to individual department formative levels.
Northwest selects and ensures the use of comparative data as part of its SSPP for
academic and service departments, including a key supplier. The information is
incorporated into Dashboard measurements and used during the strategic planning
process to set performance targets. Criteria for the selection of comparative data are
(1) the source is a member of a comparative group, (2) the source is ―best in class,‖
and (3) the source has comparable measures/indicators. In order to ensure an appropriate
group of comparative organizations, an independent consultant identified 41 peer
institutions. Breadth of comparative data has been consistently improved.
The SSPP is reviewed as part of the Annual Report Process, and the Environmental Scan
looks for internal and external change drivers. Dashboard measures are reviewed during
the Annual Report Process and on a rotating basis at senior-level and SPC meetings.
Changes to the measurement system can be made at any time as the need arises, and
Dashboards are modified on an as-needed basis.
Progress against Strategic Initiatives and departmental Tactical KQIs is systematically
reviewed at all levels of the organization using the Dashboard System, accompanying
drill-down information, and organizational performance reviews. Each department has
an electronic Dashboard that is updated during the year as data become available and
Malcolm Baldrige National Quality Award—Feedback Report 20
are reviewed and discussed at the department’s annual two-day, fall planning retreat.
For example, the Residential Facilities Master (dorm replacement) Plan approval by the
Board of Regents was based on multiple Environmental Scan inputs, including that of
students, and monitoring of its deployment is data-based. The Annual Report and Annual
Review processes provide accountability to the approach. The Dashboards, SSPP, and
annual review approaches have been fully deployed to all departments.
Data-supported priorities and opportunities arising from Northwest’s Culture of Quality
processes are shared, discussed, and acted upon with feeder schools, suppliers, and
partners. Examples are the data-driven mutual reviews of the contractual relationships
among Northwest and the two major suppliers, ARAMARK and Barnes and Noble, as
well as the data-supported and monitored collaborative relationships among Northwest’s
Professional Education Unit and metropolitan public school districts.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest reviews organization performance, information is not systematically
used to translate review findings into opportunities for breakthrough improvement to
support innovation. This may hamper Northwest’s progress toward its vision of being a
―recognized innovator in continuously improving higher education.‖
Malcolm Baldrige National Quality Award—Feedback Report 21
4.2 Information and Knowledge Management
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest makes needed data and information available through its Electronic Campus,
which includes student, faculty, support staff, and administrator access to the intranet
and Internet. Additionally, information is made available through other means including
standard reports, newsletters, local newspaper articles, reports, publications, and unique
data requests. Northwest freely distributes data upon request to any citizen as required
under the Missouri Sunshine Law. Northwest makes needed data and information
available to stakeholders, including partners, suppliers, and K–12 public school districts.
To ensure the availability of hardware and software systems in the event of emergencies,
Northwest has a comprehensive approach that includes outsourcing critical Web-based
systems to companies that maintain complete emergency fail-over computing systems,
use of uninterruptible power supplies, use of a ring configuration for high-speed Internet,
and use of multiple switches. All other major systems including human resources,
payrolls, financial systems, and student records have backup hardware and backup data
in highly secure locations.
A Computer Users Group, which includes professional and administrative staff from
offices that rely on computer usage for daily operations, meets regularly to provide user
input. Emerging needs are reviewed with programming staff, who prioritize improvement
proposals, explore whether software should be purchased, and suggest improvements
They also conduct long-range planning for data architecture needs and identify the
changing needs of faculty, students, and stakeholders.
Organizational knowledge is ―pushed‖ (collected and stored electronically) into easily
accessible systems by employees and transferred by user-friendly ways to ―pull‖
knowledge out.
Northwest uses a combination of data dictionaries, data validation tables, and data
custodians assigned per data element to ensure the integrity and reliability of data and
information. The university ensures timeliness of data and information by operating
all data systems online and in real-time with user control over input and display of
output information.
Northwest ensures security and confidentiality of data and information through a
variety of means including anti-virus software on each PC with automatic log-in updates,
outer and inner firewalls, regular review of firewall reports, and proactive policies on
computing security and user activities. A recent example is that Northwest found, within
Malcolm Baldrige National Quality Award—Feedback Report 22
hours, a young computer hacker who had accessed the university’s computer system
to add one or two pennies to two separate accounts. Security and confidentially also
are ensured through control of user names and passwords, established timeframes to
change passwords, and additional passwords to access financial and enrollment data.
Data custodians ensure appropriate access to confidential, private information and
respect for individual preferences regarding display of personal information.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest has conceptualized a Knowledge Management Process (KMP)
(Figure 4.2-1), it is not fully deployed throughout the university. Without fully deploying
organizational knowledge management, the university may limit its progress on achieving
Tactical KQIs and Strategic Initiatives.
Malcolm Baldrige National Quality Award—Feedback Report 23
Category 5 Faculty and Staff Focus
5.1 Work Systems
Your score in this Criteria Item for the site visit stage is in the 50–65 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest organizes work around discipline-based academic departments, administrative
and academic support functions, student affairs, information technology, communications
and marketing, and international advancement. Jobs are designed to encourage broad
knowledge as opposed to specific functional expertise. A decentralized decision-making
structure and focus on shared governance promote empowerment and foster agility
and innovation.
Northwest utilizes student workers as both an approach to enhance student learning and
as a way to balance work-staffing capacity. Students obtain work skills while attending
class and earning wages. Northwest has recently made Career Pathing available to the
student worker segment. This approach supports the core value of ―We focus on our
students and stakeholders.‖
Communication across departments, jobs, and locations is achieved through various
communication channels including department meetings, college meetings, Leadership
Forum, Faculty Showcases, Celebration of Quality presentations, and all-inclusive
general faculty and staff meetings. The SSPP provides a mechanism to discuss, share,
and formalize steps to complete action plans within a department. The effectiveness
of communication and skill sharing is determined using employee satisfaction surveys,
comment cards, and informal feedback.
A market-based compensation system that targets 90–100% of appropriate market
medians is based on national salary data. A variety of mechanisms are used to recognize
high-performance work and student and stakeholder focus (Figure 5.1-1). Individual and
team recognitions are distributed campus wide and to the community via the Web site.
Positive comment cards and department celebrations are used to recognize personal and
team achievements.
Career Pathing and the SSPP are available to all staff and faculty to aid in career
progression and individual development. Northwest has recently recognized the need
to expand this approach into the student worker segment. The approach has been piloted
in a select number of departments and is currently being expanded into other student
worker areas.
Malcolm Baldrige National Quality Award—Feedback Report 24
Characteristics and skills needed by potential faculty and staff are identified by
professionals in each department. Job postings contain key performance requirements,
and a work design analysis occurs when adjustments are considered in work design
or curriculum assignments. In addition, the interviewing process focuses on service
interests and capabilities, core values, technology use, and department/unit KQIs.
OPPORTUNITIES FOR IMPROVEMENT
Northwest’s team approach to organizing and managing work includes cross-functional
and ad hoc teams to meet objectives, address opportunities, and solve problems.
The approach does not ensure that skill levels and experiences of team members are
distributed within these teams. As a result, less-experienced faculty and staff members
may have limited opportunities to develop team skills.
Northwest’s lack of racial diversity represents its traditional student communities;
however, much of its future growth is anticipated to come from a more diverse student
population, including the Kansas City metropolitan area (traditional students) and
nationally and internationally (Plant Biologics initiative). Although the university
provides activities that address diversity, such as training (during orientation and on
request to others), campus-wide diversity events, and the public display of the flags
of countries that have been represented by students, a culture of diversity is not evident
at all levels within Northwest. This could have an adverse impact on the university’s
success in accomplishing its mission of ―expanding access to learning and promote
research designed to address the needs of our students and stakeholders.‖ A lack of
sensitivity to diversity has the potential to limit interest from student groups that are
seen as key to sustaining the population of the university.
While several of Northwest’s communication processes (e.g., general faculty meetings)
permit cross-department communication, these same processes do not promote skill
sharing across departments and work groups. Department meetings are typically
―information sharing‖ in nature and seldom address skill sharing. Although the
―in-service‖ meeting in the Health Center promotes skill sharing, this skill sharing
is limited to the medical staff within that department.
Northwest has not systematically deployed Career Pathing in all work units. While the
approach is available to a wide range of employee groups, several employees have not
used it and many are not aware of the details of the program. While Northwest has
expanded this approach into the student worker groups, the expansion into the student
work segment is in the early stages of deployment.
Malcolm Baldrige National Quality Award—Feedback Report 25
5.2 Faculty and Staff Learning and Motivation
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
As part of the NQSM, SI Champions develop human resource plans, along with
deployment, measurement, and communication plans, for each SI to provide a direct
link between faculty and staff training and education and Northwest’s action plans.
Large segment training usually focuses on implementing or pilot testing a new tool,
technique, or program; improving current performance; understanding better existing
or new processes; and sharing information.
Needs associated with new employees are addressed through an orientation program
that includes its Culture of Quality, core values, planning systems, ethical beliefs,
diversity, support systems, safety systems, and staff handbooks. Ongoing training
such as diversity, safety, ethical treatment of research subjects, and ergonomics is
delivered on an as-required or as-needed basis.
Northwest seeks and uses input from faculty and staff and supervisors on education,
training, and development needs through a variety of surveys and channels as shown
in Figure 5.2-1. Design and delivery are dictated by discipline-specific needs of the
individual departments/units. Organizational knowledge assets are frequently
incorporated into organization-wide education and training.
Multiple surveys help determine appropriate training methods for various employee
groups. Training delivery methods include classroom, interactive TV, asynchronous
online, one-on-one mentoring, videoconferencing, on-the-job training, and
trained facilitators.
Northwest reinforces the use of new knowledge and skills on the job through a
combination of approaches including evaluations, mentors or preceptors, and supervisors
who have responsibility to observe and mentor reinforcement. An internal ―point of
contact‖ is provided for direct internal access to support the newly acquired skill
or knowledge.
Faculty and staff are motivated to develop and utilize their full potential through learning
opportunities, full-fee waivers for on-campus courses, and 96 hours of paid release time
per year for staff for degree- and non-degree-seeking coursework. Faculty funding for
research, publications, presentations, and conference attendance is available through
operations accounts. Sabbatical and educational leaves are supported and encouraged.
Assistance in applying for grants is available on campus.
Malcolm Baldrige National Quality Award—Feedback Report 26
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest indicates that it evaluates education and training, the evaluation
primarily consists of course post-evaluations. Effectiveness of education and training,
in relation to individual and organizational performance, is not systematically evaluated.
The Site Visit Team found no specific examples where training was conducted to
address a specific organizational performance need. While some training could be
linked to individual performance or job requirements, typically training courses
self-selected by individual employees are not necessarily linked to individual or
organizational performance.
Malcolm Baldrige National Quality Award—Feedback Report 27
5.3 Faculty and Staff Well-Being and Satisfaction
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
The Health and Safety Manager addresses workplace health, safety, and ergonomics.
Multiple methods are used to identify workplace barriers and hazards, including gathering
feedback from faculty, staff, students, and other stakeholders. Faculty and staff welfare
committees identify safety and well-being needs of constituents. Teams research
problems and recommend solutions.
Northwest ensures workplace preparedness for disasters or emergencies through two
formalized plans for Crisis Management and Workplace Violence. These plans are the
subject of department-level training sessions and are available as written documents.
Various training exercises (e.g., fire drills and tornado drills) are conducted throughout
the year.
Northwest uses a collaborative process and its Faculty Senate Survey to determine the key
factors that affect faculty well-being, satisfaction, and motivation. The faculty factors are
reviewed on an annual basis. For support staff, professional staff, and administrators, the
university uses the Noel-Levitz Campus Survey to determine key factors that affect staff
well-being, satisfaction, and motivation. The Noel-Levitz survey was selected because
results could be segmented and compared to nationally normed data. A gap analysis of the
survey results helps to identify key factors that drive staff satisfaction. Key satisfiers for
staff and faculty are summarized in Figure 5.3-1.
Faculty and staff are supported through an assortment of services, benefits, and policies
(Figure 5.3-2) that include comprehensive health benefits, flexible leave policies, and
education and training assistance programs. Benefits and policies are tailored to support
the university’s cultural focus on learning by evaluating the conditions of employment
and considering work demands and time schedules. Faculty and staff welfare committees
review benefits and make recommendations for changes.
Faculty and staff results are reported on the President’s Dashboard and analyzed in
relation to other performance results. When a correlation is established to suggest that
faculty and staff perceptions are adversely or positively impacting overall performance,
actions are identified and taken to make improvements or to further capitalize on a
positive impact. Recent examples of change include technology support, change in
health insurance, workshops for student advising, and expanded parking.
Malcolm Baldrige National Quality Award—Feedback Report 28
OPPORTUNITIES FOR IMPROVEMENT
While workplace factors may vary to address different needs associated with different
worker segments, key safety and security measures are not segmented by worker group.
Improvement targets are not set for the key measures used (worker compensation injuries,
number of security incidents, and junior survey scores). Also, the key safety and security
measures are assimilated annually, which may prevent agile adjustments in processes that
support the faculty and staff work environment.
During the site visit, no evidence was provided to support Faculty Senate and Noel-Levitz
surveys are correlated with productivity, retention, absenteeism, and safety measures.
Northwest may be missing opportunities to identify trends that signal an issue
or improvement.
Malcolm Baldrige National Quality Award—Feedback Report 29
Category 6 Process Management
6.1 Learning-Centered Processes
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
The SSPP (Figure 6.1-1) is used to determine, design, deliver, evaluate, and improve
two key learning-centered processes: instruction (Figure 6.1-2) and student services
(Figure 6.1-3). The use of the SSPP aligns processes and their measures with the Decision
Drivers and Strategic KQIs for student success, satisfaction (students, faculty, and staff),
financial performance, and enrollment by gathering and incorporating needs, inputs, and
the insight of students and stakeholders in the development of learning processes. This
focus on Decision Drivers ensures a focus of the entire organization on student success.
The SSPP has gone through cycles of improvement since its introduction in 1996.
Northwest applies the SSPP to every program offering. Requirements for key
learning-centered processes are determined in the first two steps of the SSPP to ensure
linkage among the KQIs, course design, achievement measures, and improvement cycles.
The SSPP ensures that student/stakeholder input is obtained in the development of KQIs
and validated with the users of the particular process. Northwest anticipates and prepares
for individual differences in student learning rates and styles through a variety of means
including surveys and tests during Freshman Orientation and analysis of ACT scores.
The information is used in the Educational KQIs for instruction processes (Figure 6.1-2)
and student services (Figure 6.1-3). The Center for Information Technology in Education
(CITE) assists faculty in using technology to engage students in active learning and to
support individual differences in learning rates and styles.
Input is gathered from partners such as ARAMARK, Barnes and Nobles, and U.S. Bank
to establish student learning-centered and service process requirements during Step 1
(define and validate KQI) and Step 2 (establish goals and objectives) of the SSPP.
For example, partners in the Marketing Department provide input through partnership
activities with alumni. Partners in the Education Department provide input through
internship activities in professional development schools. In addition, the university
co-develops and delivers graduate programs as well as delivers undergraduate programs
with partners. By gathering input from partners, Northwest is able to meet student
expectations and requirements and address individual student differences in learning
rates and styles.
The SSPP is used to design learning-centered processes. Based on Dashboard
information, the organization identified an emerging opportunity that initiated the
exploration of an online masters program in GIScience. To address this opportunity,
the Department of Geology and Geography used the SSPP to develop the goals, critical
Malcolm Baldrige National Quality Award—Feedback Report 30
success factors, key quality indicators, measures, and the linkage to the organizational
knowledge from the University KQI. Sequencing and linkage among educational
offerings is addressed through adherence to requirements established by the Faculty
Senate. This ensures curriculum alignment.
CITE plays a key role in incorporating new technology into the learning environment.
The design of innovations using the SSPP process ensures all initiatives meet key
requirements. In addition, approaches used to monitor and improve efficiency include
broad access to online courses, a shift to a trimester schedule, and the use of a ratio
analysis to incorporate cycle time and efficiency factors. Processes are field-tested prior
to full scale implementation. This ensures quality programs, quality services, and agility.
Northwest uses multiple measures and indicators in Steps 5 (set baselines), 6 (search for
better practices), and 7 (set targets and stretch goals) of the SSPP to assess departmental
KQIs. This includes student achievement and satisfaction, including both formative and
summative measures as shown in Figures 6.1-2 and 6.1-3. Based on the Decision Drivers
and the Tactical KQIs, best practices are identified and performance goals set as
improvements are initiated.
Steps 5–7 of the SSPP are used to improve learning-centered processes. Assessment
results and course evaluations are used as a basis for improvement. Comparative data
are used to push the organization to higher levels of performance. Employer surveys
are conducted to determine how well graduates are doing compared to graduates of
other universities.
OPPORTUNITIES FOR IMPROVEMENT
Although each department at Northwest creates shorter-term action plans that may
provide input into a current or future departmental SSPP, departments reported using a
wide range of approaches to accomplish this. For example, one department reported using
the Northwest Innovation Process, another department used ―talking about the problem,‖
and another department reported using the SSPP on all projects. Without an integrated
and systematic approach to improvement within and across departments, the university
may not design and implement key learning-centered processes to maximize student
learning and success.
Although examples of in-process measurement have been identified in tutoring, career
education, auxiliary services, counseling, and international affairs to make programmatic
adjustments, day-to-day measures do not exist for Target KQIs for critical/creative
thinking and problem solving, writing, and math in university classrooms. As reported
by faculty, in-process measures are not consistently used day-to-day in the classroom for
key learning-centered processes. Failure to consistently use formative and summative
assessment may interfere with placing the focus of education on learning and the actual
needs of students.
Malcolm Baldrige National Quality Award—Feedback Report 31
6.2 Support Processes and Operational Planning
Your score in this Criteria Item for the site visit stage is in the 90–100 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest uses the SSPP to determine, design, deliver, evaluate, and improve its
support processes (Figure 6.2-1). Support processes are campus safety, facilities,
financial management, outsource partners, utility alternative fuel technology,
purchasing, communication and marketing, and human resources. The SSPP process
ensures that input is received from and validated by student and stakeholder users
of support processes.
The Strategic Service KQIs have been developed based on the key requirements for
support processes. In addition, process owners implement the SSPP to identify process
specific KQIs for each support and sub process. These are based on student and
stakeholder requirements determined through the SSPP and linked to Service KQIs
with measures.
Faculty, staff, supplier, and partner input is incorporated into determining the key support
process requirements as shown in Figures 6.2-1 during Step 1 (define and validate KQIs)
and Step 2 (set goals and objectives) of the SSPP. This is accomplished through the
Environmental Scans conducted by six strategic planning teams. Internal and external
input is included in the annual scan. These two steps ensure that the SSPP integrates the
needs, expectations, and requirements of all customers, stakeholders, and partners into
strategic planning.
Northwest designs support processes to meet all key requirements using its SSPP.
The university uses Service KQIs (Figure 6.2-1), which are identified and confirmed
during the strategic planning process. Northwest employs specific training for service
personnel to ensure that implementation meets process requirements. For example, when
out-of-control processes are identified, root cause analysis is used to isolate opportunities
for improvement, which are then shared across the organization’s support services
through such entities as the Strategic Planning Council, Student Administrative Services,
and the Dean’s Council.
Northwest incorporates new technology, organizational knowledge, the potential need
for agility, cycle time, productivity, cost control, and other efficiency and effectiveness
factors into the design of these support processes. For example, ARAMARK attempted
to improve cost control by examining alternative fuel through the SSPP, including a
pilot study and a review for cost control. It continues to incorporate efficiency and
effectiveness factors into energy savings across campus.
Malcolm Baldrige National Quality Award—Feedback Report 32
Northwest has identified several performance measures for the control and improvement
of its key support processes (Figure 6.2-1). Both in-process and summary assessment are
used, including comment card feedback, bimonthly reports, focus groups with students,
and faculty and weekly post-program monitoring.
The annual costs for support processes are evaluated within a financial context using
trend data within the organization and comparative data among peers and state and
national statistics. Also, costs associated with performance reviews, audits, and emission
tests are routinely evaluated to ensure that the desired outcomes are accomplished at a
reasonable expense.
Northwest ensures continuity of operations in the event of an emergency through its
Crisis Management Plan, which encompasses all educational, operational, and financial
emergency situations; addresses emergency preparedness; and is reviewed and updated
annually by the President’s Cabinet. Based on the results of a recent review, the
university outsourced its course management system to a third party who has built in
redundancy at all failure points at an off-site location. Additionally, the Health and Safety
Manager audits safety and risk practices, develops disaster and contingency plans, designs
personnel training, and oversees development of safety manuals.
OPPORTUNITIES FOR IMPROVEMENT
Although an effective system is in place to ensure alignment of partner performance with
the organization’s requirements through the SSPP process, there is no method in place to
ensure that day-to-day operations meet the key performance requirements. As a result, it
may be difficult for Northwest to assess on-going performance of key partners and
suppliers and quickly identify improvement opportunities.
Malcolm Baldrige National Quality Award—Feedback Report 33
Category 7 Organizational Performance Results
7.1 Student Learning Results
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Overall Academic Profile Performance (Figure 7.1-1) demonstrates the percentage of
students scoring above the national average of a nationally normed instrument. Results
on the Educational Testing Service (ETS) Academic Profile have increased from 40% in
1996–1997 to 70% in 2004–2005. In addition, the 62% in 2003–2004 exceeded the best
state competitor (60%) (Figure 7.1-1). During this period of increased performance, there
has not been an increase in incoming student ability. This demonstrates success in adding
value through education at Northwest.
Student learning also is measured using the Communications Competencies
(Figure 7.1-2) and Critical Thinking Competency (Figure 7.1-3) scores. Both results
show positive trends from Fall 2001 through Spring 2005, and both results for Spring
2005 are equal to the Masters institution level. Writing proficiency improved to 90%
in Spring 2005 from 77% in Fall 2001. Reading/critical thinking proficiency increased
to 80% in Spring 2005 from 75% in Fall 2001.
Nationally normed Major Field Examinations (Figure 7.1-4) are used to determine
student learning results. From the 1996–1997 school year through the 2004–2005 school
year, the university’s results show a positive trend from 42% to 67%, respectively.
Additionally, the 2004–2005 results exceeded the university’s best competitors.
Freshman Success Rate measures the percentage of students completing 24 or more credit
hours with a GPA of 2.0 or higher at the end of the first year. The results from 1995–1996
through 2003–2004 show a positive trend and an increase from approximately 64% to
75% (Figure 7.1-5). Additionally, the university’s performance has exceeded its best
within state competitors in their peer group of moderately selective institutions and two
of five institutions in the selective peer group.
Northwest’s Graduation Rate (Figure 7.1-6) has increased from approximately 43% in
1999 to approximately 62% in 2005. This is an increase of approximately 44%, and it
compares favorably to the best competitor’s (of five other moderately selective regional
universities) graduation rate of approximately 55%. Northwest was site visited by
AASCU to study institutions whose graduation rates are significantly higher than
expected. This supports the strategic challenge of maintaining differentiation by
producing exceptional student success.
Malcolm Baldrige National Quality Award—Feedback Report 34
Northwest meets or exceeds comparative peers for all measures presented for the Alumni
Outcomes Survey as shown in Figures 7.1-9 through 7.1-17. Key measures of computer,
discipline, personal-social, self-directed learning, communication, creating thinking and
problem solving, teamwork, multicultural, and cultural enrichment competencies show
that educational growth and development are consistently higher than alumni in the
comparative cohort.
OPPORTUNITIES FOR IMPROVEMENT
Although performance on the ACT Alumni Outcomes Survey (Figures 7.1-9 through
7.1-17) is at or better than comparative cohorts, performance trends on many of the
alumni measures have remained flat or slightly declining. This suggests that further
improvement is needed in the areas of critical/creative thinking and problem solving
competencies and multicultural competencies (Figure 6.1-2).
Although results for writing, math, and critical thinking, shown on the Provost Dashboard
profile, indicate that 10% of the student population score in the highest of three levels,
there has been a 10% or less improvement for the past five years. This is important to
the university’s ability to address KQI requirements (Figure 6.1-2).
On the National Survey of Student Engagement (NSSE) 2005 Means Comparison
Report, both freshmen and seniors report less coursework analyzing, synthesizing,
making judgments, and applying theories than the peer group. For example, the difference
in analyzing for freshmen is 2.76, less than the peer group at 2.94. The difference in
synthesizing for seniors is 2.86, less than the peer group at 3.03.
Although the organization has shown positive results in its student learning results, it
does not segment many results in a manner that would allow effective assessment of
on-going progress of various student segments. Without this segmentation, Northwest
may be missing opportunities to address specific segments of students to ensure the
university’s overall learning-centered objectives are being met.
Malcolm Baldrige National Quality Award—Feedback Report 35
7.2 Student- and Stakeholder-Focused Results
Your score in this Criteria Item for the site visit stage is in the 30–45 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Northwest uses the Noel-Levitz Student Satisfaction Survey as its institution-wide
measure of student satisfaction. Results for both freshmen and juniors exceed the
peer group in each of the past five school years (Figure 7.2-1). Key areas of service
indicators, including respect for students, competence of staff, timely service, and a
safe environment, exceed the ratings of the peer group (Figure 7.2-2).
The ACT Alumni Outcomes Survey is administered to all alumni three years after
graduation. Satisfaction of alumni with the quality of the academic program
(Figure 7.2-3) and positive referrals (Figure 7.2-5) have been greater than cohort
results since 2000.
Vendor satisfaction ratings on professionalism and courtesy, clarity of communication,
and accessibility of purchasing department staff are higher than 90% ―excellent,‖ with
no ―below average‖ or ―poor‖ ratings.
Freshmen and junior results for perceived value (Figure 7.2-4) have remained essentially
stable from 2001 to 2005 and are above comparative levels.
Employer Satisfaction of School Principals (Figure 7.2-8), a key stakeholder group, is at
approximately 95% or greater for subject matter, variety instructional strategies, effective
communication skills, and assessment strategies.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest outperforms comparative peers on a number of Service KQI
measures, several of these measures show flat to declining trends (e.g., freshmen
satisfaction with safe environment) (Figure 7.2-2).
In Northwest’s Alumni Outcomes Survey responses to the question ―Would you enroll
here again?‖ have been trending negative since 2002.
While stakeholder satisfaction measures (Figures 7.2-6 through 7.2-8) indicate levels
of satisfaction, trend and comparative information are not evident, making it difficult
to determine Northwest’s effectiveness in this area.
In Northwest’s Alumni Outcomes Survey (Figure 7.2-3), trends in satisfaction with
the quality of academic programs in the student’s major show declining satisfaction
since 2001.
Malcolm Baldrige National Quality Award—Feedback Report 36
7.3 Budgetary, Financial, and Market Results
Your score in this Criteria Item for the site visit stage is in the 70–85 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
―Moderate tuition rates‖ are one of Northwest’s key success factors. In 2004, tuition
and fee levels ranked fourth lowest of ten mid-range public institutions in the state
(Figure 7.3-4). Further, tuition and fee levels have been below its key regional competitor
every year since 1996.
Despite reduced state funding, internal allocation of budgets is maximized toward
instruction, and Northwest spends $1,200 per student more than competitors for
instruction. Instructional expenses per student full-time equivalent (FTE) show a
positive trend and increased from $5,131 in 2000 to $5,735 in 2004. From the 41 cohort
institutions, the university’s rank for instructional expenses per student FTE has improved
from 6th place in 2000 to 4th place in 2004. Additionally, Northwest’s rank for student
services per student FTE has improved from 20th place in 2000 to 10th place in 2004.
Northwest’s Fund Balance (Figure 7.3-1) is an important element in addressing the
strategic challenge of maintaining financial viability in the face of draconian reductions
in funding. Northwest’s fund balance has increased from less than $1 million in 2002 to
more than $4 million in 2004 (Figure 7.3-2). For the most recent seven years, Northwest
has been successful in maintaining its financial health and Auxiliary Debt Reserve above
the required level to maintain an A3 bond rating, the highest available for this category
of institution (Figure 7.3-2). Northwest’s ranking among state-funded competitors for
education and general funding improved from 4th in 1999–2000 to 1st in 2003–2004.
Northwest uses charitable contributions from alumni and stakeholders as a measure
of financial performance. In 2004, Northwest ranked eighth in charitable contributions
among public and private state colleges and universities, despite the fact that others are
larger institutions. Northwest’s best competitor ranked 15th.
Enrollment of Web/online students has increased from zero in 1999 to approximately
1,000 students in Fall 2005, a positive trend pattern with steady growth over the
past five years (Figure 7.3-6) that exceeds performance of two of three competitors.
Northwest has maintained its percentage of incoming freshman admitted into special
programs at approximately 20% from 2000 to 2005. Northwest’s result was better than
moderately selective in-state competitors in four of the last five years.
Northwest has identified maintaining enrollment levels as a strategic challenge.
Over the past seven years, Northwest’s total enrollment has averaged 5,209 FTEs,
which is considered to be at capacity, and enrollment has been within 100 FTEs of the
Malcolm Baldrige National Quality Award—Feedback Report 37
average in five of the past seven years. Increasing enrollment from the Kansas City
area is a key target. Freshmen enrollment from the Kansas City region has increased
from approximately 270 in 2000 to approximately 325 in 2005.
OPPORTUNITIES FOR IMPROVEMENT
Although Northwest has 2004 results for top competitors by major, it does not have
trended results and comparative levels for market performance measures (i.e., market
share; growth or loss of students, programs, and new markets; and market position
for the university’s programs and offerings). Lack of these results may hinder the
university’s ability to gauge its progress toward its specific strategic objectives,
including maintenance of enrollment levels.
No specific market analysis data are provided indicating the university may be
missing an opportunity to better analyze and understand the demographics of emerging
non-traditional student populations and segments. Strengthening these processes may
assist Northwest to better address its strategic enrollment challenges.
Malcolm Baldrige National Quality Award—Feedback Report 38
7.4 Faculty and Staff Results
Your score in this Criteria Item for the site visit stage is in the 30–45 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Faculty retention has remained at or above 95% for the last five periods, since
2000–2001 (Figure 7.4-1).
Annual salary increases for faculty, administrative staff, and support staff have exceeded
both the Consumer Price Index (CPI) and Missouri state merit employee increases for the
past twelve years (Figure 7.4-2).
Faculty satisfaction has improved from an index of 0.58 in 2000 to 0.82 in 2005
(Figure 7.4-3). The key satisfiers (faculty relations, professional work life, faculty
governance, and overall) have improved from 1998–1999 to 2004–2005 (Figure 7.4-4).
Staff job satisfaction has improved from about 70% in 1998–1999 to 73% in
2003–2004 (national comparison is about 69%) (Figure 7.4-5). Overall quality
has increased from 75% in 1998–1999 to about 80% in 2004–2005 (national
comparison is 60%) (Figure 7.4-5). Staff satisfiers of management/leadership
support, empowerment/teamwork, customer focus, and quality/productivity
improvement results are all above a comparative mean (Figure 7.4-6).
The university’s attendance and absenteeism results are better than the state results
for each of the past five years (Figure 7.4-8).
OPPORTUNITIES FOR IMPROVEMENT
Except for survey results for empowerment, results are not provided for measures or
indicators of work system performance and effectiveness, including the assurance that
faculty and staff are properly prepared to deliver Northwest’s learning-centered processes.
Northwest does not have results for faculty-staff learning levels. Lack of these results
may make it difficult for the university to gauge the effectiveness of its team structure
to support key learning-centered processes.
Although faculty retention has improved, staff retention has decreased from about
91% in 2001–2002 to about 86% in 2004–2005 (Figure 7.4-1).
Although Northwest’s Fall 2003 Worker’s Compensation Claims (approximately 15 per
HCT) (Figure 7.4-7) have improved slightly since 2002 (approximately 17 per 100), they
remain higher than the rate for 2001 (approximately 12 per 100). In addition, five of eight
comparative organizations show better performance for both 2002 and 2003.
Malcolm Baldrige National Quality Award—Feedback Report 39
7.5 Organizational Effectiveness Results
Your score in this Criteria Item for the site visit stage is in the 50–65 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
The Admissions Weekly Report cumulative total accepted for 2005 has exceeded 2003
levels since late 2004 and shows a positive trend (Figure 7.5-1). After determining the
number of students to enroll in the fall, mathematical targets are created. When progress
toward goals falls short, segmented data are analyzed and adjustments to marketing and
recruitment strategies are made.
Northwest provides tutoring assistance and measures its usage in its Academic Assistance
KQI Measure (Figure 7.5-4). Results show a positive trend over the past nine school years
from approximately 2,100 hours in 1995–96 to 8,500 hours in 2003–04 with the same
level of funding.
Data from Northwest Performance by vendor shows a 97% score on professionalism and
courtesy of staff. This exceeds the stretch goal by 20%. In addition, an overall rating of
partnership experience was 95%. This exceeds the stretch goal by 25%.
Noel-Levitz freshmen student satisfaction survey data show positive trends for residence
halls, campus organizations, student fees, and athletic programs for 2000–2004.
All measures exceed the national cohort mean, with no performance gaps from
2000–2004. This helps to ensure continual improvement of the educational climate.
Noel-Levitz freshmen and junior student satisfaction survey data show positive trends
for bookstore staff being helpful. In comparison to the national cohort mean, there were
no performance gaps from 2000–2004. This helps to ensure the responsiveness of
operational performance results to student needs.
Over the past 17 years, Northwest has shown a positive efficiency trend for
Environmental Services (Figure 7.5-6). University enrollment has increased from
10–25% per year, customer satisfaction has increased from 2–10% per year, and general
plant cost has decreased from 5–28% per year. An Alternative Energy Program has led
to cumulative savings since 1996–1996 of more than $5 million dollars (Figure 7.5-5).
Northwest provides data trending positive for operational performance results.
These includes campus food services timely service survey (3.25 on a 4-point scale),
textbook services timely service survey (3.5 on a 4-point scale), union meeting services
timely service survey (3.5 on a 4-point scale), conference center timely service survey
(3.5 on a 4-point scale), Bearcat Bookstore timely service survey (3.5 on a 4-point scale),
Malcolm Baldrige National Quality Award—Feedback Report 40
and educational services vendor performance (70% excellent exceeded stretch goal).
This ensures that key processes contribute to enhanced learning and organizational
effectiveness.
OPPORTUNITIES FOR IMPROVEMENT
The approach to improvement in services provided to Northwest is based on
discussions with the vendors rather than formative and summative measures.
Although Northwest demonstrates improving trends, relevant comparative
data are lacking to ascertain the extent that these results indicate leadership
in organizational effectiveness.
Malcolm Baldrige National Quality Award—Feedback Report 41
7.6 Leadership and Social Responsibility Results
Your score in this Criteria Item for the site visit stage is in the 50–65 percentage range.
(Please refer to Figure 5, ―Scoring Guidelines.‖)
STRENGTHS
Indicators for strategic initiative accomplishment (Figure 7.6-8) show eight of eleven
initiatives implemented, with cycles of improvement underway for five of the eight.
The remaining three initiatives are shown as ―continuing.‖
Northwest has more than 69 accredited degree programs (Figure 7.6-4) and has
received a number of awards from 1983–2001 for maintaining a safe/orderly, healthy,
well-functioning, attractive campus (Figure 7.6-4). The awards indicate Northwest’s
ongoing efforts to meet the key requirements of students and parents for an attractive,
well-maintained, student-centered environment.
The President’s Dashboard in Figure 7.0-1 reveals a color-scaled representation of results
in multiple categories, balanced in terms of importance. Purple indicates above-target
performance, green indicates at-target performance, yellow indicates below-target
performance, and red indicates well below-target performance. The majority of the
Dashboard indicators are either green or purple, indicating acceptable or above average
performance for most of the important indicators for the university.
From 1995 through 2004, Northwest has maintained an A3 bond rating, the highest
available (Figure 7.6-1), and it has had no legal, regulatory, or ethical compliance issues
(Figure 7.6-2). These results provide evidence that Northwest has effectively addressed
its financial performance Decision Driver and its cultural core value of ―We are open
and ethical.‖
OPPORTUNITIES FOR IMPROVEMENT
Northwest identifies ethics as one of its core values; however, results for
Terminations for Misconduct (Figure 7.6-6) show an increase from two in 2003–2004
to seven in 2004–2005 after three years of improving trends. This may indicate that
further refinements are needed in Northwest’s approach to ethics.
Few comparative or segmented performance indicators are provided for leadership and
social responsibility results. Without such segmentation of comparison data, it may be
difficult for the university to determine its success in achieving its core value of
―Leaders in our field.‖ In addition, there is no segmentation of data.
Malcolm Baldrige National Quality Award—Feedback Report 42
APPENDIX
By submitting a Baldrige application, you have differentiated yourself from most U.S.
organizations. The Board of Examiners has evaluated your application for the Malcolm
Baldrige National Quality Award. Strict confidentiality is observed at all times and in every
aspect of the application review and feedback.
This feedback report contains the Examiners’ findings, including a summary of key themes of
the application evaluation, a detailed listing of strengths and opportunities for improvement,
and scoring information. Background information on the examination process is provided
below.
APPLICATION REVIEW
Stage 1, Independent Review
Following the receipt of the Award applications, the first step of the Award Process review
cycle (shown in Figure 1) begins with Stage 1, the independent review, in which members of
the Board of Examiners are assigned to each of the applications. Assignments are made
according to the Examiners’ areas of expertise and to avoid potential conflicts of interest.
Each application is evaluated independently by Examiners who write comments relating to
the applicant’s strengths and opportunities for improvement and use a scoring system
developed for the Award Program. All applicants in all categories (manufacturing, service,
small business, education, and health care) go through the Stage 1 evaluation process.
Malcolm Baldrige National Quality Award—Feedback Report 43
Award Process Review Cycle
Applications Due
CD: Mid-May
Paper: Late May
Stage 1
Independent Review
June - July
Feedback Report
Judges Meet Not Selected to Applicant
Late-July
Selected
Stage 2
Consensus Review
August - September
Feedback Report
Judges Meet
Not Selected to Applicant
Mid-September
Selected
Stage 3
Site Visit Review
October
Feedback Report
Judges Meet Not Selected to Applicant
Mid-November
Judges Recommend Award
Recipients to NIST Director/
Sec’y of Commerce
Questions? Call the
Award Process Team in
the Baldrige Office at:
1-800-898-4506
Figure 1—Award Process Review Cycle
Malcolm Baldrige National Quality Award—Feedback Report 44
Stage 2, Consensus Review
Based on Stage 1 scoring profiles, the Panel of Judges selects applicants to go on to Stage 2,
the consensus review. If an applicant is not selected for consensus review, the comments
written by Examiners at Stage 1 are reviewed and used to prepare a feedback report.
For those applicants that do progress to Stage 2, a team of Examiners, led by a Senior
Examiner, conducts a series of conference calls to reach consensus on comments and scores
that capture the team’s collective view of the applicant’s strengths and opportunities for
improvement. The team documents its comments and scores in a consensus scorebook. The
consensus review process is shown in Figure 2.
Step 1 Step 2 Step 3
Consensus Planning: Consensus Calls: Post-Consensus Call
Prioritize Items for Discuss Key Business/ Activities:
Discussion Organization Factors Prepare Final Consensus
Assign Category/Item Discuss Items and Key Scorebook
Discussion Leaders Themes Prepare Feedback
Review Findings From Achieve Consensus on Report
the Independent Comments and Scores
Evaluations Document Findings
Figure 2—Consensus Review Process
Stage 3, Site Visit Review
After the consensus review process, the Panel of Judges selects applicants to receive site
visits based upon the scoring profiles. If an applicant is not selected for site visit review, one
of the Examiners on the Consensus Team edits the final consensus report that becomes the
feedback report.
Site visits are conducted for the highest-scoring applicants to clarify any uncertainty or
confusion the Examiners may have regarding the written application and to verify that the
information in the application is correct. After the site visit is completed, the team of
Examiners prepares a final site visit scorebook. The site visit review process is shown in
Figure 3.
Malcolm Baldrige National Quality Award—Feedback Report 45
Step 1 Step 2 Step 3
Team Preparation: Site Visit: Site Visit Scorebook:
Review Consensus Make/Receive Resolve Issues
Findings Presentations Summarize Findings
Develop Site Visit Issues Conduct Interviews Finalize Comments
Plan Site Visit Record Observations Prepare Final Site Visit
Review Records Scorebook
Prepare Feedback Report
Figure 3—Site Visit Review Process
Application reports, consensus scorebooks, and site visit scorebooks for all applicants
receiving site visits are forwarded to the Panel of Judges, which makes final
recommendations on which applicants should receive an Award. The Judges discuss
applications in each of the five Award categories separately, and then they vote to keep or
eliminate each applicant. If more than three applicants remain in a particular Award category,
the Judges rank order the applicants and eliminate those that rank lowest. This process is
repeated until the top three applicants remain. Next, the Judges decide whether each of the
top applicants should be recommended as an Award recipient based on an ―absolute‖
standard: the overall excellence and the appropriateness of the applicant as a national role
model. The process is repeated for each Award category; there may be as many as three
recipients in each of the categories. The Judges’ review process is shown in Figure 4.
Step 1 Step 2 Step 3
Panel of Judges’ Review: Evaluation by Category: Assessment of Top
Application Reports Manufacturing Organizations:
Consensus Scorebooks Service Overall Strengths/
Site Visit Scorebooks Small Business Opportunities for
Feedback Reports Education Improvement
Health Care Appropriateness as
National Model of
Performance Excellence
Figure 4—Judges’ Review Process
Malcolm Baldrige National Quality Award—Feedback Report 46
Judges do not participate in discussions or vote on applications in which they have a
competing or conflicting interest or in which they have a private or special interest, such
as an employment or a client relationship, a financial interest, or a personal or family
relationship. All conflicts are reviewed and discussed so that Judges are aware of their own
and others’ limitations on access to information and participation in discussions and voting.
Following the Judges’ review and recommendations of Award recipients, the Site Visit Team
leader edits the final site visit scorebook that becomes the feedback report.
SCORING
The scoring system used to score each Item is designed to differentiate the applicants in
the various stages of review and to facilitate feedback. The Scoring Guidelines for Business,
Education, or Health Care (shown in Figure 5) are based on (1) evidence that a performance
excellence system is in place; (2) the maturity of its processes as demonstrated by
Approach (A), Deployment (D), Learning (L), and Integration (I); and (3) the results it is
achieving.
In the feedback report, the applicant receives a percentage range. The percentage range is
based on the Scoring Guidelines, which describe the characteristics typically associated with
specific percentage ranges.
An applicant’s total scores fall into one of eight scoring bands. Each band corresponds to a
descriptor associated with that scoring range. Figure 6 provides scoring information on the
percentage of applicants scoring in each band at Stage 1. Scoring adjustments resulting from
the consensus review and site visit review stages are not reflected in the distribution.
Malcolm Baldrige National Quality Award—Feedback Report 47
SCORE PROCESS (For Use With Categories 1 –6)
No systematic approach is evident; information is anecdotal. (A)
0% or 5% Little or no deployment of an approach is evident. (D)
An improvement orientation is not evident; improvement is achieved through reacting to
problems. (L)
No organizational alignment is evident; individual areas or work units operate
independently. (I)
The beginning of a systematic approach to the basic requirements of the Item is evident.
10%, 15%, (A)
20%, or The approach is in the early stages of deployment in most areas or work units, inhibiting
25% progress in achieving the basic requirements of the Item. (D)
Early stages of a transition from reacting to problems to a general improvement orientation
are evident. (L)
The approach is aligned with other areas or work units largely through joint problem
solving. (I)
An effective, systematic approach, responsive to the basic requirements of the Item, is
30%, 35%, evident. (A)
40%, or The approach is deployed, although some areas or work units are in early stages of
45% deployment. (D)
The beginning of a systematic approach to evaluation and improvement of key processes is
evident. (L)
The approach is in early stages of alignment with your basic organizational needs
identified in response to the other Criteria Categories. (I)
An effective, systematic approach, responsive to the overall requirements of the Item, is
50%, 55%, evident. (A)
60%, or The approach is well deployed, although deployment may vary in some areas or work
65% units. (D)
A fact-based, systematic evaluation and improvement process and some organizational
learning are in place for improving the efficiency and effectiveness of key processes. (L)
The approach is aligned with your organizational needs identified in response to the other
Criteria Categories. (I)
An effective, systematic approach, responsive to the multiple requirements of the Item, is
70%, 75%, evident. (A)
80%, or The approach is well deployed, with no significant gaps. (D)
85% Fact-based, systematic evaluation and improvement and organizational learning are key
management tools; there is clear evidence of refinement and innovation as a result of
organizational-level analysis and sharing. (L)
The approach is integrated with your organizational needs identified in response to the
other Criteria Items. (I)
An effective, systematic approach, fully responsive to the multiple requirements of the
90%, 95%, Item, is evident. (A)
or 100% The approach is fully deployed without significant weaknesses or gaps in any areas or
work units. (D)
Fact-based, systematic evaluation and improvement and organizational learning are key
organization-wide tools; refinement and innovation, backed by analysis and sharing, are
evident throughout the organization. (L)
The approach is well integrated with your organizational needs identified in response to
the other Criteria Items. (I)
Figure 5—Scoring Guidelines for the Education Criteria
Malcolm Baldrige National Quality Award—Feedback Report 48
SCORE RESULTS (For Use With Category 7)
There are no organizational performance results or poor results in areas reported.
0% or 5% Trend data are either not reported or show mainly adverse trends.
Comparative information is not reported.
Results are not reported for any areas of importance to your key organizational
requirements.
A few organizational performance results are reported; there are some improvements
10%, 15%, and/or early good performance levels in a few areas.
20%, or Little or no trend data are reported.
25% Little or no comparative information is reported.
Results are reported for a few areas of importance to your key organizational requirements.
Improvements and/or good performance levels are reported in many areas addressed in the
30%, 35%, Item requirements.
40%, or Early stages of developing trends are evident.
45% Early stages of obtaining comparative information are evident.
Results are reported for many areas of importance to your key organizational requirements.
Improvement trends and/or good performance levels are reported for most areas addressed
in the Item requirements.
50%, 55%, No pattern of adverse trends and no poor performance levels are evident in areas of
60%, or importance to your key organizational requirements.
65% Some trends and/or current performance levels—evaluated against relevant comparisons
and/or benchmarks—show areas of good to very good relative performance.
Organizational performance results address most key customer, market, and process
requirements.
Current performance is good to excellent in most areas of importance to the Item
requirements.
70%, 75%, Most improvement trends and/or current performance levels are sustained.
80%, or Many to most reported trends and/or current performance levels—evaluated against
85% relevant comparisons and/or benchmarks—show areas of leadership and very good relative
performance.
Organizational performance results address most key student, stakeholder, market, process,
and action plan requirements.
Current performance is excellent in most areas of importance to the Item requirements.
Excellent improvement trends and/or sustained excellent performance levels are reported
90%, 95%, in most areas.
or 100% Evidence of education sector and benchmark leadership is demonstrated in many areas.
Organizational performance results fully address key student, stakeholder, market, process,
and action plan requirements.
Figure 5—Scoring Guidelines for the Education Criteria (Continued)
Malcolm Baldrige National Quality Award—Feedback Report 49
2005 Scoring Band Descriptors
Band Band % Applicants Descriptors
Number in Band1
0–275 1 5 The organization demonstrates the early stages of developing and implementing approaches to
Category requirements, with deployment lagging and inhibiting progress. Improvement efforts
focus on problem solving. A few important results are reported, but they generally lack trend and
comparative data.
276–375 2 17 The organization demonstrates effective, systematic approaches responsive to the basic
requirements of the Items, but some areas or work units are in the early stages of deployment. The
organization has developed a general improvement orientation that is forward-looking. The
organization obtains results stemming from its approaches, with some improvements and good
performance. The use of comparative and trend data is in the early stages.
376–475 3 34 The organization demonstrates effective, systematic approaches responsive to the basic
requirements of most Items, although there are still areas or work units in the early stages of
deployment. Key processes are beginning to be systematically evaluated and improved. Results
address many areas of importance to the organization’s key requirements, with improvements
and/or good performance being achieved. Comparative and trend data are available for some of
these important results areas.
476–575 4 28 The organization demonstrates effective, systematic approaches responsive to the overall
requirements of the Items, but deployment may vary in some areas or work units. Key processes
benefit from fact-based evaluation and improvement, and approaches are being aligned with
organizational needs. Results address key customer/stakeholder, market, and process
requirements, and they demonstrate some areas of strength and/or good performance against
relevant comparisons. There are no patterns of adverse trends or poor performance in areas of
importance to the organization’s key requirements.
576–675 5 14 The organization demonstrates effective, systematic, well-deployed approaches responsive to the
overall requirements of the Items. The organization demonstrates a fact-based, systematic
evaluation and improvement process and organizational learning that result in improving the
effectiveness and efficiency of key processes. Results address most key customer/stakeholder,
market, and process requirements, and they demonstrate areas of strength against relevant
comparisons and/or benchmarks. Improvement trends and/or good performance are reported for
most areas of importance to the organization’s key requirements.
676–775 6 2 The organization demonstrates refined approaches responsive to the multiple requirements of the
Items. These approaches are characterized by the use of key measures, good deployment,
evidence of innovation, and very good results in most areas. Organizational integration, learning,
and sharing are key management tools. Results address many customer/stakeholder, market,
process, and action plan requirements. The organization is an industry2 leader in some areas.
776–875 7 0 The organization demonstrates refined approaches responsive to the multiple requirements of the
Items. It also demonstrates innovation, excellent deployment, and good-to-excellent performance
levels in most areas. Good-to-excellent integration is evident, with organizational analysis,
learning, and sharing of best practices as key management strategies. Industry leadership and
some benchmark leadership are demonstrated in results that address most key
customer/stakeholder, market, process, and action plan requirements.
876–1000 8 0 The organization demonstrates outstanding approaches focused on innovation, full deployment,
and excellent, sustained performance results. There is excellent integration of approaches with
organizational needs. Organizational analysis, learning, and sharing of best practices are
pervasive. National and world leadership is demonstrated in results that fully address key
customer/stakeholder, market, process, and action plan requirements.
____________________
1. Percentages are based on scores from the Stage 1 review.
2. Industry refers to other organizations performing substantially the same functions, thereby facilitating direct comparisons.
Figure 6—Scoring Band Descriptors
Malcolm Baldrige National Quality Award—Feedback Report 50