Get documents about "Flowchart of Hiring Through Job Portals - PDF
W
Description
Flowchart of Hiring Through Job Portals document sample
Document Sample
Career Service Authority
Recruiting Process
Performance Audit
August 2009
Office of the Auditor
Audit Services Division
City and County of Denver
Dennis J. Gallagher
Auditor
The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is
responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the
proper and efficient use of City resources and providing other audit services and information to City
Council, the Mayor and the public to improve all aspects of Denver’s government. He also chairs the
City’s Audit Committee and oversees the City’s Comprehensive Annual Financial Report (CAFR)
The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee
assists the Auditor in his oversight responsibilities of the integrity of the City’s finances and operations,
including the integrity of the City’s financial statements. The Audit Committee is structured in a manner
that ensures the independent oversight of City operations, thereby enhancing citizen confidence and
avoiding any appearance of a conflict of interest.
Audit Committee
Dennis Gallagher Maurice Goodgaine
Robert Haddock Jeffrey Hart
Charles Husted Bonney Lopez
Timothy O’Brien
Audit Staff
John Carlson, JD, CIA, CICA, Deputy Director
Nancy Howe, Internal Audit Supervisor
Jacob Claeys, CICA, Lead Internal Auditor
Marcus Garrett, CIA, CGAP, Lead Internal Auditor
Mary Mutchler, CICA, Senior Internal Auditor
Travis Henline, Senior Internal Auditor
Emily Gibson, Staff Internal Auditor
You can obtain free copies of this report by contacting us at:
Office of the Auditor
201 W. Colfax Avenue, Dept. 705 Denver CO, 80202
(720) 913-5000 Fax (720) 913-5026
Or view an electronic copy by visiting our website at:
www.denvergov.org/auditor
City and County of Denver
201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000 FAX 720-913-5247
www.denvergov.org/auditor
Dennis J. Gallagher
Auditor
August 20, 2009
Jeff Dolan, Executive Director
Career Service Authority
City and County of Denver
Dear Mr. Dolan:
Attached is the Auditor‘s Office Audit Services Division‘s report of their audit of the Career
Service Authority Recruiting Process for the year ended December 31, 2008. The purpose of the
audit was to evaluate the operational efficiency and effectiveness of the City‘s recruiting
process and to determine whether internal controls in place were adequate under the
circumstances.
Audit work identified opportunities for improvement in all four aspects of the recruiting function
examined, including strategic planning, the timeliness of the recruiting function, consistency in
recruiting practices, and oversight. The weaknesses identified in CSA‘s recruiting process inhibit
the efficiency and effectiveness of the process and limit accountability.
If you have any questions, please call Kip Memmott, Director of Audit Services, at
720-913-5029.
Sincerely,
Dennis J. Gallagher
Auditor
DJG/mkm
cc: Honorable John Hickenlooper, Mayor
Honorable Members of City Council
Members of Audit Committee
Ms. Kelly Brough, Chief of Staff
Mr. Claude Pumilla, Chief Financial Officer
Mr. David T. Roberts, Chief Services Officer
Mr. David Fine, City Attorney
Mr. L.Michael Henry, Staff Director, Board of Ethics
Ms. Lauri Dannemiller, City Council Executive Staff Director
Ms. Beth Machann, Controller
To promote open, accountable, efficient and effective government by performing impartial reviews and other audit
services that provide objective and useful information to improve decision making by management and the people.
We will monitor and report on recommendations and progress towards their implementation.
City and County of Denver
201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000, FAX 720-913-5247
www.denvergov.org/auditor
Dennis J. Gallagher
Auditor
AUDITOR’S REPORT
We have completed an audit of the Career Service Authority‘s Recruiting Process for the period
January 1, 2008 through December 31, 2008. The purpose of the audit was to examine and
assess the City‘s recruiting processes to identify possible inefficiencies and opportunities for
improvement. Audit work focused primarily on strategic planning, the timeliness of the recruiting
function, consistency in recruiting practices, and adequate oversight as key performance
indicators.
This performance audit is authorized pursuant to the City and County of Denver Charter, Article
V, Part 2, Section 1, General Powers and Duties of Auditor, and was conducted in accordance
with generally accepted government auditing standards. Those standards require that we plan
and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis
for our findings and conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on our audit
objectives.
Audit work identified opportunities to improve all four aspects of the recruiting function
examined. Specifically:
CSA currently lacks both a clear definition of what the merit system seeks
to accomplish and a strategic plan that establishes goals and helps the
agency adjust to future developments;
CSA‘s recruiting and hiring process appears to take longer than other
jurisdictions;
CSA personnel are not consistent in how they document or perform their
recruiting activities and weak controls such as poor security over tests
create situations that potentially threaten fair and open competition for
City positions; and
Accountability is hampered by the lack of sound performance measures,
limited data, and ineffective tools for both evaluating employees‘
performance and reviewing the timeliness and quality of recruiting efforts.
We extend our appreciation to the CSA personnel who assisted and cooperated with us during
the audit.
Audit Services Division
Kip Memmott, MA, CGAP, CICA
Director of Audit Services
To promote open, accountable, efficient and effective government by performing impartial reviews and other audit
services that provide objective and useful information to improve decision making by management and the people.
We will monitor and report on recommendations and progress towards their implementation.
TABLE OF CONTENTS
EXECUTIVE SUMMARY 1
Opportunities Exist to Improve the Recruiting Function 1
INTRODUCTION & BACKGROUND 4
Overview of Denver’s Career Service Authority 4
Overview of CSA’s Recruiting Process 5
CSA Recruiting Uses a Variety of Assessment Methods 8
Equal Employment Opportunity Laws Applicable to the City’s Recruiting
Process 9
CSA Has Recently Experienced Organizational Change 10
SCOPE 10
OBJECTIVE 10
METHODOLOGY 11
FINDING 12
Opportunities Exist to Improve the Recruiting Function 12
Lack of a Clearly Defined Merit System and a Weak Strategic Plan Hinder
CSA’s Effectiveness 12
The City’s Recruiting and Hiring Process Does Not Produce Timely Results 15
Inconsistencies and Problems Executing CSA’s Processes Hinder Objectivity
in the City’s Recruiting Efforts 17
Accountability Is Hampered by Limited Management Information and Poor
Oversight Mechanisms 22
RECOMMENDATIONS 28
APPENDIX A 32
City Positions Not in the Career Service (Charter § 9.1.1 E) 32
AGENCY RESPONSE 33
EXECUTIVE SUMMARY
Opportunities Exist to Improve the Recruiting Function
Audit work identified opportunities to improve all four aspects of the recruiting function
examined — strategic planning, the timeliness of the recruiting function, consistency in
recruiting practices, and oversight. The weaknesses identified in CSA‘s recruiting process
inhibit the efficiency and effectiveness of the process and limit accountability. In
summary:
CSA currently lacks both a clear definition of what the merit system seeks to
accomplish and a strategic plan that establishes goals and helps the agency
adjust to future developments;
CSA‘s recruiting and hiring process appears to take longer than other jurisdictions;
CSA personnel are not consistent in how they document or perform their
recruiting activities, and weak controls such as poor security over tests create
situations that potentially threaten fair and open competition for City positions;
and
Accountability is hampered by the lack of sound performance measures, limited
data, and ineffective tools for both evaluating employees‘ performance and
reviewing the timeliness and quality of recruiting efforts.
Specifically, audit work identified the following areas for improvement.
Lack of a Clearly Defined Merit System and a Weak Strategic Plan Hinder CSA‘s
Effectiveness
Audit work determined that the components of the City‘s merit system are not
specifically defined. As a result, key stakeholders do not have a consistent understanding
of what the merit system should encompass. This lack of clarity has implications for how
the recruiting process is carried out.
Without a clear understanding and definition of what a merit system means, CSA is
limited in its ability to determine whether current recruiting processes reflect merit
principles, whether some processes such as testing are even necessary, and whether a
merit system is effective or necessary.
In addition, audit work found CSA does not have a comprehensive strategic plan.
Although CSA does have information typically contained in a strategic plan, it is located
in three separate documents. This limits the organization‘s ability to use the information to
plan effectively for the future.
The City‘s Recruiting and Hiring Process Does Not Produce Timely Results
A key issue for a recruiting function is whether it succeeds in placing new employees in
positions as expeditiously as possible. The most qualified candidates are generally desired
by many employers, so as the length of time to hire increases, the opportunity to hire the
O f f ice of t he A udit or
Page 1
most qualified decreases. Audit work found that, relative to other jurisdictions, CSA‘s
recruiting process takes longer. A lengthy recruiting and hiring process can negatively
affect overall customer satisfaction. There are steps CSA can take to improve the overall
efficiency of the process.
For example, an audit sample found the average time to fill a position in 2008 was 79.9
days. Compared against two industry standards, the City‘s process is slower. CSA is
primarily a customer service agency with a variety of customers, including hiring
agencies, potential job candidates, as well as current City employees. While most
agencies reported they are fairly satisfied with the relationship and service they receive
from CSA, several agencies also felt the recruiting process is too complicated or overly
stringent. Audit work identified several steps that could be taken to shorten the review
process to include:
Eliminating or compressing steps;
Identifying ways to increase efficiency by the hiring agency; and
Shortening the process for certain types of positions.
Inconsistencies and Problems Executing CSA‘s Processes Hinder Objectivity in
the City‘s Recruiting Efforts
Audit work found CSA lacks policies and procedures to help prevent discrepancies and
potential problems in the recruiting process. Detailed examination of the recruiting
process showed numerous inconsistencies in how CSA personnel conducted and carried
out various steps of the process. In addition, gaps and lapses in some procedures create
situations in which fair and open competition for positions is potentially affected.
Rule 3 serves as the primary policies and procedures for Recruiting. Audit work showed
that while Rule 3 does provide some guidance, it is inadequate on its own because it is
broad and vague. Recruiting has no additional written policies and procedures, and the
result is incomplete or inconsistent recruiting practices. Specific examples include:
Key recruiting steps are not documented by Recruiters;
Recruiting does not have procedures for deciding when to eliminate recruiting
steps; and
Recruiters are inconsistent in utilizing special and preferred qualifications.
Additionally, audit work revealed significant weaknesses in the physical security of test
books. Auditors conducted an inventory of the test books stored in CSA‘s test vault and
identified three discrepancies.
Accountability Is Hampered by Limited Management Information and Poor
Oversight Mechanisms
Audit work determined CSA has little ability to monitor and report on its performance. It is
generally not collecting the kinds of measures needed to accomplish this task effectively.
Further, in several instances when it did report information about its performance in order
for the Board to make key decisions about the recruiting process, the information was
Cit y and Count y of Denver
Page 2
inaccurate. Likewise, key oversight mechanisms, including performance evaluations of
employees and quality control reviews of the agency‘s work are weak.
Specifically, audit work determined there are two ways in which the current
performance measurement system limits the ability to assess the efficiency and
effectiveness of the recruiting function. First, monitoring and collection of performance
information is inconsistent. Second, the performance indicators monitored yield little
information about efficiency and effectiveness, and the indicators do not align with best
practices.
The limited focus on such indicators reduces CSA‘s and the Board‘s ability to analyze the
efficiency and effectiveness of recruiting activities. For example, the cost of recruiting is
unknown. CSA does not know how much time Recruiters spend on each recruitment or
the total cost to hire an employee. Weaknesses in current performance measures
impede CSA‘s ability to determine whether the City‘s resources are being wisely and
effectively spent. The limited information also reduces CSA‘s ability to plan strategically
for the future.
Furthermore, audit work identified several instances of inaccurate reporting. These
involved performance information in various CSA reports and the content of
presentations to the CSA Board. Reporting must be accurate in order for key
stakeholders to hold Recruiting accountable for its performance. Inaccurate information
diminishes the ability of stakeholders to evaluate the overall effectiveness of Recruiting.
Additionally, accurate information is necessary to determine whether CSA is adhering to
the merit system.
Audit work found existing quality controls over the recruiting process are vague and
largely undocumented. Recruiting has developed a general reporting structure for
quality review of Recruiters‘ work, but no way to monitor whether reviews occur. The lack
of a well defined and documented quality control process limits CSA‘s knowledge of
performance, their ability to hold individual personnel accountable, and increases the
risk of inaccuracies.
CSA was specifically established as an independent agency to limit inappropriate
political influence, but because of this independence, the CSA Board should actively
provide oversight and hold CSA accountable for carrying out the agency‘s mission and
accomplishing its goals. Action is needed on multiple fronts to strengthen the City‘s
recruiting function. In many respects, these issues are interrelated.
O f f ice of t he A udit or
Page 3
INTRODUCTION & BACKGROUND
& BACKGROUND
Overview of Denver’s Career Service Authority
The City and County of Denver is one of the largest employers in the Denver
metropolitan area. About 68% of the City and County‘s more than 13,000 employees are
part of what is called the Career Service. These employees include general positions
such as administrative personnel, laborers, and professionals. 1 Filling these positions with
qualified applicants is a sizeable task. In fiscal year 2008, the City and County had 1,769
new hires for positions within the Career Service.
The Career Service Authority (CSA) is the primary agency that provides human resource
services for Career Service employees. CSA has over 80 authorized positions, including
Recruiters, Training Specialists, Classification/Compensation Specialists, and Benefit
Specialists. The agency‘s duties include:
Recruiting, examining, and certifying applicants for employment and promotion;
Administering the personnel rules that govern appointments, promotions,
demotions, transfers, and layoffs;
Providing Citywide training;
Setting compensation and classification details for City employees;
Administering employee benefits; and
Managing employee grievances and other employee relations.
Established under the City Charter in 1954, CSA's overall mission is to uphold and
administer a merit system. Among other things, a merit system allows job applicants to
compete for and obtain employment based on their knowledge, skills, and abilities.2
Merit systems have been utilized by government for decades and were intended to
combat the "spoils" system in which government jobs were openly bought and sold.
The City Charter establishes CSA as an independent agency—that is, an agency
removed from the Mayor‘s direct control. CSA‘s operations are overseen by a governing
board, called the Career Service Authority Board (the Board). The Board is comprised of
1
Examples of positions that are not part of the Career Service include elected officials and uniformed staff of the police and fire
departments. Appendix A lists the positions not included in the Career Service.
2
The federal government has a detailed framework for achieving fair and open human resource policies and practices called
the Merit System Principles and Prohibited Personnel Practices. The City and County of Denver is not required to abide by these
practices.
Cit y and Count y of Denver
Page 4
five members who are appointed by the Mayor and confirmed by City Council for five-
year staggered terms. The Board‘s duties include:
Overseeing the Career Service Board Personnel Rules (Rules) that govern
employment practices within the Career Service. These Rules address hiring, pay,
classification, transfers, dispute resolution, promotions, training, and other
personnel matters. The Board may revise or rescind proposed Rule changes at
any time provided the process is performed in accordance with the rule-making
process specified in the Rules;
Overseeing CSA, including appointing CSA‘s Executive Director; and
Overseeing the Career Service Hearing Office, and serving as a quasi-judicial
body to decide appeals of decisions of the Career Service Hearings Officers. 3
Overview of CSA’s Recruiting Process
The Recruiting Section (Recruiting) within CSA‘s Workforce Management Division is
responsible for providing recruitment services for Career Service positions. Rule 3 governs
the process for recruiting potential employees. Candidate selection is a key component
of the recruiting and hiring process and is instrumental in helping to ensure the City hires
the most qualified candidates and upholds principles of a merit system. City agencies
play an important role in the hiring process; however, CSA will perform some agency
functions upon request.
The general process to fill open positions is depicted in Figure 1.4 As the figure shows,
CSA is only one of the City agencies involved in the process. In general, CSA‘s focus is on
advertising the position and establishing a list of the most highly qualified candidates. The
hiring agency then decides which of these candidates to hire. The process can be
separated into three main phases as follows:
Planning and Position Announcement. When a hiring agency has a vacancy, it
typically contacts the City‘s Budget and Management Office to verify available
funding. If the Budget and Management Office determines funding is available, it
completes an approval to fill form and returns it to the hiring agency, which then
contacts CSA Recruiting. Then recruiting preparation begins. This preparation
includes a CSA Recruiter working with the hiring agency to determine the
recruitment strategy, the methodology for testing applicants, and the posting
(job announcement) plan. When complete, Recruiting posts the open position to
the City website and any other agreed-upon external websites.
Applicant Screening. As individuals apply for an open position, Recruiting reviews
every application received and creates what is called an eligible list. This list
3
City and County of Denver Career Service Board Personnel Rules, Rule 2-11A: Duties and Organization of the Board, Effective
December 23, 2005, http://denvergov.org/HumanResources/RulesandPolicies/tabid/432517/Default.aspx (accessed March 20,
2009).
4
The process described here is used for most recruitments, but not all. Other approaches taken by CSA are discussed elsewhere
in this report.
O f f ice of t he A udit or
Page 5
includes all applicants who meet the position‘s minimum qualifications. For these
applicants, Recruiting administers an assessment—that is, a test designed to
further determine applicants‘ relative abilities—and ranks applicants in order of
their scores. Using these scores, Recruiting creates what is called a certified list
(also known as a referred list or referral list), which comprises the top 20 ranked
candidates. It then transmits this certified list to the agency.
Interview and Selection. The hiring agency determines which candidates from the
certified list it wishes to interview. CSA Rules require agencies to interview a
minimum of three candidates from the certified list. The hiring agency negotiates
salary, makes a verbal offer, conducts background checks, and sends an offer
letter. CSA may partner with the hiring agency or perform these tasks as well. The
CSA Recruiter verifies that the hiring agency complied with City policies, such as
interviewing at least three candidates from the certified list. After the selected
candidate begins work, the City Controller‘s Office completes I-9 paperwork.
Cit y and Count y of Denver
Page 6
Figure 1: City Recruiting Process
O f f ice of t he A udit or
Page 7
CSA Recruiting Uses a Variety of Assessment Methods
CSA Recruiting uses a variety of assessment methods, both in screening applicants‘
qualifications and in developing and administering examinations. When an open position
is posted, the position description lists the required minimum qualifications, such as the
minimum level of education or relevant experience required. These minimum
qualifications are set for each job class by CSA‘s Classifications Division. Neither
Recruiting nor hiring agencies may amend these qualifications under any circumstances.
At the same time, agencies may include two additional types of qualifications, as
follows:
Special qualifications. Rule 3 allows agencies to require additional qualifications
beyond the minimum if the skills are necessary to perform the job. For example,
an agency seeking to recruit an Administrative Support Assistant may require an
additional qualification to be bilingual if, as a regular part of that job‘s duties, the
hired person will be interacting with non-English speakers. CSA refers to these as
special qualifications and Recruiters may screen applicants for special
qualifications in the same manner they screen for minimum qualifications.
Preferred qualifications. These qualifications are not covered in Rule 3 or in any
other written CSA policy. They are qualifications that a hiring agency would prefer
the ideal candidate to have, but are not necessary to perform the position. For
example, the minimum qualifications required for a Senior Accountant are a
Bachelor‘s Degree and two years of accounting experience, but a particular
agency may prefer their Senior Accountant have government accounting
experience. This could be stated on the job announcement as a preferred
qualification. There are no written restrictions regarding when an agency may
request preferred qualifications in a recruitment. Additionally, there are no written
instructions on how these are to be used in recruiting.
CSA administers an examination for every position in the Career Service. These
examinations are developed by CSA Recruiters and Test Specialists in consultation with
the hiring agency. According to CSA‘s Test Operations Manual, the following are the
most common tests given to Career Service job applicants:
Written test questions. These are test booklets typically administered to high
volume recruitments or at agency request. Questions originate from the County
of San Bernardino, California and are used widely by personnel agencies around
the country. Referred to as the ‗Western Region Item Bank,‘ this database
contains more than 54,000 test questions covering 375 subject matter categories.
Supplemental questions. These are job-specific open-ended questions that are
graded using established rating criteria. They are typically part of the job posting,
allowing CSA to obtain applicant answers with the application. Generally, this
type of exam is developed for professional-level job classifications.
Cit y and Count y of Denver
Page 8
Performance test. These tests simulate an actual job requirement and are
typically administered for Equipment Operator and Heavy Equipment Operator
candidates. These examinations are expensive to administer and, therefore, tend
to be included in a limited number of exam plans.
Skills test. CSA also has programs available to conduct skills based testing on
computer applications, such as Microsoft Word, Excel and Access.
Other test options may be developed between the Test Specialists and hiring agency
based on need.
Equal Employment Opportunity Laws Applicable to the City’s
Recruiting Process
As an employer, the City must abide by federal Equal Employment Opportunity (EEO)
laws. These laws and regulations include:
Title VII of the Civil Rights Act of 1964 - Prohibits discrimination and other unlawful
employment practices based on race, color, religion, sex, or national origin;
The Americans with Disabilities Act (ADA) - Prohibits discrimination on the basis of
disability;
The Age Discrimination in Employment Act (ADEA) - Prohibits age discrimination in
employment; and
The Equal Pay Act (EPA) - Prohibits sex-based wage discrimination as part of the
Fair Labor Standards Act.
While CSA personnel take many steps to comply with Equal Employment Opportunity
Commission (EEOC) regulations, the concept of adverse impact is directly related to
issues discussed in this report. Specifically, the EEOC has Uniform Guidelines that
incorporate a single set of principles which are designed to assist employers in
eliminating employment practices which discriminate on the grounds of race, color,
religion, sex, and national origin. These guidelines provide a framework for determining
the proper use of tests and other selection procedures. This framework particularly
addresses adverse impact which is defined as ―a substantially different rate of selection
in hiring, promotion, or other employment decision which works to the disadvantage of
members of a race, sex, or ethnic group.‖5
5
Equal Employment Opportunity Commission (EEOC), Uniform Guidelines on Employee Selection Procedures (1978), 43 FR
38295 (August 25, 1978), 29 CFR Part 1607, Section 16B, http://www.uniformguidelines.com/uniformguidelines.html (accessed
April 7, 2009).
O f f ice of t he A udit or
Page 9
CSA Has Recently Experienced Organizational Change
A new Executive Director joined CSA at the end of 2007 and initiated an extensive
reorganization beginning in 2008. This greatly impacted the Recruiting Section as well as
CSA‘s other divisions. Recruiters‘ responsibilities were expanded and most Recruiting
support staff was eliminated. Further, CSA hired a new Recruiting Supervisor in mid-2008.
A customer service survey completed as part of audit work indicated that since these
major changes have taken place, agencies have seen improvement. This audit seeks to
identify areas for additional improvement and add to the progress already made by CSA
leadership.
In 2006, CSA Recruiting began phasing in NEOGOV, an online recruiting management
system. Recruiters now use NEOGOV to recruit for all agencies in the City. In 2008, CSA
launched NEOGOV Online Hiring Center, an additional software package that allows
the hiring agencies to utilize NEOGOV in their part of the recruiting process.
SCOPE
The scope of the audit was to examine CSA‘s recruiting processes and practices.
Consequently, it focused primarily on the Recruiting Section within CSA's Workforce
Management Division. Audit work also included evaluating the CSA Board and the
duration of City agencies‘ portion of the hiring process. The audit period extended from
January 1, 2008 through December 31, 2008.
OBJECTIVE
Our objective was to assess the following aspects of the CSA recruiting process:
Does Recruiting have clearly articulated goals and a suitable plan for
accomplishing them?
Is the process producing timely results in terms of screening applicants and filling
open positions?
Are components of the process consistently implemented, so that the process is
objective and fair?
Does management have accurate, useful, and timely information that can be
used to monitor and improve the process, as well as effective controls for
ensuring accountability?
Cit y and Count y of Denver
P a g e 10
METHODOLOGY
Evidence gathering and analysis techniques used to meet audit objectives included, but
were not limited to:
Interviewing CSA personnel and personnel with human resources responsibilities
at other City agencies;
Attending a NEOGOV training course;
Attending CSA Board meetings;
Documenting relevant CSA policies and practices;
Analyzing the City Charter, the Denver Revised Municipal Code, CSA Rules, and
federal requirements related to recruiting and hiring;
Tracking new job postings for 45 days including both business days and
nonworking days;
Reviewing job postings, position qualifications, job testing, pre-employment
screenings, and CSA employee documentation practices;
Benchmarking against other municipalities regarding their recruiting and hiring
practices;6
Attending applicant examinations conducted by CSA;
Testing various recruitment-related practices;
Collecting and analyzing NEOGOV data; and
Evaluating NEOGOV usage and capabilities.
6
Auditors selected the following benchmark jurisdictions due to the similarity of their population and total staff size to
Denver’s: Long Beach, CA; Portland, OR; Fulton County, GA; and Oklahoma City, OK. Some of these jurisdictions also utilize
NEOGOV.
O f f ice of t he A udit or
P a g e 11
FINDING
Opportunities Exist to Improve the Recruiting Function
Audit work identified opportunities to improve all four aspects of the recruiting function
examined—planning, timeliness, consistency, and oversight. The weaknesses identified in
CSA‘s recruiting process inhibit the efficiency and effectiveness of the process and limit
accountability. In summary:
CSA currently lacks both a clear definition of what the merit system seeks to
accomplish and a strategic plan that establishes goals and helps the agency
adjust to future developments;
CSA‘s recruiting and hiring process appears to take longer than other jurisdictions,
and CSA‘s ability to monitor its performance is limited by gaps and inconsistencies
in the data it collects;
CSA personnel are not consistent in how they document or perform their
recruiting activities, and weak controls such as poor security over tests create
situations that potentially threaten fair and open competition for City positions;
and
Accountability is hampered by the lack of sound performance measures, limited
data, and ineffective tools for both evaluating employees‘ performance and
reviewing the timeliness and quality of recruiting efforts.
Lack of a Clearly Defined Merit System and a Weak Strategic Plan
Hinder CSA’s Effectiveness
Audit work determined that the components of the City‘s merit system are not
specifically defined. As a result, key stakeholders do not have a consistent understanding
of what the merit system should encompass. This lack of clarity has implications for how
the recruiting process is carried out. In addition, the absence of a comprehensive
strategic plan limits CSA‘s ability to adjust to future developments.
CSA Has Not Clearly Defined the Components of a Merit System
Although the principles of the merit system are the foundation of CSA's existence, CSA
does not have a specific definition for the City‘s merit system or a consistent view of what
its components should be. This results in differing interpretations of the merit system
throughout the City. As part of audit work, auditors asked 16 key personnel to define the
merit system, including three in CSA management, all five Board members, and eight
Human Resource (HR) representatives from various City agencies. No more than 50% of
the respondents gave the same general definition and two did not recognize the term
merit system. Table 1 summarizes the interviewees‘ responses.
Cit y and Count y of Denver
P a g e 12
Table 1: Merit System Definitions Provided by Survey Respondents
“Merit System” means: No. of % of Responses*
Responses
Fair and Competitive Hiring 8 50.0%
Testing or Ranking to Hire Based on Skills 7 43.8%
Pay for Performance or Fair Pay 6 37.5%
Consistent/Fair Rules and Employee 5 31.3%
Treatment
Opposite of “Spoils” or Political Patronage 2 12.5%
Don’t Know 2 12.5%
*Auditors obtained 16 responses. Multiple answers were accepted so totals do not equal
100%.
This lack of clear and consistent focus has implications for how the system is administered
and what it accomplishes. For example:
Recruiting personnel reported the merit system is the reason testing is required of
all candidates. However, developing and administering a test adds time and
costs to the recruiting process and is not necessarily needed to conform to merit
system principles. Audit work found three of the four benchmarking cities
reported adherence to the merit system, and none of these cities require all
candidates be tested as part of their recruiting process.
In January 2009, the Board approved reducing the minimum amount of time a
job announcement can be posted from five to two business days. This might
jeopardize the merit principle of fair and open competition. For example, the
federal government recommends a minimum posting of five days. Additionally,
two of the four benchmark jurisdictions require posting a job announcement for a
minimum of five business days, and one requires a minimum posting of 7 business
days.7
Without a clear understanding and definition of what a merit system means, CSA is
limited in its ability to determine whether current recruiting processes reflect merit
principles, whether some processes such as testing are even necessary, and whether a
merit system is effective or necessary.
7 The fourth city requires a minimum posting of 3 days, but typically posts job opportunities for two weeks.
O f f ice of t he A udit or
P a g e 13
Lack of a Comprehensive Strategic Plan Limits CSA‘s Ability to Adjust for Future
City Demands
Audit work found CSA does not have a comprehensive strategic plan. Although CSA
does have information typically contained in a strategic plan, it is located in three
separate documents. This limits the organization‘s ability to use the information to plan
effectively for the future.
A study conducted by the International Public Management Association for Human
Resources (IPMA-HR) indicates the absence of a strategic plan reduces a human
resource organization‘s ability to function effectively. The study states, "The failure to have
a strategic plan in place suggests that the organization approaches HR management in
a fragmented and reactive fashion that, by its very nature, is less effective and more
costly than working within a thoughtful design."8 The study recognizes that developing a
strategic plan requires commitment from agency management. According to the study,
"If organization leadership does not recognize value in HR, strategic planning may not be
a visible priority, and thus, not required or mandated by leadership. Strategic planning
requires that HR have the staff, time and energy needed to complete a planning process
regardless of how simple or broad the resulting plan." 9 Without a comprehensive strategic
plan, CSA is in a less advantageous position to incorporate new ideas or strategies into
the City‘s HR efforts.
One strategy currently utilized by the City is branding. Branding, a fairly new concept in
the public sector, has been used by private-sector companies as a way to sell products.
According to the IPMA-HR, ―Branding is strategic and often depends on consistency in
messaging, communications and standards. This same concept can be applied to
public sector organizations seeking to establish themselves as more than just an entity
tasked with basic service delivery. Public sector organizations are beginning to recognize
the importance of reputation or how the organization is seen through the eyes of a
potential employee. Branding is about taking charge of that reputation, defining the
benefits of working in a public sector environment and marketing these factors to
potential employees.‖10 A review of CSA‘s branding strategy revealed materials to
include:
Recruiting banners used at job fairs identifying the City logo;
Pamphlets on training, organizational development, and job opportunities;
Brochures promoting the City and County of Denver as a great place to work;
and
Full-page color print outs used for executive recruitments.
8
EquaTerra, “Recruiting and Staffing in the Public Sector: Results from the IPMA-HR Research Series”, International Public
Management Association for Human Resources, http://ipma-hr.org/content.cfm?pageid=72 (accessed June 3, 2009).
9
Ibid.
10
Ibid.
Cit y and Count y of Denver
P a g e 14
CSA also utilizes electronic formats offering candidates the ability to receive information
regarding available positions by e-mail and text
messaging. Additionally, CSA advertises on
CSA uses Twitter and Facebook to
social media sites such as Twitter.com and
promote City job openings.
Facebook.com.
However, although CSA uses a variety of
strategies to promote the City, it appears Recruiters may not always be aware of or
actively utilizing them. CSA‘s branding strategy was not mentioned in interviews
conducted with Recruiting staff. Furthermore, staff in charge of career fairs and university
outreach stated CSA does not have a branding strategy. If Recruiting staff is not
consistently utilizing the branding tools available, the City risks losing qualified candidates
that are unaware of the benefits of working in the public sector, or who may not have
considered the City and County of Denver for employment.
The City’s Recruiting and Hiring Process Does Not Produce Timely
Results
A key issue for a recruiting function is whether it succeeds in placing new employees in
positions as expeditiously as possible. The most qualified candidates are generally desired
by many employers, so as the length of time to hire increases, the opportunity to hire the
most qualified decreases. Audit work found that, relative to other jurisdictions, CSA‘s
recruiting process takes longer. A lengthy recruiting and hiring process can negatively
affect overall customer satisfaction. There are steps CSA can take to improve the overall
efficiency of the process.
Available Data Indicate Other Jurisdictions Process Job Applicants More Quickly
An audit sample found the average time to fill a position in 2008 was 79.9 days.11
Compared against two industry standards, the City‘s process is slower.
Comparison with other jurisdictions that use the same recruiting software – In 2006,
IPMA-HR issued a benchmarking study that surveyed public-sector hiring
agencies using NEOGOV, the online recruiting software CSA began using in
2006.12 This study found the mean number of days reported for ―notification of
vacancy until reports to work‖ (which represents an earlier starting time point and
therefore a longer timeframe than was measured in the audit sample) was 49
days, or nearly 31 days shorter than CSA‘s average time.
Comparison with federal standards – The federal government has an 80-day
standard for its hiring process. While this standard may appear very close to the
City‘s average of 79.9 days, closer comparison shows CSA‘s process is actually
slower in several respects. First, the federal government starts its measurement
11
This is the number of days from when a Recruiter first enters recruitment information in NEOGOV (after step 4 on Figure 1,
page 7) and the new employee reports to work.
12
International Public Management Association for Human Resources (IPMA-HR), “2006 Recruitment and Selection
Benchmarking,” IPMA-HR, http://ipma-hr.org/content.cfm?pageid=42 (accessed January 20, 2009).
O f f ice of t he A udit or
P a g e 15
earlier in the process—the day the request to fill a position is received, and CSA
does not measure recruiting preparation time. If auditors had measured CSA‘s
processing time from this earlier point, the average processing time would be
greater. Second, the federal hiring time includes 14 days to advertise the job,
compared to the minimum of two for CSA, and time for completing an extensive
security background check the City does not perform. This indicates other parts
of CSA‘s process are taking much longer.
Customer Service is Negatively Impacted by Timeliness Issues
Audit work found weaknesses identified in CSA‘s recruiting processes and practices
negatively impact customer satisfaction. CSA is primarily a customer service agency with
a variety of customers, including hiring agencies, potential job candidates, as well as
current City employees. Audit work included a survey of the City‘s hiring agencies. While
most agencies reported they are fairly satisfied with the relationship and service they
receive from CSA, several agencies also felt the recruiting process is too complicated or
overly stringent. In a customer satisfaction survey completed by CSA, employees who
were less than completely satisfied were asked to offer suggestions for improvement to
make the selection process better. Of those who responded, 52% stated they would like
to see a faster process.
Options Exist for Making the Process More Timely
Audit work identified several steps that could be taken to shorten the recruiting process.
Eliminating or compressing steps – Audit work identified some steps in the
recruiting process that are not mandated by law or Rule and could potentially be
eliminated. One example is the current requirement that the City‘s Budget and
Management Office (BMO) verifies available funding prior to initiating recruiting
activities. However, the hiring agency should be able to determine whether
adequate funding is available. Even if this step is retained, there are ways to
modify and compress the process. One of the benchmark jurisdictions begins
recruiting as soon as the recruiting agency is notified that a position is open,
rather than waiting for BMO‘s approval. This allows the City to begin its advertising
and screening efforts while funding is approved. When Recruiting is notified the
position can be filled, it is able to send its certification list to the hiring agency
within one day.
Identifying ways to increase efficiency by the hiring agency – Audit testing
showed the hiring agency‘s portion of the hiring process comprised
approximately 45-50 days of the 79.9 day average, or about 60% of the total
time. While this portion of the process is largely outside of CSA‘s control, they are
currently not monitoring this time. By measuring and addressing this time, CSA
could pursue strategies for working with City agencies to decrease it.
Shortening the process for certain types of positions – CSA does not have formal
policies and procedures in place to allow agencies to fill positions when there is
an immediate need. For example, two of the four benchmark jurisdictions allow
Cit y and Count y of Denver
P a g e 16
individual agencies to hire their own on-call (temporary) employees. Allowing
City agencies to bypass CSA by directly hiring or utilizing a temporary agency to
fill on-call positions could meet their needs quickly while reducing CSA‘s
workload. Additionally, a 2000/2001 study conducted by IPMA/NASPE identified
immediate job offer processes for hard to fill positions as a best practice.13
Inconsistencies and Problems Executing CSA’s Processes Hinder
Objectivity in the City’s Recruiting Efforts
Audit work found CSA lacks policies and procedures to help prevent discrepancies and
potential problems in the recruiting process. Detailed examination of the recruiting
process showed numerous inconsistencies in how CSA personnel conducted and carried
out various steps of the process. In addition, gaps and lapses in some procedures create
situations in which fair and open competition for positions is potentially affected.
CSA Personnel Execute the Process Inconsistently
Rule 3 serves as the primary policies and procedures for Recruiting. Audit work showed
that while Rule 3 does provide some guidance, it is inadequate on its own because it is
broad and vague. Recruiting has no
additional written policies and procedures, Recruiting processes are
and the result is incomplete or inconsistent inconsistent.
recruiting practices. Specifically, the following
inconsistencies were identified:
Key Recruiting Steps Not Documented by Recruiters – Audit testing indicated CSA
Recruiters document key steps inconsistently. Rule 3 does not specifically address which
steps are to take place, and when auditors requested written policies and procedures,
they found the steps were not written. The Recruiting Supervisor reported the following
procedures should occur:
When an approval to fill form is required by the Budget and Management Office,
Recruiters should attach the completed form in NEOGOV;
Recruiters should be using at least two subject matter experts (SMEs), or when an
expert from the agency grades supplemental questions asked of applicants. There
are no guidelines designating when it is appropriate for Recruiters to grade the
questions or when a SME should be used;
Recruiters should complete a service level agreement for any recruitment that has
not been conducted by that specific Recruiter before. It is not required for high-
volume recruitments or for citywide mass recruiting (i.e. Administrative Support
Assistant). Any time an agreement is completed, it should be attached in
NEOGOV; and
13
International Public Management Association/ National Association for State Personnel Executives, “2000/2001 IPMA/NASPE
HR Benchmarking Project: Best Practices”, IPMA-HR, http://ipma-hr.org/content.cfm?pageid=380 (accessed January 20, 2009).
O f f ice of t he A udit or
P a g e 17
During a typical recruitment, Recruiters should notify applicants when they do not
meet the minimum qualifications, when they are deemed eligible, when they are
referred to the hiring agency and when the position is filled. This is automatically
documented in NEOGOV.
To determine if these steps were performed, auditors tested two recruitments for each of
the ten Recruiters. The testing compared the procedures identified by the Recruiting
Supervisor with recruitment data from NEOGOV. In order for management to adequately
ensure requirements are fulfilled, each of these recruiting steps should be documented in
NEOGOV. As shown in Table 2, documentation was not present the majority of the time.
Table 2: Required Recruiting Steps Compared With NEOGOV Data
Recruitment Documentation Number of Times
Documentation Not
Present in NEOGOV
Attachment of approval to fill form 8 of 18*
Evidence provided that more than one subject 15 of 18**
matter expert was used
Service Level Agreement Attached 16 of 20
Applicants notified at all appropriate stages of 17 of 20
recruitment
*Two recruitments tested were exempt from this requirement.
**Two recruitments did not use supplemental questions as a test
mechanism.
These findings indicate Recruiters are inconsistent when entering information into
NEOGOV.
Additional audit work found some data from the time to hire sample had to be excluded
due to inaccuracy and incompleteness. Specifically, 11 requisitions that auditors tried to
verify had to be excluded due to errors found (i.e. the person recorded as hired for that
position was actually hired for another position). Furthermore, auditors found variations in
how Recruiters recorded recruitments using an existing certified list. In some cases, little to
no data was recorded about these particular recruitments which made it difficult to
determine who was hired when. NEOGOV has the potential to provide useful information
that can be used for monitoring and overseeing the efficiency and effectiveness of the
recruiting process, but this potential cannot be tapped if basic information entered into
the system is incomplete and inaccurate.
No Procedures for Deciding When to Eliminate Recruiting Steps – Depending on the
circumstances of a recruitment, the recruiting process may be legitimately shortened
without violating merit principles. For example, if an agency is hiring for an Administrative
Cit y and Count y of Denver
P a g e 18
Support Assistant and a certified list is already available, Recruiters would not need to
complete many of the steps in the workflow process. Similarly, if the agency hired the
same staff position on many occasions, Recruiters may be able to eliminate the hiring
agency strategy meeting. When Recruiters are legitimately able to adjust the recruiting
process, agencies may be able to fill their open positions faster. While CSA provided
instances when the recruiting process may be shortened, there are no written policies
identifying when steps may be eliminated or who has the authority to eliminate certain
steps.14
Inconsistency in Creating Eligible Lists – Recruiters could be formulating eligible lists
differently because Rule 3 is unclear as to when applicants may be placed on the
eligible list. According to Rule 3, an eligible candidate is ―an applicant for a vacant
position who meets the criteria required for placement on a list‖.15 ―Criteria‖ is not
defined, which means it is subject to interpretation. For example, the eligible list could
include applicants that only meet minimum qualifications, or it could include those who
meet minimum qualifications and pass applicable tests.
Inconsistency in Utilizing Special and Preferred Qualifications – Recruiters gave
inconsistent answers as to when special and preferred qualifications can be used in
screening, eliminating and ranking applicants. Specifically, only six Recruiters reported
they could screen and eliminate applicants based on special qualifications. CSA Rule 3 is
clear that Recruiters may screen for special qualifications. Further, six Recruiters reported
preferred qualifications could not be used in ranking applicants for the certified list. CSA
Rules do not address the use of preferred qualifications.
Using special and preferred qualifications to the fullest extent possible is important in
order to ensure Recruiters are providing the best customer service to agencies. Recruiters
and agencies do not have the ability to amend minimum qualifications. CSA‘s
Classifications Division reported minimum qualifications are broad by design, as they
must apply to whole job classes. Further, Classifications reported special and preferred
qualifications should be used when possible in order for agencies to get the best
qualified candidate for the position. It appears from Recruiter interview responses that
these qualifications are being inconsistently or inaccurately used. In the case of preferred
qualifications, it is unclear how these relate to the certified list of candidates agencies
choose from.
Additionally, the Recruiting Supervisor reported to auditors that special qualifications may
not be used for some entry-level positions, which contradicts audit work. When special
and preferred qualifications are not used to the fullest extent it can increase applicant
numbers and workload, eligible lists could be formulated differently, and it could reduce
the number of highly qualified candidates.
14
Examples include recurring recruitments and recruiting for parks, seasonal and labor positions.
15
City and County of Denver Career Service Board Personnel Rules, Rule 3-5(C): Selection Definitions, Effective May 4, 2007,
http://denvergov.org/HumanResources/RulesandPolicies/tabid/432517/Default.aspx (accessed June 10, 2009).
O f f ice of t he A udit or
P a g e 19
Inconsistency in Developing Supplemental Test Questions – Recruiters and Test Specialists
develop supplemental questions individually and questions may vary from recruiter to
recruiter, even for the same job classification. Further, the established CSA review process
for supplemental questions is spread throughout several documents and does not
appear to be clearly monitored. As a result, questions may not be developed, graded
and documented in a consistent manner.
Identified Control Weaknesses Potentially Affect Fair and Open Competition for
Positions
In some aspects of the recruiting process the lack of sufficient policies and procedures
creates situations that could jeopardize the fairness and openness of competition for
positions. These instances were as follows:
Certified Lists Can Remain Open for Lengthy Periods and Be Used for Multiple
Recruitments – Audit work found that candidates are being hired after they are
technically no longer eligible due to a flaw in Rule 3. Rule 3 places an expiration date on
the ―eligible‖ list, but not on the ―certified‖ list.16 CSA sends a certified list to the hiring
agency and the agency can hire from this list for an undetermined length of time, even
when the eligible list may have lapsed.
For example, auditors identified a candidate who was made eligible and whose name
was sent to the agency via the certified list on August 31, 2007. Because the eligible list
expires after 90 days, this candidate technically became ineligible for consideration on
November 29, 2007. However, the candidate was hired on May 27, 2008, roughly nine
months after being made eligible. The hiring agency was not in violation of Rule 3, since
there is no expiration date for the certified list. Contacting candidates when sufficient
time has elapsed could negatively impact the City‘s reputation and compromises its
ability to hire quality candidates. The lack of closure on the certified list circumvents the
intent to hire in a timely manner.
The lack of an expiration date on the certified list allows it to be reused. Although there is
an interest in efficiency when using a certified list several times, there needs to be
balance. An expiration date on one list and not the other creates imbalance between
using an existing certified list and conducting a new recruitment.
Audit work showed the practice of reusing certified lists may be extensive. When auditors
attempted to review recruitments in September 2008 for time to hire information, only 17
positions could be tested, even though there were approximately 70 requisitions created
in that month. This was mainly due to most new hires coming from an existing certified list,
meaning a new recruitment was not conducted for the requisition.17 In other words, only
17 recruitments were conducted (that could be verified) out of 70 requisitions created
16
For an explanation of these two lists, see “Overview of CSA’s Recruiting Process” in the Introduction and Background section
of this report.
17
In some cases, auditors had to exclude the requisition, not because the hiring occurred off an existing certified list, but
because too much data was missing due to Recruiters entering information into NEOGOV inconsistently.
Cit y and Count y of Denver
P a g e 20
that month. This calls into question the balance between conducting full scale
recruitments versus using an existing certified list to fill positions. When existing lists are
used too much, at some point it will impact the quality of people hired.
Weaknesses in Security Over Tests May Compromise Results – Audit work revealed
significant weaknesses in the physical security of test books. Auditors conducted an
inventory of the test books stored in CSA‘s test vault and identified three discrepancies:
Eight of 22 (36%) tests counted did not match the Test Specialists‘ master inventory
list;
The vault contained test books for three positions that were not on the master
inventory list; and
The master inventory listed two types of test books that were not in the vault.
These discrepancies indicate Recruiters do not consistently use the Test Content Check
Out Log or Test Administration Security Control Record to sign out test books from
inventory. In addition to limiting CSA‘s ability to manage test inventory, it is difficult to
determine which Recruiter removed test books from the vault.
Besides finding discrepancies in the number of test books on hand, auditors also
identified other inventory or security weaknesses that further inhibit Recruiting‘s ability to
manage test inventory:
Tests that are destroyed or are no longer in use are not accounted for on the
master inventory list;
CSA Personnel were unable to identify the number of tests originally printed, so no
historic (original) inventory numbers were maintained for comparison. The first
documented inventory of the test vault was completed by CSA in January 2009;
Recruiting has not established a standard number of test books to be printed for
each examination, making it more difficult to determine whether the number on
hand is complete. For example, auditors counted 49 test books for one test,
matching the number on the master inventory list. However, it is more likely a round
number, such as 50 books, was originally printed; and
A security issue exists related to access to the vault where tests are kept. A key to
the vault is available to all Recruiting staff, and Recruiting does not have a process
to check the key in and out.
Weaknesses in the physical security of test books increase the risk of exam results being
compromised. This could potentially allow under or unqualified applicants to pass exams.
Because qualified applicants are ranked on the certified list according to test score,
certain applicants may appear more qualified than they are, making it more difficult for
hiring agencies to determine and hire the best candidate.
O f f ice of t he A udit or
P a g e 21
These weaknesses are more significant when considering the time needed to take
corrective action. Recruiting personnel reported that when a written test is lost or stolen,
they develop a new test for that position. The creation of a new test may take between
two to six months, depending on the type of questions contained in the test and the time
commitments required by other projects in Recruiting.
Increased Legal and Public Relations Liability Related to Possible Identity Theft – Audit
work found supplemental test books for one position contained valid social security
numbers. Access to the vault, combined with availability of personal information such as
names and social security numbers, could potentially lead to identity theft and a
corresponding lack of trust from the clients CSA serves.
Pass Points May Unintentionally Screen Out Qualified Candidates – Audit work
determined Recruiting‘s use of pass points may increase the risk of similarly qualified
candidates being inconsistently removed from further consideration. 18 Specifically, the
candidate‘s test score, regardless of the pass point, determines their rank. However,
utilizing a pass point may eliminate statistically similar candidates from further
consideration.19
Furthermore, although adverse impact may not be entirely removed, CSA‘s current use
of fixed pass points may open the City to claims of discrimination. Test Specialists began
assigning fixed pass points to each test with the implementation of NEOGOV. The prior
automated software allowed Test Specialists to run comprehensive reports that helped
them account for and remove adverse impact by adjusting the pass point. However,
Test Specialists no longer run these types of reports due to NEOGOV‘s limited reporting
capabilities.
Accountability Is Hampered by Limited Management Information
and Poor Oversight Mechanisms
Audit work determined CSA has little ability to monitor and report on its performance. It is
generally not collecting the kinds of measures needed to accomplish this task effectively.
Further, in several instances when it did report information about its performance in order
for the Board to make key decisions about the recruiting process, the information was
inaccurate. Likewise, key oversight mechanisms are weak, including performance
evaluations of employees and quality control reviews of the agency‘s work.
Most Available Information is Not Useful in Assessing Efficiency and Effectiveness
Performance Indicators Are Inconsistent and of Limited Use – Audit work determined
there are two ways in which the current performance measurement system limits the
ability to assess the efficiency and effectiveness of the recruiting function. First,
18
The minimum score candidates must achieve on the test to continue in the recruitment process.
19
For example, in a test with a pass point of 65 and an expected error rate of 3 points, one candidate’s score of 64 is statistically
the same as another candidate’s score of 67; however, the candidate who scores 64 will be screened out for not passing the
test.
Cit y and Count y of Denver
P a g e 22
monitoring and collection of performance information is inconsistent. Second, the
performance indicators monitored yield little information about efficiency and
effectiveness, and the indicators do not align with best practices.
CSA‘s collection of performance information is sporadic. According to the Recruiting
Supervisor, performance information is not produced regularly and is mostly prepared for
outside meetings or other purposes. This limits Recruiting‘s ability to conduct trend
analyses and identify progress or emerging problems. The lack of consistent performance
monitoring creates difficulties when using any of this information in decision making.
Moreover, internal accountability is difficult to achieve if information is being collected
sporadically and only for outside use.
When CSA does collect and report performance information, it tends to focus on agency
workload. Specifically, CSA Recruiting has reported on a limited basis the following
performance indicators:
number of job postings;
applications received;
applications per posting;
testing sessions held; and
positions filled.
These indicators provide little information about the agency‘s efficiency or effectiveness.
In contrast to the kinds of information CSA collects and reports, best practices research
indicates the following types of indicators provide better information about an agency‘s
performance:
time to hire (including measuring the time each step takes);
cost per hire;
time spent per recruitment;
volume of applications;
turnover rates;
completion of probation; and
qualified applicants per posting.
CSA has reported in the past on two of these indicators—volume of applications and
time to hire—but only on a limited basis. The limited focus on such indicators reduces
CSA‘s and the Board‘s ability to analyze the efficiency and effectiveness of Recruiting
activities. For example, the cost of recruiting is unknown. CSA does not know how much
time Recruiters spend on each recruitment or the total cost to hire an employee.
Although performance indicators quantifying cost are only a piece of what should be
O f f ice of t he A udit or
P a g e 23
monitored according to best practices, cost is important in helping to determine
Recruiting efficiency and effectiveness. Weaknesses in current performance measures
impede CSA‘s ability to determine whether the City‘s resources are being wisely and
effectively spent. The limited information also reduces CSA‘s ability to plan strategically
for the future. The effectiveness of any plan or steps to improve functions is diminished
without information demonstrating how goals will be accomplished.
Available Information is Often Incorrect or Reported Inaccurately
Audit work identified several instances of inaccurate reporting. These involved
performance information in various CSA reports, and the content of presentations to the
CSA Board.
Issues Identified With Data Submitted in CSA Reports – Audit work identified key problems
in the creation of two different reports submitted to the City.
In its 2008 annual report, CSA reported the average number of days to fill a
position was 29.5 days. While audit work suggests this figure represents the time it
takes to create a certified list, worded this way, the number appears to represent
the time it takes until the new employee reported for work.
Beyond the annual report, CSA reports on its success in meeting bonus goals.20
CSA reported it met its 2008 goals of creating a certified list for non-exempt
(hourly) and exempt (salary) positions within 30 and 45 days, respectively.
Specifically, CSA reported taking an average of 27.4 days to create a certified list
for non-exempt positions and 32.1 days to create a certified list for exempt
positions. Auditors‘ attempts to recreate these figures produced slightly higher
averages.
Auditors attempted to review CSA‘s calculations but were unable to do so because CSA
did not document their methodology for data verification or save the supporting data.
Consequently, auditors performed a separate, more detailed analysis of a sample of
recruitments conducted during the same period covered in CSA‘s reports.21 This analysis
indicated a higher average time to create the certified list than CSA reported.
Table 3 compares CSA figures to figures calculated by auditors. Specifically, the bonus
goal for creating a certification list for non-exempt positions is 30 days. CSA reported an
average time of 27.4 days, which qualified for the bonus, while auditors‘ sample showed
an average time of 30.5 days, which would not have qualified for the bonus. The
discrepancy resulting from this audit work highlights the importance of the data in
NEOGOV, and the importance of CSA ensuring data used for reporting is accurate and
reproducible.
20
Bonus goals are annual performance measures established by agencies. The City pays bonuses to employees when agencies
meet their bonus goals.
21
CSA’s figures were for the time period January – October 2008. Auditors tested a sample from every other month in the same
time period.
Cit y and Count y of Denver
P a g e 24
Table 3: Measurement to Fill Positions
Measurement CSA Reported Auditor’s Office
Calculation
ANNUAL REPORT: Average Time to fill 29.5 Days22 79.9 Days
a Position
BONUS GOALS: Time to create a 27.4 Days 30.5 Days
certification list for non-exempt
positions
BONUS GOALS: Time to create a 32.1 Days 35.4 Days
certification list for exempt positions
Additionally, audit work revealed the methods CSA used to calculate its averages
produced inaccurate and unreliable results. For example, CSA used a pre-programmed
NEOGOV report which pulls the first date CSA refers candidates to the agency (the end
date of CSA‘s bonus goal calculation), not the date the actual candidate hired was
referred. Audit work identified several instances where the report generated a date
much earlier than the hired candidate‘s actual date referred – possibly skewing CSA‘s
average calculations. For example, auditors identified a case where the NEOGOV report
pulled July 10, 2008 as the date the candidate was referred to the agency, but upon
verification, the candidate hired for this position was actually referred to the agency on
September 24, 2008. Therefore, it is possible the date used for calculation in this case was
more than two months shorter than the actual date the candidate was hired. The only
way auditors were able to obtain 100% reliable data was to verify individual records by
comparing the date referred in NEOGOV to the hire date in PeopleSoft.23 CSA did not
consistently take this step in conducting the analysis for its reports and there is no way for
auditors to know when CSA verified referred dates.
Problems with this report may be eliminated if CSA utilized custom NEOGOV reports. CSA
is currently making minimal use of this capability. Overall, accountability could be
improved if CSA takes greater advantage of this function.
Inaccurate Information Presented to the Board – CSA Management presented a large
model of the recruiting workflow process to the Board to support a Rule change, but
reported to auditors that the flowchart was inaccurate. On January 15, 2009, CSA
requested a change to Rule 3-21 regarding the minimum required days for posting job
positions. The previous Rule required that job postings appear for a minimum of 5 business
22
This figure may represent the time to create a certified list. However, CSA was unable to provide documentation to
substantiate whether it represents time to create a certified list or time to fill.
23
PeopleSoft is the City’s human resource and accounting software system.
O f f ice of t he A udit or
P a g e 25
days, whereas the new Rule reduced that to a minimum of two business days. CSA
reported that the Rule change was necessary to limit the number of applications
because processing time increases as the volume of applications increases.
As CSA stated in this public hearing, a day would count towards the total only if the
posting appeared for the full day. For example, if a job was posted on a Monday
afternoon, Monday would not count as a day. This requirement would ensure that the
posting appeared all day Tuesday and all day Wednesday with the job closing at
midnight on Wednesday. Board members approved the change but requested that CSA
update them on how many times jobs were posted for less than five days (Quick Posts),
which supervisors requested the shorter postings, which positions they were for, and the
impact of shortening the time.
On March 19, 2009, CSA presented some of the requested information related to the
impact of the Rule change. Audit work determined CSA did not report all of the
requested information, and presented information that was inaccurate. Specifically:
CSA reported three Quick Posts had been made since the January Rule change.
However, audit work determined that a total of 12 positions were posted for less
than five days as of March 19; and
CSA presented the ―average applicants per position‖ figure which auditors could
not successfully recreate. It should be noted that the problems in re-creating this
data could be due to a recent archiving project in NEOGOV, but without the
original methodology used to create these figures it cannot be verified that this is
the case.
Reporting must be accurate in order for key stakeholders to hold Recruiting accountable
for its performance. Inaccurate information diminishes the ability of stakeholders to
evaluate the overall effectiveness of Recruiting. Additionally, accurate information is
necessary to determine whether CSA is adhering to the merit system.
Performance Evaluations Are Vague and Not Timely
Audit work identified several ways in which CSA management‘s review of staff
performance provided limited accountability. First, the document most commonly used
to discuss staff performance (the duty matrix) does not adequately define roles,
responsibilities, standards, and expectations for Recruiters. The duty matrix contains a
general breakdown of duties for which Associate, Senior and Recruiting Specialists are
responsible. However, this matrix is deficient in several respects as an accountability tool:
The matrix does not contain any definitions or explanation of what each duty
entails. For example, the matrix lists both customer service and client
management—terms that, without further definition, can be interpreted
differently and confused easily. It is difficult to hold Recruiters accountable for
unclear duty assignments.
The matrix lacks an explanation of how it should be applied or how performance
will be measured. All Recruiters interviewed had negative comments about the
Cit y and Count y of Denver
P a g e 26
duty matrix, and seven of ten Recruiters stated the matrix was not an accurate
breakdown of duties. Without clarifying documents, the matrix is an ineffective
management tool.
A second problem with staff accountability is that while better evaluation tools exist, they
are not used to assess most Recruiters‘ performance. The duty matrix was the only
standardized Recruiter performance evaluation tool provided to auditors which was
used consistently in 2008.
A third problem is none of the CSA Recruiting staff received their annual performance
evaluations, called Performance Evaluation Plan Report (PEPR), in a timely manner.
Specifically, ten of the 13 (77%) Recruiting staff members received a forced evaluation
on their most recent PEPR.24 At the time of testing, the remaining three staff had been
employed less than one year, so a formal evaluation was not due. Performance
appraisals provide several important benefits for employees, including feedback on
performance and areas for performance improvement. They also benefit the employer
by helping improve employees‘ understanding of job responsibilities and standards of
performance, thereby providing management with a means to hold them accountable.
Data obtained from CSA indicated the timeliness of performance evaluations is an issue
citywide. It appears the City‘s framework to hold supervisors accountable for completing
PEPRs in a timely manner is inadequate. Audit work found that the State of Colorado has
laws in place to hold supervisors accountable when they do not complete evaluations.
These include progressive discipline for each year the evaluation is not completed,
ranging from suspension (unpaid days off) to
demotion.25 The Rules state that supervisors
Timeliness of performance
failing to complete PEPRs within 30 days of the
evaluations is a citywide problem.
due date ―may be subject to discipline.‖26 This
rule is vague about what kind of disciplinary
actions may be used and leaves this responsibility up to the agencies. The agencies are
not required to do anything. While CSA does not have the authority to insist on timely
PEPRs citywide, it does have the authority to request a citywide Rule change to address
these problems.
Quality Control and Oversight Processes Need Strengthening
Audit work found existing quality controls over the recruiting process are vague and
largely undocumented. Recruiting has developed a general reporting structure for
quality review of Recruiters‘ work, but no way to monitor whether reviews occur.
Recruiting management was able to list some items that should be checked during
quality reviews, such as spelling and content accuracy of job announcements, but could
not provide any written documentation of such reviews or the requirements. The lack of
24
A forced evaluation is when City employees do not receive their formal annual evaluation within 30 days of the due date. As a
result, the employee is automatically rated as “Successful” and given the corresponding pay increase.
25
C.R.S. § 24-50-104(1)(c.5) (2009).
26
City and County of Denver Career Service Board Personnel Rules, Rule 13-61 (C4): Merit Date, Effective January 1, 2008,
http://denvergov.org/HumanResources/RulesandPolicies/tabid/432517/Default.aspx (accessed June 10, 2009).
O f f ice of t he A udit or
P a g e 27
a well defined and documented quality control process limits CSA‘s knowledge of
performance, their ability to hold individual personnel accountable, and increases the
risk of inaccuracies.
Further, audit work found there is a lack of quality controls in the testing process. For
example, CSA does not appear to perform consistent quality reviews of exams to ensure
testing effectively measures applicant competency. Additionally, although Recruiting
had some policies and procedures related to checking written exams out and back in
for testing applicants, audit work found recruiting personnel were not adhering to them.
The absence of quality controls in the testing process is a significant problem because it
increases the risk of litigation related to unfair testing procedures.
CSA Governance Can be Enhanced
CSA was specifically established as an independent agency to limit inappropriate
political influence, but because of this independence, the CSA Board should actively
provide oversight and hold CSA accountable for carrying out the agency‘s mission and
accomplishing its goals.
Action is needed on multiple fronts to strengthen the City‘s recruiting function. The
preceding sections have detailed a number of issues that need addressing. In many
respects, these issues are interrelated. For example, while better accountability
mechanisms are needed, these mechanisms will not work well if good performance
indicators are not in place. Simply conducting more performance evaluations and
quality reviews will accomplish little if there are no indicators that measure what the
process is accomplishing. However, even the best performance indicators are of limited
use if employees are not consistently entering data into the NEOGOV system or making
use of the system‘s reporting capabilities.
RECOMMENDATIONS
The following recommendations are designed to help CSA and its Board address issues
identified by audit work. Recommendations are grouped to reflect the sequence of
topics discussed in the report, but in some cases recommendations made under one
heading may help address matters under other headings as well.
Establishing Strategic Direction and Planning
1. To better establish strategic direction and planning for the recruiting function, CSA, in
consultation with the Board, should take the following steps:
Establishing a clear, actionable definition of the merit system and making it a part
of agency documents—for example, in the purpose statement or the
introductory statement of the CSA Rules. The definition of the merit system should
be detailed enough to identify when Rules are appropriate and how it
specifically applies to all personnel issues;
Cit y and Count y of Denver
P a g e 28
Consider increasing the minimum job posting time from two to five business days
to align with the federal government and other benchmarking jurisdictions;
Developing, in a single document, a strategic plan that includes measurable
goals and reportable performance measures that can be tracked over time; and
Ensure all Recruiters are aware of materials and tools available to promote CSA‘s
branding strategy.
Improving the Timeliness of Recruiting
2. CSA should evaluate the full recruiting and hiring process in an effort to identify steps
where efficiencies could be realized. These steps could include:
Identifying steps that hiring agencies can take to shorten their portion of the hiring
process;
Pursuing rule changes that allow for quick hiring, such as authorizing agencies to
hire their own on-call positions or utilize temp agencies; and
Starting initial recruitment steps at the time the hiring agency decides to fill an
open position rather than waiting until the Budget and Management Office
determines funding for the position is available.
Making Policies and Procedures More Thorough and Consistent
3. To ensure greater consistency in the recruiting process and integrity in data collection,
CSA should create written policies and procedures that cover the following:
Explaining when Recruiters may eliminate steps in the recruiting process;
Specifying the documentation to be attached in NEOGOV to ensure consistent
and complete recruiting records;
Specifying procedures for developing eligible and certified lists;
Specifying when subject matter experts should be used in applicant testing,
including how many should be used;
Specifying what information should be consistently entered and how to enter it
into NEOGOV, including recruitments that utilize an existing certified list;
Clarifying when and how it is appropriate to use special or preferred
qualifications and promote usage by the hiring agency, including:
o Using special qualifications to establish clear and concise requirements for
the open position and help limit the number of applications from
candidates that clearly do not meet the qualifications needed for the
position.
o Ranking eligible candidates based on preferred qualifications for the
certified list, in conjunction with test scores; and
O f f ice of t he A udit or
P a g e 29
Clearly documenting the development of supplemental test questions and
specifying that supplemental questions are standardized for like job postings, to
the degree possible.
4. CSA should seek a Rule change establishing an expiration date for certified lists to
ensure that these lists are not used beyond the expiration of the eligible lists.
5. To better ensure the security of test books, CSA should take the following steps:
Perform an inventory of the test vault in order to create a baseline for future
inventories, establish a set number of test books for each examination, and
maintain documentation of test books that are destroyed;
Consider replacing manual test books with computer based or video based tests.
Video based tests have been shown to produce a more positive response from
candidates, and reduce adverse impact;
Eliminate pass points on written tests. By eliminating the pass point, each eligible
candidate has the opportunity to move forward in the hiring process; and
To address test security and pass point issues, CSA could also consider eliminating
testing from the recruiting process, or keeping testing for positions only when
necessary, such as for positions that may require physical demonstration of a
needed skill.
Strengthening Accountability
6. To better assess the efficiency and effectiveness of the recruiting function, CSA should
develop specific performance measures that coincide with those identified by best
practices. At a minimum, these measures should include time to hire, turnover rates, and
cost per hire. Other important measures and steps include probation passing rates, time
spent per recruitment and analyzing the various components of the recruiting process to
identify which parts are taking longer. CSA should then incorporate performance data
findings into management practices.
7. To ensure information for performance measures and reports is accurate and
consistent over time, CSA should take the following steps:
Ensure the methodology used to produce data is documented and can be
reproduced. Additionally, backup data behind figures reported publicly should
be retained to support such figures;
Verify all information before presenting it to the Board or reporting it to City
officials; and
Measure them consistently over time.
8. To improve personnel performance evaluation, CSA should:
Cit y and Count y of Denver
P a g e 30
Develop clear, detailed, and measurable roles and responsibilities for Recruiting
personnel. This should include Recruiters as well as HR and Testing Specialists.
Duties should be better defined for each level of Recruiter, including
expectations for how each duty should be performed and how performance will
be measured; and
Identify specific disciplinary actions to hold management accountable when
PEPRs are not completed in a timely manner.
9. To strengthen monitoring of job processes, CSA should develop quality controls, to
include:
Documentation on what should be reviewed, who should review it, at what
points in the recruiting process reviews should happen, how often it should occur
and how it will be monitored;
Monitoring compliance with policies and procedures; and
A specific quality control process for ensuring data reliability and accuracy.
10. To help analyze and interpret information about the recruiting process, CSA should
increase its use of NEOGOV‘s custom reporting tools. At a minimum, create reports for
job posting data and requisition life cycle data to ensure figures reported are accurate
and useful.
O f f ice of t he A udit or
P a g e 31
APPENDIX A
City Positions Not in the Career Service (Charter § 9.1.1 E)
The Career Service shall comprise all employees of the City and their positions except:
(i) elected officers;
(ii) members of the Mayor's cabinet;
(iii) the Director of Excise and Licenses;
(iv) up to fifty employees appointed to serve at the pleasure of the Mayor in positions
specifically designated or created by the Mayor in any department or agency of the City
under the direct control of the Mayor;
(v) county court judges and magistrates;
(vi) members of the Classified Service of the Police and Fire Departments, the Police Chief if
not a member of the Classified Service, and the Undersheriff;
(vii) attorneys and part-time employees employed by the District Attorney, other employees
of the District Attorney excluded from the Career Service and placed in an alternate merit
personnel system pursuant to state law, and up to ten employees appointed to serve at the
pleasure of the District Attorney in positions specifically designated or created by the District
Attorney in the District Attorney's office;
(viii) certified public accountants employed by the Auditor and up to five employees
appointed to serve at the pleasure of the Auditor in positions specifically designated or
created by the Auditor in the Auditor's Office;
(ix) employees of the Denver Art Museum, the Denver Museum of Nature and Science, the
Denver Zoological Gardens, and the Denver Botanical Gardens;
(x) persons retained on a contractual basis to perform professional or technical services for
limited periods of time;
(xi) employees of the City Council, Library Commission, Civil Service Commission, Board of
Adjustment, and Denver Water; and
(xii) any hearing officers and up to two employees in positions specifically designated or
created by the Career Service Board, appointed to serve at the pleasure of the Board.
(xiii) any employee appointed to serve at the pleasure of the mayor for the purpose of
monitoring internal investigations and disciplinary actions in the Department of Safety, and
any employees appointed by the monitor to serve at the pleasure of the monitor. The
appointment of any monitor by the mayor pursuant to this or any other provision of the
charter shall require confirmation by the city council.
(xiv) the Director of Elections and no more than one other employee in a position specifically
designated or created by the Clerk and Recorder, appointed to serve at the pleasure of the
Clerk and Recorder. Any employee of the Denver Election Commission as of July 16, 2007
and formerly excepted from the Career Service pursuant to this section shall retain his or her
position as an employee of the Clerk and Recorder if the employee qualifies to retain the
position in accordance with the rules of the Career Service Board.
Cit y and Count y of Denver
P a g e 32
AGENCY RESPONSE
O f f ice of t he A udit or
P a g e 33
Cit y and Count y of Denver
P a g e 34
O f f ice of t he A udit or
P a g e 35
Cit y and Count y of Denver
P a g e 36
O f f ice of t he A udit or
P a g e 37
Cit y and Count y of Denver
P a g e 38
O f f ice of t he A udit or
P a g e 39
Cit y and Count y of Denver
P a g e 40
O f f ice of t he A udit or
P a g e 41
Cit y and Count y of Denver
P a g e 42
O f f ice of t he A udit or
P a g e 43
Related docs
Other docs by blw19492
