Chapter 7.
What’s Your Strategy? Thinking
Through the Error Reduction
Challenge
1
Chapter 7. Section 1.
Introduction
Learning Objectives
Assess, analyze and cure QC findings
Develop & implement a long-term error
resolution plan
2
Getting Ready
• It’s time to begin thinking about taking the
Summit information home.
• In the Chapters 6 & 7 we are changing the
nature of the conversation
– From: What PHAs must and must not do
– To: Short and long term strategies for
responding to the RIM reviews and
implementing an effective error reduction
program.
3
Getting Ready
Discussion
• What will be the most challenging part
of an error reduction initiative at your
PHA?
4
This Chapter
• Focuses on strategies for:
– Identifying root causes of errors
– Developing a plan for correcting and
preventing errors
• Includes a 2-part problem solving model
that will be used to think about:
– Responding to RIM reviews
– Reducing and preventing future errors
5
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for Errors
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
6
What People Often Do…
• “I have a problem. Let’s make a plan.”
– They forget the steps that come before the plan.
– They solve the wrong problem!
– Find out what the problem is before you fix it!
• As you go through the examples, you may say,
“I know what needs to be done – we need to do
training.” But let’s go through the steps to see.
7
RIM Reminder
Within 45 days:
– Correct individual tenant file errors identified by the
RIM
– Corrective action plan for any errors not resolved in
45 days
• Within 6 months:
– Implement corrective actions to resolve systemic
findings and prevent future errors
– Corrective action plans that exceed six months must
be approved by HUD Headquarters.
8
Chapter 7. Section 2.
Problem Analysis Strategy
9
Section 2. Problem Analysis
Strategy
• Our strategy starts with “problem analysis”
• But don’t we already know what’s wrong?
– HUD’s RIM reviews may have identified some
problems
– But … knowing what to do about the problems
requires a deeper understanding of exactly
what happened and why.
10
Problem Analysis
Step 1: Clarify Area of
Concern
Step 2: Understand What
Problem Really Happened
Analysis
Step 3: Identify Reasons
for Errors
11
Step 1: Clarify the Area of
Concern
• Work with one issue at a time
– We’ll use Child Care as a running example
• Use as many sources of information as
possible
– RIM review findings
– Quality control results
– Complaints (internal/external)
• After an assessment of each issue, think
about how they fit together
12
Clarify the Area of
Concern
• Identify the requirements: Is it clear how
each issue should have been handled?
• To be sure, review:
– HUD and PHA policies and procedures
– Forms, checklists and other tools
• Ask yourself:
– Are policies and procedures clear?
– Do they ensure consistent handling of files?
– Are any situations not covered?
13
Concern: Child Care
Page 7-5. The RIM Reviewer found that:
• Staff didn’t follow PHA policy for determining
which family member is enabled to work
• Staff did not ask if child care costs were paid or
reimbursed by a third party
• 3rd party verifications were missing with no
explanation
• No documentation of the “reasonableness” of
the child care expenses
• Ineligible expenses counted as child care
14
Concern: Child Care
Page 7-5 (continued)
• Where would we find what should have
happened?
– HUD regulations [24 CFR 5.603 and 5.611]
– PHA’s Admission and Occupancy Policy
– Public Housing Occupancy Guidebook
– Notice PIH 2001-15 (HA) [expired]
– PHA forms [Interview form, third-party
verification form]
15
Concern: Child Care
• We have clarified the area of concern. We
know:
– What the RIM Reviewer (or our own QC
process) tells us happened.
– What should have been done according to
HUD and PHA policies.
• Next step: Understand what really
happened.
16
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for Errors
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
17
Step 2: What Really Happened?
• Begin to assess the underlying problem
– Who made the errors (everyone, several
people, one person)?
– How frequently were errors made?
– What kinds of errors were made?
– Did a special situation cause the error?
• Start with your experience but don’t jump
to conclusions
18
What Really Happened?
• To understand the underlying problem you
may decide additional data collection is
needed.
• If so:
– Pick files processed by different specialists
– Select enough files to pick up on problems
and see trends
– Choose cases that are likely to have the type
of error you are analyzing
19
What Really Happened?
• Talk with people
– Consider the advantages of individual vs.
group conversations
– Consider how to get diverse perspectives.
Variety is the key!
• Best/worst performers?
• Oldest/newest staff?
• Members of different teams?
– Talk to enough people, but not too many
20
What Really Happened?
• Think about how you ask the questions.
– Can you walk me through this part of the
process? (Not: Why did you …)
– What is the hardest or most confusing part?
– What do you do if you have a question? (Not:
Why don’t you know …)
– What would help you do this more easily/more
accurately.
21
What Happened: Child Care
Page 7-8.
• Staff automatically used the spouse’s
income to cap child care.
– PHA policy says use family member with
lowest income unless information shows
someone else is enabled to work
• Staff didn’t ask about reimbursement from
other sources.
– Some staff said they didn’t know to do it;
others said they knew, but “forgot” to ask
22
What Happened: Child Care
Page 7-8 (continued)
• Missing third-party verifications
– Staff reported that third-party providers often
don’t cooperate, so they used other
verification method to get work done quickly.
• Missing documentation of reasonableness
– Staff are confused about how they are to
handle “reasonableness”
23
What Happened: Child Care
Page 7-8 (continued)
• Ineligible expenses allowed for a 15-year old
child with a disability
– The staff member knows the rule.
– A review of her other cases found no other
errors. She just made a mistake.
• Now we understand what happened. Next:
Why did it happen?
24
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for
Errors
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
25
Step 3: Identify Reasons for
Errors
• To understand why:
– Reflect on what you know from looking at files
and interviewing staff
– Remember, the stated reasons may not get to
the root problem
26
Reasons for Errors:
• Keep asking “why”? For example:
– Staff didn’t know how
– Why didn’t they know?
• No one told them – why didn’t someone tell
them?
• Someone told them – but they
misunderstood? Why did they
misunderstand?
• They knew it but forgot – why did they
forget?
27
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for
Errors: Stated and Root
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
28
Reasons for Errors
(See Chart on p. 7-10)
Stated Reason Root Cause
• Staff “didn’t know” • Policy/Procedures?
WHAT to do • Lack of Knowledge
• Inadequate supervision?
• Inadequate training?
• Staff didn’t know • Knowledge/training?
HOW • Skill or ability?
• Clear procedures?
• Inadequate supervision?
29
Reasons for Errors
Stated Reason Root Cause
• Staff COULDN’T • Inadequate infrastructure
• Excessive workload
• Forms / other tools
• Computer capacity?
• Staff WON’T/DON’T
– Always done it • Organizational Culture
– Didn’t know why it was • Inadequate supervision
important or didn’t
• Lack of clear procedures
believe it was
important
30
Reasons for Errors
Page 7-11
• Common Root Causes of Errors
– Inadequate Written Tools
– Inadequate Computer Hardware and Software
– Inadequate Communication
– Limited Staff Capacity
– Inadequate Facilities
– Inadequate Supervision
– Non-productive Organizational Culture
– Policies and procedures
31
Reasons: Child Care
Page 7-12
• Capping the deduction
– Stated Reason: Staff Confused
– Root Cause: Newer staff don’t have
knowledge and skills to make the decision
• Checking for costs were paid by others
– Stated Reason: “We forgot, doesn’t happen
often
– Root Cause: Interview form doesn’t have a
question
32
Reasons: Child Care
Page 7-12 (continued)
• Missing 3rd party verifications
– Stated reason: Small providers don’t respond
so we move on to other forms
– Root cause:
• No place to explain why 3rd party wasn’t used
• Staff are focused more on timeliness than
compliance.
• Verification form may be too complicated for small
providers.
33
Reasons: Child Care
Page 7-12 (continued)
• Staff didn’t document reasonableness
– Stated Reason: “We’re confused”
– Root Cause: Staff don’t have knowledge and
skills to make this decision
• Counted expenses for 15 year old
disabled child
– Stated Reason: “I forgot”
– Root Cause: Carelessness, isolated incident
34
Problem Analysis Review
• Before trying to solve a problem, have
you?
– Clarified the area of concern?
– Determined what really happened?
– Identified reasons for the errors?
• If yes, it’s time to develop a strategy.
35
Chapter 7. Section 3.
Response Strategy
36
Section 3. Response
Strategy
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
37
Section 3. Response
Strategy
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
38
Step 1: Find Appropriate
Solution
• Don’t jump to conclusions about what to
do.
– Do you have a personal preference or bias
about addressing errors?
• Sometimes a problem needs to be tackled
from more than one angle
– For example, you may need to remind staff
about rules and improve policies and improve
forms)
39
Find Appropriate Solutions
• Timing can be a barrier.. Think about
solutions for the:
– Short-term.
– Medium-term
– Long-term
• Doable vs. deluxe. The “ideal” solution is
not always possible. Consider:
– A quick fix, followed by a long term solution
– How much effort is really needed
40
Solutions: Child Care
Page 7-17.
• Capping the deduction
– Root cause: New staff (especially) need information
– Solutions: Case studies/discussion
• Checking for costs paid by others
– Root Cause: Interview form didn’t have a question
– Solution: Revise the form
• Missing 3rd party verification
– Root Cause: Combination of procedure and attitude
– Solutions: Fix form; enhance procedure; remind staff
41
Solutions: Child Care
• Documenting Reasonableness
– Root Cause: Staff didn’t know how
– Solutions: Case studies and discussion
– What else?
• PHA counted expenses for 15 year old
disabled child
– Root Cause: Carelessness, isolated incident
– Solution: Fix the file, no further action
42
Coordinating Solutions
• Few PHAs have only one type of error.
• Even though each issue is analyzed separately,
solutions should be coordinated.
• Set priorities that focus on errors with large
financial impact or that happen frequently.
• Be realistic about resources required.
• Look for efficiencies such as training on several
error topics at the same time.
• Adjust timeframes to avoid overloading a few
staff.
43
Section 3. Response
Strategy
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
44
Step 2. Develop An
Implementation Plan
• Why an implementation plan?
– Understanding is not enough, action must
follow
– Writing down a plan increases the chances it
will be successful
• Tips for developing the plan
– Identify key players and bring everyone on
board
– Remain flexible. Be ready to revise based
upon the suggestions of other players.
45
Implementation Plan
• More tips
– Get specific and detailed. Now is the time to:
• List all of the steps
• Think strategically about who can best
carry out each task
– Assign responsibilities and deadlines.
• Designate staff with appropriate interest
and skills.
• Consolidate the individual plans to identify
overloaded individuals and confirm
priorities.
46
Implementation Plan
• Page 7-20 – Child Care Example
– Column 1: What we decided to do in general
terms
– Column 2: Who needs to take what specific
actions
– Columns 3-6: When will this be done?
47
Implementation Plan
• Page 7-21 – A summary plan that:
– Addresses errors related to Child Care
– Illustrates the results of other analyses.
• Page 7-22 - Summary of the Problem
Solving Process including:
– What to ask
– What to do
48
Chapter 7. Section 4.
The Error Reduction Process
Two Examples:
Best Housing Authority (BHA)
Quality Housing Authority (QHA)
49
Introduction
• Purpose of the two examples
– More illustrations of the problem solving
process
– Consider solutions for additional types of
errors
• We will give overview now – you can study
the details when you go home
50
Overview of the Process
Problem Solution
51
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for Errors
Find Appropriate solutions
Response
Strategy
Develop an
Implementation Plan
52
Clarify the Concern
• The Scenario: HUD conducted RIM reviews at
both PHAs and found identical errors:
– 3rd party verifications are not being
consistently obtained.
– No explanation provided for why another form
of verification was used
– Inadequate documentation of differences
between file data and HUD-50058 data
53
Best Housing Authority
(BHA)
Problem
Analysis
Strategy
54
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for
Errors
55
BHA Problem Analysis Steps
• Step 1: Clarify the areas of concern
– We did that (same for both PHAs)
• Step 2: Understand what really happened
– What did really happen? …
56
What Really Happened -
BHA
• File reviews confirmed the problem
• Errors were concentrated among new staff
(more than 50% on the job less than 2 years)
• Staff reported they were complying when
possible
• Staff confused about what 3rd party means (e.g.,
“Don’t payroll stubs come from a 3rd party?”)
• Staff indicated there were reasons for making
changes to the family’s final income but there is
no place to document this.
57
Step 3: Identify the Reasons
for Errors
Page 7-26 (review chart)
• Column 2 – Stated Reason for Error
• Problem 1: No 3rd party verification in file
– Some staff didn’t know the paper work they
were accepting was not 3rd party
– 50% of staff are new and lack of experience
affected the error rate
58
Reasons for Errors - BHA
Page 7-26
• Column 3 – Possible Root Causes
• Problem 1:
– Inadequate policies and procedures
– Staff lack skills
– Supervisors aren’t catching errors.
59
Step 3: Identify the Reasons
• Problem 2: Lack of 3rd party documentation
• Stated Reason: Staff said they didn’t know
where to put information about why they took
a lesser form of verification
• Possible Cause: Inadequate forms – no
establish method for recording the decisions
to take something less than third party
written verification
60
Step 3: Identify the Reasons
• Problem 3: Lack of change documentation
• Stated Reason: Staff did not understand
the importance of documenting changes in
income
• Possible Cause:
– Inadequate forms
– Inadequate skills
– Inadequate supervision
61
Best Housing Authority
(BHA)
Response
Strategy
62
Approach to Error Reduction
Find Appropriate
Response solutions
Strategy
Develop an
Implementation Plan
63
Response Strategy – BHA
Page 7-28 Possible Solutions to problems
(look at column 3)
• BHA will focus on:
– Improving policies, procedures, forms
– Skills of staff
– Effectiveness of supervision
64
Develop Implementation
Plan
• Possible Solutions
– Need Action Plan to implement solution
• Need Strategy to carry it out
– Who needs to do what by when
65
Develop Implementation
Plan
• Review policies and procedures to ensure
that 3rd party requirements are clear
– Who’s going to do it
– When is it going to be done?
• Review chart on pages 7-29 and 7-30
– Time frames ranging from “Now” to greater
than six months
66
Quality Housing Authority
(QHA)
Problem
Analysis
Strategy
67
Approach to Error Reduction
Clarify Area of Concern
Understand What Really
Problem Happened
Analysis
Identify Reasons for
Errors
68
Step 1: Clarify the Area of
Concern
• Were they the same as BHA?
• Will the causes, solutions, and
implementation plan be the same, do you
think?
69
Step 2. Understand What
Really Happened
• File reviews confirmed the problem
• Team A staff report supervisor says no
point wasting time if 3rd party won’t
respond
• Staff think the PHA’s priority is on-time
reexaminations.
• Since they were following policy, they
believed no documentation was needed
70
Step 2. Understand What
Really Happened
• Verification form is confusing to employers
• Fax machines are not working well making
it difficult for employers to return
verifications.
• Computer software doesn’t provide a
place for notes to explain why 3rd party
wasn’t used
71
Step 3. Identify Reasons
• See chart on page 7-33
72
Quality Housing Authority
(QHA)
Response
Strategy
73
Approach to Error Reduction
Find Appropriate
Response solutions
Strategy
Develop an
Implementation Plan
74
Reason for the Errors -
QHA
Page 7-33
• Column 2 – Stated Reason
– Supervisors communicated that on-time
reexams are more important than verifications
– Employers have little incentive to cooperate
and complain verification form is unclear
– FAX verifications not effective
– Staff didn’t understand importance of
documenting changes in income .
75
Reason for the Errors –
QHA (continued)
Page 7-33
• Column 3 – Possible Root Causes
– Inadequate supervision – some supervisors
have given faulty information
– Organizational culture – emphasis on
timeliness at the expense of accuracy
– Inadequate training of staff
– Inadequate tools
– Inadequate facilities (poorly maintained faxes)
76
Response Strategy - QHA
Page 7-36 – Possible Solutions
• Initiate a long-term culture shift focused on
balancing timeliness with error-free processing
• Education for staff
• Input from employers re: forms and process
• Develop forms and procedures for documenting
use of other than 3rd party and changes to
income.
77
Implementation Plan
• Strategy for who is going to do what by
when
• Review chart on page 7-36 and 7-37
78
Chapter 7. Section 5.
Using the Error Reduction
Process
79
Section 5. Using the Error
Reduction Process
• Back Home. How to start:
– Break the problems into manageable steps
– Go step by step
– Keep an open mind
– Think broadly about causes and solutions
– Resource limitations may require implementation in
several stages.
– RIM findings take priority, but don’t stop there.
– Start (enhance) your QC program as soon as
possible
80
Exercise: Where Do We
Go From Here?
1. Identify your PHA’s 2 largest error issues
2. Identify the two most likely root causes of
these errors
3. Identify one possible solution for each
root cause.
4. What are the two most important
messages you will be taking home from
the summit to supervisors and staff in
your agency?
81
Learning Objectives
• Assess, analyze and cure quality control
findings
• Develop and implement a long-term error
resolution plan
82
What Have We Learned?
83
Can you now . . .
• Develop systems, standards and practices
that will reduce income and rent errors?
• Achieve successful results in future RIM
reviews by improving the PHA’s
understanding of the RIM process?
• Develop well-conceived policies and
effective procedures, taking into account
mandatory requirements and prohibitions
vs. areas of policy discretion?
84
Can you now . . .
• Gather, analyze and document information
through improved interviewing and
analysis skill-development?
• Apply HUD requirements and standards
for effective verification techniques?
• Understand and address difficult aspects
of rent calculation where errors are most
likely to occur?
85
Can you now . . .
• Create and maintain a quality control
system to reduce rent calculation errors
• Assess, analyze and cure quality control
findings
• Develop and implement a long-term error
resolution plan
86
87
Assessment
• Here’s how it works . . .
– 1. You have 2 Scantron sheets, both to be
filled out, plus an assessment with questions
• One to take home
• One to turn in
– 2. When finished, you will turn in the
assessment with your Scantron sheet
88
Assessment
• You have until 11:15 to complete it
• In about 2 weeks, you can log onto the same
website you registered on
(www.nmauniversity/rhiip) and get the answers
for the test – then compare to your Scantron
sheet
• VERY IMPT! Each test will be listed by the
Summit you attended and the type of program
covered (HCV or Public Housing)
89
Session
• Need session close-out PPTs here.
• Trainers – need ideas
90