Docstoc

inspections

Document Sample
inspections Powered By Docstoc
					                                                                                                                 

Delegation	Assessment	of	City	of	St.	
Paul’s	Retail	Food	Inspection	Program	
                                        Executive	Summary	

Background	
In January and February of 2013 the Minnesota Department of Agriculture (MDA) deployed an 
evaluation team to assess the administrative and field components of City of St. Paul’s retail food 
inspection program.  This assessment consisted of three elements:  document review, independent 
facility inspections, and an evaluation of St. Paul inspection staff conducting inspections.  


RECOMMENDATION	
Based on this evaluation of the St. Paul retail food inspection program, the evaluation team is 
recommending that the delegation agreement with St. Paul be terminated and a transition plan for MDA 
to assume inspectional authority be implemented.   

Based on the field evaluation of St. Paul inspection staff, it became immediately evident that the 
majority of the inspection staff could not conduct a risk‐based inspection of a retail food facility.  In 
addition to this critical deficiency, the MDA evaluation team also believes that the complexity of 
specialized processing in retail food facilities along with the need for well trained and expert staff to 
conduct Hazard Analysis and Critical Control Point (HACCP) inspections is outside of the long‐term 
capacity of the St. Paul program.  The team believes that in the interests of food safety and public 
health, the city will be best served by returning the inspection of retail food firms (grocery stores, etc.) 
to MDA effective July 8, 2013. 

Evaluation	Findings	
Administrative	Evaluation	

Significant	Concerns	
    St. Paul’s electronic inspection system (AMANDA) is severely deficient and omits or misclassifies 
        many (50+) critical food code citations.   
    Furthermore, the inadequacies of the AMANDA system affect the proper initiation of 
        enforcement action.  Algorithms based on citation risk classification (noncritical, critical minor or 
        critical major) in AMANDA determine whether or not a firm receives a violation notice letter, is 
        cited for a penalty or other enforcement actions.   


Note:  This Executive Summary should be read in conjunction with all supporting evaluation 
documentation for a full understanding of the final recommendation. 
                                                                                                               
                                                                                                   Page 1 of 3 
 
                                                                                                                         
             Enforcement actions are subject to the city council review if the firm does not respond.  It is 
              then up to the council to impose the penalty and order the firm to correct it.  If the issue 
              remains, the action is taken back to the city council to suspend the license. This enforcement 
              structure has a political body determining enforcement actions and fines and represents an 
              inadequate separation of governmental powers.  
             No St. Paul inspection staff are currently standardized.  Standardization is a process by which 
              inspectors are individually evaluated during 4‐8 inspections every three years.  This objective 
              process ensures accuracy and uniformity in interpretations regarding the Minnesota Food Code 
              and is a nationally recognized practice.   
             Current facility inspection delinquent rates are 65% for High Risk facilities, 35% for Medium Risk 
              facilities and 39% for Low Risk facilities.  These delinquency rates further indicate that staffing is 
              inadequate to perform all functions of the delegation agreement.  The agency currently has 
              plans to hire 2 additional staff to which would result in 2 additional FTEs for MDA facility 
              inspections for a total of 3 FTE.  The evaluation team has concerns about the quality of staff 
              being assigned to MDA facilities since the current St. Paul inspector assigned to MDA performed 
              very poorly in the field evaluations (even though this inspector was identified as the agency’s 
              HACCP expert).  
             The agency originally stated that there are no firms with variances. However, the agency later 
              provided reports for firms with a ‘variance’. There are other firms that should have a variance 
              based on MDA approved variances. An example noted during the evaluation was a firm that is 
              holding cheeses at ambient temperature.  Due to these responses, it appears the agency does 
              not understand when a variance is required.  

Field	Evaluation	

Significant	Concerns	
    St. Paul inspection staff (5 inspectors for 11 inspections) were evaluated by MDA inspectors 
        using criteria specified by the FDA’s National Retail Food Regulatory Program Standards 
        (Standard 4)1.  Based on the Inspector Uniformity calculations, St. Paul would require a score of 
        82 percent to demonstrate competency of their overall inspection staff.   St Paul received a 
        score of 61 percent.  Several significant categories had inspector competency scores of only 20 
        percent.   
    St. Paul inspection staff could not fully identify processes occurring within an establishment 
        during an inspection and could not identify the food safety hazards occurring within those 
        processes.  They also could not successfully identify the firm’s procedures needed to control 
        these hazards.   For example, a variety of issues were noted in regards to determination of 
        active managerial control on the part of the firm.  These included assumptions regarding 

                                                            
1
     http://www.fda.gov/Food/GuidanceRegulation/RetailFoodProtection/ProgramStandards/ucm245494.htm 

Note:  This Executive Summary should be read in conjunction with all supporting evaluation 
documentation for a full understanding of the final recommendation. 
                                                                                                                     
                                                                                                         Page 2 of 3 
 
                                                                                                                      
        operator knowledge of food safety, incomplete review of Duties of the Certified Food Manager 
        including identification of hazards and training, incomplete review of Demonstration of 
        Knowledge including questions regarding employee hygiene controls and temperature controls 
        related to Potentially Hazardous Food handling.  
       The AMANDA inspection system does not allow for orders to be issued on‐site since the report is 
        entered by the inspector after the inspection is completed.   Oftentimes, St. Paul inspectors 
        were observed simply leaving the firm and not discussing their findings or observations with the 
        Person‐In‐Charge.  If inspection findings are not discussed with firms in‐person then 
        opportunities for corrective actions or education are routinely missed. 
       Since inspection reports are mailed to firms after the completion of the inspection (sometimes 
        in excess of several weeks after the inspection) St. Paul lacks the ability to gauge a firm’s 
        comprehension of the written orders and also cannot address critical issues that need urgent 
        corrective action.  For example, if something needs correction within 48 hours and the report 
        isn’t written until 3 days after the inspection, this violation is not being properly enforced.   
       St. Paul staff were repeatedly observed not writing or failing to identify critical violations during 
        field evaluations by MDA inspection staff.  For example, when a St. Paul inspector identified 
        rodent dropping and urine in contact with coffee filters during an inspection, the inspector did 
        not require the firm to immediately clean up the droppings or dispose of the coffee filters.  
        Instead, the inspector allowed the firm 7 days to correct the immediate food safety hazard.   
       Independent inspections conducted by MDA inspectors compared to recently conducted 
        inspections by St. Paul in the same firm identified significant differences in the number and 
        criticality of violations identified.  St. Paul inspection staff routinely failed to identify critical risk 
        factors that could impact public health (food adulteration, temperature control, and cross‐
        contamination issues).   
       The City lacks the ability to assess the safety of special food handling processes such as smoking 
        and curing of meats, acidification of rice for sushi, canning of acidified and low‐acid canned 
        foods and reduced oxygen packaging. 
       Of significant concern was St. Paul’s inability to properly assess the need for and evaluate 
        specialized processing occurring in firms.  These processes require variances or Hazard Analysis 
        Critical Control Point (HACCP) plans to ensure products are being safely produced and need to 
        be regularly reviewed by the inspector.   The evaluation found severe deficiencies in the ability 
        of St Paul to evaluate these processes.  These deficiencies ranged from not identifying 
        specialized processes that were occurring during the inspection, not taking adequate actions on 
        site to determine the safety of practices or food products as well as not adequately reviewing 
        HACCP materials that were available and determining operator conformance with plans. 
        Additionally, St Paul was unable to adequately communicate the MN Food Code HACCP 
        provisions to the firm.  Also, no submitted or reviewed HACCP plans were noted for any facilities 
        during the inspection evaluation process. 


Note:  This Executive Summary should be read in conjunction with all supporting evaluation 
documentation for a full understanding of the final recommendation. 
                                                                                                                    
                                                                                                        Page 3 of 3 
 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                     Rating
             Evaluation Criteria                                                                    Comments
                                                      0-2
                                                              Administrative
                                                                    Ordinance
*Has the Health Agency promulgated an                            The Minnesota Food Code (Minn. R. 4626) is adopted by reference by the City of
                                                        2
ordinance adopting the Minnesota Food Code,                      Saint Paul in ordinance Sec. 331A.07(a).
Minnesota rules chapter 4626?
Is the adopted ordinance less restrictive than                   Ordinance Sec. 331A.07(b) and (c) amend six provisions in a manner that is more
or in conflict with the provisions of the               1        restrictive than the Minnesota Food Code.
Minnesota Statutes?
                                                                      Liaison
*Does the Healthy agency have at least one                       The agency currently has two assigned liaisons. Peter Kishel is the liaison for
                                                        2
assigned liaison personnel?                                      general communication. Fong Lor is the liaison for standardization and HACCP.
*Does the Health Agency liaison attend                           The State of Minnesota provided three mandatory trainings for Delegated Agencies
mandatory training?                                              in the past three years, 2 were for general training and 1 was specifically for
                                                        1        labeling. The agency does not have training attendance records prior to 1/1/13.
                                                                 MDA records show the previous liaison, Bill Gunther, attended these mandatory
                                                                 trainings.
*Is the Health Agency Liaison contact                            The current standardization liaison, Fong Lor, was standardized, but that
standardized in the retail food establishment                    standardization expired in 2008.
                                                        0
(as delegated in this agreement, including
retail grocery and convenience store) setting?
Has the Health Agency Liaison received                           The agency does not have any training records for relevant HACCP courses for
                                                        0
HACCP training?                                                  either of the named liaisons.
Does the delegated Health Agency liaison                         The agency has no records to show that this training has occurred. However several
train their staff in MDA procedures?                    1        staff members have attended MDA mandatory training, but no staff have attended
                                                                 any of the MDA training for new inspection staff.
                                                                  Qualified Staff
Have at least 75% of the delegated Health                        The agency has no records to show that staff have attended HACCP training.
Agency Staff involved in the retail food                1        Individuals were asked to provide training documentation; of the 9 staff members
establishment inspection been trained in                         currently conducting inspections at the start of the evaluation, one staff member

                                                                                                                                                1 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                          Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
HACCP?                                                         (hired Jan 2013) has attended several HACCP and Risk Based Inspections Training
                                                               Courses, 4 had documentation of limited HACCP or Risk Based Inspections.
                                                               Training (FSP-Plus), and the remainder had no documentation of any HACCP or
                                                               Risk Based Inspections Training.
Have at least 75% of the delegated Health                      Currently, none of the staff are standardized. One staff member was previously
Agency Staff involved in the retail food              0        standardized but that standardization expired in 2008.
establishment inspection been Standardized?
Does the delegated Health Agency send staff                    Saint Paul does not have training records prior to 1/1/13. MDA training records
to Minnesota Department of Agriculture                1        demonstrate that some St. Paul staff were present during MDA mandatory trainings
training?                                                      over the past 3 years.
*Does the Health Agency have adequate                          Using the CFP Staffing Level Assessment, 1.7 FTEs are required for the number of
staffing to perform all functions of this                      facilities and inspection rates. Prior to 1/18/2013, the agency had about 1.5 FTEs
delegation agreement?                                          designated to MDA firms. Currently, the agency has only 1 inspector (1 FTE)
                                                               assigned to conduct MDA inspections. The agency currently has plans to hire 2
                                                               additional staff to which would result in 2 additional FTEs for MDA facility
                                                               inspections for a total of 3 FTE.
                                                      0
                                                               Current facility inspection delinquent rates are 65% for High Risk facilities, 35%
                                                               for Medium Risk facilities and 39% for Low Risk facilities. These delinquency
                                                               rates further indicate that staffing is inadequate to perform all functions of the
                                                               delegation agreement.

                                                               Plan review is currently staffed at a level of 0.4FTE and based on the review, is
                                                               sufficient.
Does the Health Agency employ staff capable                    All staff currently conducting inspections at the start of the evaluation are
of becoming Registered                                         Registered Sanitarians/Environmental Health Specialists. However, one staff
Sanitarians/Environmental Health Specialists          1        member has allowed his registration to lapse (expired in 2011). The agency’s
within two years?                                              position description states that a Bachelor’s Degree (including specified hours of
                                                               science credits) is a requirement of the position.
Is there a written plan for workload sharing                   The agency has a policy that describes acceptable inspectional coverage from
for vacancies in the program that exist over                   within the agency. Additionally, St Paul is a member of a Twin Cities Mutual Aid
                                                      2
thirty days? What agency will do the work?                     Agreement which provides support during emergencies, training, drills, or
                                                               exercises.



                                                                                                                                               2 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
Are staff members with REHS or RS                              The agency does not have a policy that specifies continuing education, but all
credentials given the opportunity to obtain at                 current staff members state that they are given ample opportunity to obtain CEUs.
                                                       2
least 24 contact hours of State-approved
continuing education?
Have all staff members received the training                   The agency does not have training records prior to 1/1/13. Some individuals
necessary to maintain competency in each               0       provided limited training records, however, they are not sufficient to verify
area covered by the delegation agreement?                      competency.
                                                                  Licensing
Does the Health Agency issue licenses or                        Based on city ordinances, licensing requirements are the same or similar to MDA.
permits for all delegated firms?                                However, the agency did not provide a complete list of Mobile Units. It was also
                                                       1
                                                                noted that some firms licensed for MDA should be classified as MDH facilities due
                                                                to majority of immediate food service.
Has the Health Agency provided the State                        This information is publicly available on the city web site.
                                                       2
with a copy of its fee structure for licensing?
                                                  Compliance and Enforcement Activities
*Does the Health Agency have adequate                           The regulatory agency has procedures for taking enforcement action. The
procedures in place for taking regulatory                       AMANDA electronic inspection system drives the initiation of enforcement action
action, which is comparable to State statute?                   including the generation of violation notice letters and then the initiation of an
                                                                enforcement action. There is a documented Fine Matrix that determines the fine
                                                                amount and license/suspension revocation steps. Fine amounts and enforcement
                                                                action steps are also laid out in city ordinance.

                                                                The enforcement action is then routed by the liaison to the agency licensing
                                                                department as a referral for enforcement action to be taken. The licensing
                                                       0
                                                                department works with the city attorney to execute the enforcement action. Final
                                                                fine assessments and license suspension or revocation acts are subject to review by
                                                                the city council.

                                                                Although the fine and license suspension/revocation determination is in a written
                                                                procedure, there are no additional written procedures that outline the enforcement
                                                                process. The agency does not have procedures that outline who is responsible for
                                                                reviewing and referring AMANDA notifications. Furthermore, a review of the
                                                                AMANDA system revealed multiple instances where code references were


                                                                                                                                                3 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                         Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                               incorrectly entered, incorrectly classified or not present in whole or in part. (See
                                                               Analysis of AMANDA System)

                                                               Enforcement procedures driven by an electronic inspectional system are not
                                                               sufficient if inspectors cannot make proper entry of code citations, or the code
                                                               citations entered are not appropriately classified. Classification is especially
                                                               important as it is one of the main characteristics driving the enforcement
                                                               procedures and determination.
Does the Health Agency enforce the minimum                     Ordinance Sec. 331A.07(b) and (c) amend six provisions in a manner that is more
and maximum standards found in the                             restrictive than the Minnesota Food Code.
Minnesota Food Code (Minnesota rule chapter
4626), State statutes, and State rules as                      Furthermore, the Amanda system has default language that is inconsistent with
                                                     0
referenced in the Minnesota Food Code?                         code provisions, sometimes it is more restrictive and sometimes it is less
                                                               restrictive. There are also several entries in the violation database that are
                                                               recommendations by the regulatory agency for standards that are not found in the
                                                               Food Code.
Does the Health Agency have legal services                     The agency has an attorney specifically assigned to its program, Daphne
available to it for performing enforcement           2         Lundstrom. She provides legal services for adverse actions and appeals of adverse
actions?                                                       actions.
Has the Health Agency taken regulatory                         Five enforcement histories were reviewed for firms that had prior enforcement
action against violators?                                      history over the past 5 years. Enforcement reviewed for the past 5 years showed
                                                               that 5-11 enforcement actions were taken each year. Although these numbers may
                                                               differ slightly because the data did not separate out MDA and MDH facilities and
                                                               that information had to be manually compared to current licensing lists, which may
                                                               differ from licensing during the time the enforcement action was taken.
                                                     2
                                                               While the agency has taken regulatory action, the inadequacies of the citations in
                                                               the Amanda system may result in citations not being correctly cited, consistently
                                                               cited or cited at all. These deficiencies may affect the ability of the regulatory
                                                               agency to take regulatory action against violators, or to take appropriate regulatory
                                                               action against violators.
*Does the Health Agency have misdemeanor                       The agency has misdemeanor prosecution authority comparable to MDA. Neither
                                                     2
and felony prosecution authority?                              the agency, nor MDA has felony authority.
*Does the Health Agency maintain                               The agency has records of complaints, enforcement actions and inspection records
                                                     2
documentation of complaints enforcement                        going back more than 5 years.

                                                                                                                                                  4 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
actions and inspection records for five years?

Are all variance requests from the Minnesota                    The agency originally stated that there are no firms with variances. However, the
Food Code reviewed jointly with the State?                      agency later provided reports for firms with a ‘variance’.

                                                                Examples:
                                                                    Time as a Public Health Control; the food code doesn’t require a variance
                                                                      for this, only prior notification
                                                                    A Jerky product where water activity was the control (no documentation as
                                                                      to joint review with the state)
                                                                    An acidified rice operation; the food code does not require a variance only
                                                      0               a HACCP Plan

                                                                There are other firms that should have a variance based on MDA approved
                                                                variances. An example noted during the evaluation was a firm that is held cheeses
                                                                at ambient temperature.

                                                                Due to these responses, it appears the agency does not understand when a variance
                                                                is required.

                                                                Neither MDA plan review staff nor management recall a request for consultation
                                                                on any equipment or facility issues.
                                                   Document Retention and Reporting
Are all relevant program documents retained                     Agency policy specifies a retention schedule of a minimum of 5 years. Various
for at least 5 years and available to the State       2         documents reviewed in the Amanda inspection system dated prior to 2008.
upon request?
Is a list of State Licensed and Permitted Retail                State Agency began providing establishment lists in November 2012. Prior to this,
Food Establishments provided to the State on          1         the agency was not providing necessary information annually.
a yearly basis?
Has the Health Agency provided the State                        The agency routinely submits tracking reports to MDA. Approximately 90 were
documentation of inspection activities on a                     submitted in 2012, 100 in 2011, and 140 in 2010.
                                                      2
monthly basis by completing the inspection
tracking forms provided by the state?
Does the Health Agency maintain                       0         No records were provided to document any variance requests.


                                                                                                                                                5 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
documentation of all variances, issued or
denied?
                                                               Self-Assessment
*Does the Health Agency’s management                             The agency does not conduct a self-assessment of risk factors, they stated they
conduct a self-assessment every two years? As                    work with firms one-by-one to educate them based on what they are finding or not
part of this assessment, does the Health                         finding in the field.
                                                        0
Agency maintain baseline data of risk factors
and other factors that may contribute to food
borne illness?
*Does the Health Agency have an established                      The agency does not have a program to measure the success in the reduction of risk
program to measure the success in the                   0        factors.
reduction of those factors?
                                   




                                                                                                                                               6 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                     Rating
             Evaluation Criteria                                                                      Comments
                                                      0-2

                                                                  Program
                                                               Risk Categories
Has the Health Agency defined at least 3 risk                    The agency is using the Risk Categories established by MDA currently. The use of
categories using factors described in                   2        the Risk Categories is documented in their policies.
Minnesota Statute Chapter 157.
Are 100% of firms categorized by their risk?                     While all facilities were assigned a risk category, some of those facilities were
                                                                 improperly categorized. For example, a Walgreens facility was noted as a High, a
                                                                 convenience store with prepackaged food and limited beverage service was noted
                                                                 as a High, and a facility that had limited handling of RTE foods was noted as a
                                                        1
                                                                 High.

                                                                 It was also noted that the agency is using risk categories incorrectly. Many firms
                                                                 had multiple risk categories assigned, rather than selecting the highest risk level.
*Has the delegated Health Agency performed                       The agency has not inspected 100 percent of firms denoted as high risk on an
100 percent of the high-risk inspections                         annual basis. The agency has a 35 percent inspection rate of high risk facilities
consisting of at least one routine inspection on        0        using a 365 day inspection rate. This indicates a 65% delinquency rate.
at least an annual basis as determined by
audit?
Has in the delegated Health Agency                               The agency has not inspected 90 percent of the medium risk firms within 18
performed 90 percent of the medium risk                          months. The agency has a 65 percent inspection rate of medium risk firms using an
                                                        0
inspections consisting of at least one routine                   inspection rate of 548 days. This indicates a 35% delinquency rate.
inspection in an 18-month period?
Has the delegated Health Agency performed                        The agency has not inspected 80 percent of low risk firms within 24 months. The
at least 80 percent of the low risk inspections                  agency has a 61 percent inspection rate of low risk firms using an inspection rate of
                                                        0
consisting of at least one routine inspection in                 730 days. This indicates a 39% delinquency rate.
a 24-month period?
                                                                  Inspections
*Does the Health Agency show proper                              The agency does not have a written policy that directs staff to show proper
                                                        1
credentials when entering a firm for                             credentials, however, management states they are verbally trained to do so. Field

                                                                                                                                                    7 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                         Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
conducting an inspection?                                      inspectors were noted displaying credentials at the initiation of inspections during
                                                               the majority of the inspector evaluations. Some variation was noted with staff
                                                               presenting business cards only and staff wearing credentials visibly but not
                                                               presenting them.
*Does the Health Agency provide a copy of a                    An inspection report is not left with the firm at the conclusion of the inspection; the
written inspection report to the firm at the                   report is compiled at the agency headquarters and is mailed to the firm. Agency
conclusion of an inspection, or within State                   inspectors try to send out the inspection report on the following day however there
approved timetables?                                           is an undocumented policy of within 5 days. Neither the agency nor MDA has any
                                                               documentation that this timeframe was approved.

                                                               During the field evaluation, none of the agency field inspectors were noted leaving
                                                               complete inspection reports at the conclusion of inspector evaluations. Some
                                                               inspectors left summary observation notes with the Risk Factor Checklists at the
                                                               conclusion of the inspection. The agency did not provide a policy regarding the
                                                               completion and submission of inspection reports and Risk Factor Checklists.
                                                     0
                                                               Since final inspection reports were not left with the operator in person, there was
                                                               no verification that the firm received the report or that the operator fully reviewed
                                                               the information it contained. Operator understanding of violations and corrective
                                                               actions needed is typically completed by MDA inspectors during the exit
                                                               interview. The MDA exit interview with the firm includes a review of the
                                                               complete inspection report. Signatures from the operator would then be obtained to
                                                               document receipt of the report by MDA.

                                                               Also, it took up to 2 weeks for MDA field evaluators to receive the written
                                                               inspection reports from the evaluated inspectors. Additionally, most of the
                                                               inspection reports provided to MDA, historical and in relation to inspector
                                                               evaluations, did not have signatures.
Does the Health Agency provide technically                     Incorrect Food Code citations were noted on multiple inspections reports as well as
and legally accurate information in                            incorrect risk factor checklist categories. Items that should have been critical Food
accordance to the MN Food Code on                              Code violations or Risk Factors were listed as non-critical violations or Good
Inspection Reports?                                  0         Retail Practices.

                                                               For example:
                                                                sanitation issues for food contact surfaces were reported as non-food contact


                                                                                                                                                  8 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                         Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                 surfaces,
                                                                temperature violations due to inadequate initial reheating were documented as
                                                                 improper hot holding,
                                                                washing produce in a hand sink was addressed as produce washing (GRP)
                                                                 rather than improper use of a hand sink (Risk Factor).

                                                               Additionally, the Risk Factor Checklist markings frequently did not match with the
                                                               orders that were issued in the inspection report or were incorrect. For example,
                                                               items would be marked Out on the Checklist with no associated order issues or
                                                               orders would be issued with no Out of Compliance marking on the checklist.
                                                               Items were marked as Not Observed on the Checklist when clear observations were
                                                               made during the inspection to indicate adequate compliance or not applicable
                                                               markings.

                                                               The default language in the Amanda system is at times inconsistent with the
                                                               regulatory citation.

                                                               These inconsistencies include:
                                                                    additional requirements that are not in the code language;
                                                                    language that is less restrictive than the code language;
                                                                    language that is more specific than the code language and may exclude
                                                                       violations that would be appropriate under the code language;
                                                                    citations that include language from other citations; and
                                                                    citations with missing or incorrect regulatory references.

                                                               The inconsistencies between the code language and the default system language do
                                                               not provide firms with factually correct legal and technical information. Even if
                                                               the inspector changes the default language, if they do not independently look up
                                                               the exact food code reference, they may be unaware that the system language is
                                                               inaccurate.
Do the inspection documents and interviewing                   Significant deficiencies were noted during inspector evaluations regarding proper
process reflect Active Managerial Controls in                  determination of active managerial control. Four out of five inspectors evaluated
place (item nos 1A, 1B, and 2 on form)?              0         were noted with deficiencies in this area. MDA evaluators observed inspectors not
                                                               identifying managerial control activities within the facility and lack of proper
                                                               interview techniques with operators. Some inspectors were noted making


                                                                                                                                             9 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                           Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                assumptions regarding production practices or controls within the facilities.

                                                                For example:
                                                                 a soup was noted being hot held at proper temperatures using a thermometer
                                                                   during the inspection but the inspector did not determine if the operator was
                                                                   taking temperatures to assure proper hot holding conditions or how recently the
                                                                   product had been cooked or reheated,
                                                                 the inspector reviewed supplies to the hand sink but did not determine when the
                                                                   operator required hand to be washed or when restriction of bare hand contact
                                                                   was necessary,
                                                                 inspectors were taking cold holding temperatures to determine compliance but
                                                                   questions were not asked regarding the operator performing monitoring of
                                                                   temperatures,
                                                                 dates were noted on products but no questions were asked to determine what
                                                                   those dates actually represented or when product would be discarded.

                                                                Questions regarding CFM and employee illness were noted by MDA evaluators to
                                                                be limited to posting of certificates and the presence of illness logs. Review with
                                                                managers of full illness policy requirements regarding reporting to PIC for specific
                                                                symptoms, diagnosed illnesses or customer complaints were not pursued to
                                                                determine full compliance. Knowledge of illness symptoms requiring reporting
                                                                was not reviewed with employees either.

                                                                The necessary components of duties of the CFM (identification of hazards,
                                                                monitoring, training and self-inspection) were then not determined or evaluated by
                                                                the inspectors. Duties of the PIC for oversight of processing activities and
                                                                Demonstration of Knowledge were not adequately determined due to limited
                                                                verbal interaction with operators or lack of proper open ended questioning.

Do the inspection reports contain statement of                  Inspection reports reviewed during file reviews included all required information
the violations, the corrective action necessary                 for orders that were issued. However, a number of the compliance dates observed
for compliance and set dates for compliance?                    were inappropriate based on the severity or type of violation. For example: 30
                                                       1
                                                                days were given for compliance with having and adequate supply of soap at hand
                                                                sinks and 30 days was given for compliance with cold holding violations, the same
                                                                order was written twice for the same out of compliance observation but had


                                                                                                                                                10 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                          Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                               different compliance dates .

                                                               Reports can also be very confusing to interpret due to additional remittance
                                                               information, formatting, repetition and conflicting statements. For example, an
                                                               inspection report with a compliance date of 30 days for not having a HACCP plan
                                                               for specialized process but language in the observations implied and immediate
                                                               discontinuance of operations. Orders are issued to the facility regarding in an
                                                               inspection report for remodel plans that were submitted rather than what is
                                                               observed at the time of inspection – such as “an adequate supply of hot water must
                                                               be provided” when no production is occurring. Also, it is very confusing when
                                                               there are no orders issued as the introductory statement states “the following
                                                               deficiencies were noted” and it then goes straight into definitions of the violations
                                                               types and to the signature.
Does the Health Agency perform routine                         The agency has licenses for vending machines. However, the agency has an 85%
                                                      0
vending machine inspections?                                   delinquency rate of inspections of vending machines.
Has the staff of the delegated Health Agency                   Only one member of the Agency staff has been standardized (by MDA) – that
performed satisfactorily in the necessary             0        certificate has expired. No other staff have been subsequently standardized.
standardization inspections?
Does the Health Agency utilize risk control                    No risk control plans or other long term control tools were noted in the inspector
                                                      0
plans?                                                         evaluations, facility evaluations or the file reviews.
*Does the Health Agency review plans for                       The Agency requires a plan review submission for the new construction of firms as
food establishments and inspect these                          well as significant remodels of existing firms. The plan reviewer conducts a
establishments for compliance using the                        review of the plan along with performing an onsite plan review inspection a
Minnesota Food Code requirements?                              majority of the time. The agency has one primary plan reviewer and another
                                                               employee who reviews mobiles and seasonal temporary food stands.

                                                               MDA reviewed seven plans. In the seven plans, the agency plan reviewer used the
                                                      1        food code and noted only one deficient item: one of the facilities had a foodservice
                                                               remodel and some of the deficiencies on the letter were not corrected and the
                                                               facility was granted approval.

                                                               The deficiencies were as follows:
                                                                Could not determine if the cabinets were supported by six inch legs or a hollow
                                                                 enclosed cabinet. When the manager was asked to remove the kick plates, he
                                                                 was not able to. Since the kick plates are not easily removable, these cabinets

                                                                                                                                               11 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                          Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                 would need to be considered having an enclosed hollow base which is not
                                                                 approved under 4626.0545.
                                                                Wood cutouts were not properly sealed leading to exposed wood edges which is
                                                                 a violation of 4626.0540.

                                                               In addition, there were perforated acoustic ceiling panels installed above half of the
                                                               area being remodeled. Since the area was a food preparation area, these ceiling
                                                               panels would be in violation of MN Food Code 4626.1335.
*Does the delegated Health Agency                              Significant deficiencies related to the evaluation of labeling compliance were noted
understand and inspect for food standards,                     in inspector evaluations.
labeling, misbranding and false advertising?
                                                               Some Inspectors were noted:
                                                                   not reviewing labels
                                                      0            stating that the names of products were sufficient
                                                                   not identifying the lack of manufacturer/distributor information
                                                                   not identifying undeclared allergens.

                                                               No evaluations relating to standards, misbranding or advertising were otherwise
                                                               noted during inspection and facility evaluations or file reviews.
*Does the Health Agency emphasize a                            Significant deficiencies in this area were found by MDA evaluators during the
HACCP based approach on inspections?                           inspection evaluations. The agency inspectors were not determining the complete
                                                               process flows for products being produced and were not identifying all the process
                                                               types being conducted in the facility.

                                                               For example:
                                                                an inspector was evaluating single control points in a flow such as cooking but
                                                                  did not identify that the product was cooled or determine what methods and
                                                      0
                                                                  controls were used for cooling,
                                                                an inspector was noted reviewing facility items while not addressing concerns
                                                                  related to potato salad being prepared on site during the inspection,
                                                                an inspector was noted focusing on a single specific product flow (chili) while
                                                                  ignoring other food types cooked from raw that were handled with different
                                                                  process flows (cooking poultry).

                                                               Additionally, all the risks associated with specific process points were not fully

                                                                                                                                                12 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                         Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                               evaluated when they were identified.

                                                               For example:
                                                                an inspector was noted reviewing the deli slicers for cleanliness to site while not
                                                                  determining what products were sliced there for cross contamination potential,
                                                                  if bare hand contact was restricted, the length of time the slicer was in use prior
                                                                  to cleaning, and what cleaning procedures were applied to the equipment,
                                                                an inspector was noted focusing on the preparation of raw-RTE foods in sushi
                                                                  without addressing the concerns related to the specialized processing
                                                                  requirements for the production of acidified rice.
Do the inspection documents provide accurate                   File reviews by evaluators did not identify submitted or approved HACCP plans
information relating to HACCP provisions for                   for facilities that were identified specifically for specialized processing inspections.
a specialized process?                                         Specialized processing was also noted occurring in the facilities reviewed during
                                                               facility and inspector evaluations - no associated history of HACCP plan
                                                               submission or field approval were noted for those facilities.


                                                               Specialized processing Inspector evaluations determined a significant deficiency in
                                                               the inspectors’ ability to review plan materials or provide information regarding
                                                               specialized processing.

                                                               For example:
                                                     0          HACCP materials were not requested by the inspector,
                                                                the inspector did not review the HACCP plans presented,
                                                                the inspector did not gather enough detailed information regarding the
                                                                  processing activities to determine whether it was being conducted in accordance
                                                                  with the written plan,
                                                                the inspector did not identify all the product types that were being ROP’d in
                                                                  order to evaluate if they met the restrictions,
                                                                the inspector reviewed only the top 3-4 records in a pile of approximately 50
                                                                  presented,
                                                                the inspector addressed a live molluscan shellfish tank being operated without a
                                                                  plan by having the product discarded, but dealt with the issue as operating it
                                                                  beyond display purposes only rather than requiring plans to be submitted when

                                                                                                                                                 13 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                           Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                   that was the intent of the operator due to multiple issues with non-compliance in
                                                                   the past,
                                                                 The inspector was noted not addressing the specific processing elements that
                                                                   required a HACCP plan in sushi production.
*Does the Health Agency focus it’s                              Facility evaluations, inspector evaluations and file review all noted a significant
inspections on risk factors (5) and Public                      focus on Good Retail Practice items rather than Risk Factors. The inspector
Health Interventions (4) as described in the                    evaluations also determined significant deficiencies in the inspectors’ abilities to
Minnesota Department of Agriculture and                         adequately determine compliance status of risk factors. Evaluators observed
Minnesota Department of Health’s joint                          inspectors not taking opportunities for cooking and reheating temperatures as well
manual “Minnesota Food Code                                     as observations of basic direct food handling such as service, packaging and
Standardization Procedures”.                                    preparation.

                                                                For example:
                                                                 an inspector was noted focusing a significant amount of inspection time on
                                                                   reviewing the preparation of one specific product while a variety of other
                                                                   product types were being removed from cooking equipment,
                                                                 an inspector was noted review facility and equipment conditions while not
                                                                   observing preparation of RTE trays, production of potato salad and preparation
                                                                   and packaging of bread items,
                                                       0         an inspector was noted having a conversation with the PIC while his back was
                                                                   turned to the food worker finishing bakery preparation activities, the inspector
                                                                   then continued his review of the food preparation are when activities were no
                                                                   longer occurring and did not observe food service activities that were occurring,
                                                                 an inspector was noted focusing on potential sewer pipes over stored foods
                                                                   rather than the direct food handling activities related to the specialized
                                                                   processing occurring.

                                                                Additionally, the necessary immediate correction for risk factors that are identified
                                                                as out of compliance is not consistently obtained. While some inspectors obtain
                                                                corrections on site, some do not.

                                                                For example:
                                                                 specialized processing involving the acidification and canning of vegetables but
                                                                   the inspector only requested recipes to be sent at a later time rather than
                                                                   reviewing the product condition on site or removing product from sale


                                                                                                                                                14 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                           Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                 soiled food contact surfaces that require immediate attention were not addressed
                                                                    on site – they were issued in the report that was mailed at a later date
                                                                 cold holding violations are noted but no product was discarded or moved for
                                                                    cooling and proper cold holding.
Does the Health Agency perform timely                           The file review determined that follow-up inspections are being conducted.
follow-up inspections for noncompliance of                      Follow-up inspections do not typically occur until 30 days after an initial
previously issued orders?                                       inspection. Some were noted at earlier intervals from a few days to a few weeks
                                                                for critical items such as hot holding. However, some follow-up inspections were
                                                       1
                                                                noted for review of opened bags of non-PHF such as rice. Follow-up inspections
                                                                for repeat violations were noted with additional repeat violations issued. Some
                                                                instances of multiple repeat violations during routine inspections did not result in
                                                                follow-up inspections.
*Are Field Audits conducted using a                             A Field Inspector Training/Evaluation Worksheet is available but has not yet been
prescribed number of facilities and approved           0        used.
methods?
                                         Foodborne Illness and Complaints Investigations
Does the Health Agency coordinate all food                      Agency policy specifies that they will work with MDH Epi when investigating
borne illness complaint investigations with the                 food and waterborne illness complaints. Initially, foodborne complaints are taken
                                                       2
Minnesota Department of Health                                  at the Agency and then forwarded to MDH.
epidemiologist office?
Is appropriate follow-up action taken,                          The agency provided a listing of all complaints received for 2012 however it does
including report writing, making                                not distinguish between MDA and MDH facilities. A sampling of 12 complaints
recommendations for further action and                          was selected for review – none of the complaints were foodborne illness
conducting subsequent investigations?                           complaints. The response on the complaints selected for review appeared to be
                                                                appropriate and within a satisfactory time-frame.

                                                                Documentation of complaint follow-up was noted during the file reviews – both
                                                       1
                                                                illness and non-illness complaints were noted. The adequacy of the response was
                                                                difficult to determine due to the inconsistency in the documentation for the
                                                                complaint itself as well as the actions of the inspector to fully review the operation
                                                                in relation to the issue.

                                                                For example:
                                                                 Complaint for MDH referred illness investigation included the pathogen,

                                                                                                                                                 15 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                           Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                  targeted products and had a full report regarding review as well as additional
                                                                  intervention materials and employee illness direction.
                                                                 Complaint was noted regarding a strange odor in the seafood department in a
                                                                  particular location – the report just stated that an odor was not noted. No
                                                                  documentation regarding the review of sanitation procedures or schedule was
                                                                  noted.
                                                                 Complaint was noted for specific food product but illness was only mentioned
                                                                  in relation to whether or not other illness complaints were received – not that
                                                                  the original complaint about that product was in regards to illness.
                                                                 Complaint was noted regarding lack of sinks and the operator using same sink
                                                                  for food and dishes – the reports stated the food use did not appear to be
                                                                  happening. However, every inspection report for this facility is noted with the
                                                                  statement that no food prep sink is available and only pre-processed/cut produce
                                                                  can be used.

                                                                No reference to additional or supplemental information was noted such as tracking
                                                                to how the complaint was originally received or processed, referral from MDH, etc.
*Does the Health Agency participate in                          Recently, Agency staff worked with MDA staff on an oyster related outbreak
special surveys and investigations that did not                 where the source and record keeping were of concern, glass in ice cream produced
include sampling?                                      2        at a St Paul firm as a result of an adulterated ingredient, cheese sold at special
                                                                event (from out of state suppliers), and a packaged food sold by an unlicensed MN
                                                                manufacturer at a St Paul special event.
*Has the Health Agency taken appropriate                        MDA assigned recall effectiveness check audits for several recalls dating back to
action during food emergencies and product                      2009 (Peanut Corp of America) through 2012. During one assignment in 2010, 53
recalls including recall effectiveness checks          1        audit checks were assigned, but only 44 were completed and returned to MDA.
when requested and have recalled food
products been removed from sale?
                                                  Compliance and Enforcement Activities
*Are 100% of plan reviews submitted                             The agency does not review submitted plans and notify the firm of the findings
reviewed and recorded within 30 days? Are                       within 30 days 100 percent of a time. However, the agency is reviewing plans and
the firm's notified of the review findings                      providing findings to the firm within 30 days a majority of the time. In the seven
within 30 days?                                        1        plans reviewed, three were over the 30 days by four to ten days.

                                                                Issues found that may deter the plans from being processed within 30days:
                                                                1. When the plans are submitted, they go to the project facilitator who evaluates the

                                                                                                                                               16 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                         Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                               plan to see if all necessary zoning, building, and health information is provided. If
                                                               the information was not provided, the facility provider works with the firm to get
                                                               the information in. Once the information is in or the plan is deemed to have all of
                                                               the necessary information, the plans are then sent to all the relevant departments
                                                               including health. As a result, plans may take longer than 30 days because of the
                                                               firms not providing the necessary information to complete the plans.
                                                               2. The city of St. Paul only has one plan reviewer for their MDA and MDH
                                                               facilities. Since their plan reviewer is responsible for reviewing the plans and
                                                               conducting all onsite plan review inspections, it leads to longer review times.
                                                               There may not be enough resources for plan review.
                                                               3. The plan reviewer conducts all onsite inspections. This may lead to more
                                                               dialogue between the plan reviewer and firm which reduces the importance of
                                                               getting the plan completed within thirty days since the plan is being reviewed and
                                                               requirements are being expressed through dialogue.
Does the Health Agency collaborate with the                    MDA plan review section does not recall any calls or emails from the agency in
state when necessary regarding plans which           1         regards to discussing or seeking guidance in regards to any plans.
need special consideration?
Are appropriate enforcement actions taken                      Repeat violations for orders issues were documented in the file review. Follow-up
when necessary?                                                and documentation of repeat violations was also noted during the inspector
                                                               evaluations.

                                                               In review of the enforcement records pulled, appropriate enforcement actions are
                                                               taken when the AMANDA system identifies that an enforcement action should be
                                                               administered based on the information currently in AMANDA.

                                                               However, in reviewing the AMANDA order database, multiple provisions from the
                                                     0
                                                               food code were not in the system, were misclassified as critical or noncritical,
                                                               misrepresented the rule, or did not include all the subparts of the rule. In addition,
                                                               the system classifies violations into three categories (Critical Major, Critical Minor
                                                               and Non-critical). These classifications are inconsistent with the Food Code
                                                               classifications and instances were noted when rules were classified as critical
                                                               minor for code violations that could have serious public health implications. In
                                                               order for this enforcement system to properly function, the inspector must be able
                                                               to enter the correct code and it must properly classify the code. Otherwise the
                                                               results that flow from the AMANDA system are not going to produce an


                                                                                                                                                17 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                            Minnesota Department of Agriculture Delegation Evaluation 
Delegated Agency:  City of Saint Paul 
Date: June 5, 2013 
                                                                 appropriate enforcement outcome.

                                                                 Furthermore, all enforcement actions where the firm does not correct the fine
                                                                 and/or does not come into compliance are reviewed by the City Council before a
                                                                 suspension can be issued. If the fine or correction is not resolved, the City Council
                                                                 hears the situation again prior to revoking the firm’s license. Having the decision
                                                                 making for enforcement actions rest in an elected and partisan political body is not
                                                                 sufficient to ensure that enforcement decisions are made appropriately and without
                                                                 political influence. This setup does not provide sufficient checks and balances
                                                                 between the administrative and political sections of the regulatory authority.
Does the Health Agency conduct                                   Reinspections were conducted according to the files reviewed. Reinspections do
reinspections, prepare and issue appropriate                     not typically occur until 30 days after the initial inspection. The agency did have
                                                        1
follow up letters, warning letters and notices                   record of letters issued for repeat violations.
of violation when required?
                                                            Sampling and Laboratory
*Does the Health Agency use approved                             The agency stated that they follow the directions for collecting samples given to
laboratory procedures when submitting                            them by the requesting agency. The agency does not have any laboratory sampling
samples?                                                0        procedures and does not maintain their own sampling equipment. MDA records
                                                                 demonstrated that the agency did not submit samples during the previous 5 years,
                                                                 instead MDA inspectors collected samples for 2 facilities regulated by the agency.
Does the Health Agency perform sample                            No sampling surveys have been assigned. The Agency has submitted limited
surveys in the time and manner prescribed?                       samples to the MDA laboratory:
                                                      N/A             to assess pH and water activity
                                                                      percent meat in a food item
                                                                      consumer sample for gross foreign matter.
*Does the Health Agency perform timely
                                                      N/A
follow-up of sampling data?
*Has the Health Agency provided its staff                        Sampling equipment is not available to field staff.
                                                        0
with adequate durable sampling equipment?
*Does the Health Agency follow proper                            The Agency has a sampling form that does not provide for chain of custody
procedures concerning chain of custody?                 0        documentation. The MDA form would need to be completed prior to the lab
                                                                 accepting any samples.
* Denotes High Importance 


                                                                                                                                                 18 
Rating Scale: 2 – Acceptable; 1 – Needs Improvement; 0 ‐ Unacceptable 
                                             Minnesota Department of Agriculture Field Evaluation Results                  Delegated Agency:  City of Saint Paul
                                                                                        Date: June 5, 2013
Inspector Uniformity Calculation: City of St. Paul 
Inspector ID Establishment          Date           1         2             3           4           5           6           7           8          9          10         11       12       13
                      1          1/30/2013         1         0             0           0           0           0           1           1          0          NA          1        1        0
      1
                      6          1/28/2013         0         0             0           1           0           1           1           1          0           1          1        1       NA
      2               2            2/4/2013        1         0             0           1           0           1           1           1          0           0          1        1       NA
                      3          1/28/2013         1         0             0           1           0           0           1          NA          1           1          1        0        0
      3               7            2/5/2013        1         0             0           0          NA           0           1           0          0           1          1        1       NA
                   11 ‐ SP         2/8/2013        0         0            NA          NA          NA          NA          NA          NA         NA          NA         NA       NA        0
                      4            2/5/2013        1         0             0           1           0           0           1           1          1           0          1        1        0
      4               8            2/7/2013        1         1             1           0           0           0           1           1          1           0          1        1       NA
                   10 ‐ SP       1/31/2013         1         0             0           0           0           0           1           1          1          NA          1        1        0
                      5            2/7/2013        1         1             0           1           1           0           0           1          1          NA          1        1        1
      5
                      9          1/20/2013         1         1             1           1           1           0           1           1          1          NA          1        1       NA
                    Competency Count                    9          3            2           6           2           2           9           8          6           3         10        9        1
                      Competency %                   0.82       0.27         0.20        0.55        0.22        0.20        0.90        0.89       0.60        0.50       1.00     0.90     0.17
              SP = Scheduled Specialized Processing Inspection Evaluation
                                                                                Questions 3‐12                                Calculation for Uniformity of Inspections
                                                                      Count (94 Possible)         57      Pass Rating for program with 4‐9 inspectors: 82%     
                                                                      Rating Percentage          61%      Ref: Voluntary National Retail Food Regulatory Program Standards
              *Competencies calculated within an individual category are not statistically significant since fewer than 10 inspectors were evaluated
            1 Did the inspector show proper credentials when entering the firm prior to conducting the inspection?
            2 Do the inspection documents and interviewing process reflect Active Managerial Controls in place (item #1‐3 on form)?
            3 Did the inspector determine and document the compliance status of each risk factor and intervention (i.e., IN compliance, OUT of Compliance, Not Observed, or Not Applicable) 
            4 Did the inspector complete an inspection report that is clear, legible, concise, and accurately records findings, observations, and discussion with establishment management?
            5 Did the inspector interpret and apply laws, regulations, policies and procedures correctly?
            6 Did the inspector cite the proper code provisions for CDC‐identified risk factors and Food Code interventions?
            7 Did the inspector review past inspection findings and act on repeated or unresolved violations?
            8 Did the inspector follow through with compliance and enforcement procedures in accordance with the jurisdiction’s administrative procedures?
            9 Did the inspector obtain and document on‐site corrective action for out‐of‐control risk factors at the time of inspection as appropriate to the type of violation?
              Did the inspector document that options for the long term control of risk factors were discussed with establishment managers when the same out‐of‐control risk factor occurred 
           10
              on consecutive inspections?  Options may include but are not limited to risk control plans, standard operating procedures, equipment and/or facility modification, menu 
           11 Did the inspector verify that the establishment is in the proper risk category and that the required inspection frequency is being met?
           12 Does the inspector file reports and other documents in a timely manner?
           13 Does the inspector provide accurate information relating to HACCP provisions for a specialized process?
                        Minnesota Department of Agriculture Field Evaluation Results                  Delegated Agency:  City of Saint Paul
                                                                   Date: June 5, 2013
1.Did the inspector show proper credentials when entering the firm prior to conducting the inspection?
Full credentials were shown at the beginning of 9 out of 11 inspections.  One inspection had badge worn visibly but not presented.  One inspection had business 
card only presented.
2. Do the inspection documents and interviewing process reflect Active Managerial Controls in place (item #1‐3 on form)?
Evaluators noted a lack of dialog and/or improper interviewing techniques during 8 out of 11 inspections.  A variety of issues were noted in regards to 
determination of active managerial control including assumptions regarding operator knowledge, incomplete review of Duties of the CFM including identification of 
hazards and training, incomplete review of Demonstration of Knowledge including questions regarding employee hygiene controls and temperature controls 
related to PHF handling.
3. Did the inspector determine and document the compliance status of each risk factor and intervention (i.e., IN compliance, OUT of Compliance, Not Observed, 
or Not Applicable) through observation and investigation?
All risk factors were not pursued for observations or investigation‐ for example, hand washing and preventing contamination from hands was not consistently 
reviewed, all types of products handled by facilities were not reviewed.  Observations that were identified for risk factors were not fully reviewed‐ for example, 
sanitizer concentration was taken but it was not established that food contact surfaces receive treatment, dates were identified on products but it was not 
established if the date system met the requirements of date marking rules.
4. Did the inspector complete an inspection report that is clear, legible, concise, and accurately records findings, observations, and discussion with 
establishment management?
No final reports were reviewed with the operators on site during evaluations.  Because of this, there is no clear record of the operator having received or reviewed 
the completed inspection report.  Not all significant observations noted in the inspection had orders issued in the inspection report.  Orders were noted in the 
inspection reports that were not discussed with the operators during the inspection ‐ which may cause significant confusion on the part of the operator receiving in 
inspection report.  Risk Factor form category markings often did not match up with inspection report findings, such as categories being marked out with no 
associated order issued in the inspection report.  Instances were noted where observation or remittance text in an order seemed to be in conflict with the 
compliance dates that were issued.  Majority of inspection reports provided to MDA did not contain signatures with dates from the inspector. 

5. Did the inspector interpret and apply laws, regulations, policies and procedures correctly?

Inspectors were noted not adequately addressing the following types of items: product labeling to include undeclared allergens, undeclared ingredients, 
undeclared net quantity of contents and manufacturer/distributor information, potential environmental sources of contamination, unapproved sources for meat 
products, produce preparation activities, cooling procedures, in use utensil storage, non‐food contact surface sanitation and equipment disrepair resulting in 
uncleanable surfaces.  Lack of understanding regarding the use of variances and how it related to HACCP provisions was also noted.  
6. Did the inspector cite the proper code provisions for CDC‐identified risk factors and Food Code interventions?
Code provisions for orders issues are auto populated by the agency's electronic inspection system.  However, the inspector selects the order to be issued and 
identifies the CDC Risk Factors and Interventions categories.  Multiple instances were noted where improper code provisions or CDC Risk Factor categories were 
identified in inspection reports.  In some instances, non‐critical Code provisions were used when more appropriate critical provision would apply.  For example, 
soiled food contact surfaces of equipment were listed as non‐food contact surfaces violations, repair orders were issued regarding coolers with improper 
temperatures with no order relating to improper cold holding of potentially hazardous foods, improperly used chemicals were listed as Cross Contamination Risk 
Factor issues rather than Chemical Use issues. 



                                                                                Page 2 of 5
                        Minnesota Department of Agriculture Field Evaluation Results                  Delegated Agency:  City of Saint Paul
                                                                   Date: June 5, 2013
7. Did the inspector review past inspection findings and act on repeated or unresolved violations?
Inspectors were noted with copies of previous inspections and reviewing compliance with previous findings.  Repeat violations as well as corrections were 
documented by the inspectors in the current inspection reports.
8. Did the inspector follow through with compliance and enforcement procedures in accordance with the jurisdiction’s administrative procedures?
Inspectors demonstrated a variety of compliance and enforcement competencies during the evaluations for the violations that were identified.  This included the 
voluntary discard of foods, issuance of repeat orders, issuance of warning letters and discussions regarding the progressions to fines.  One evaluation noted 
improper enforcement proceedings ‐ the firm had significant infestation issues and while photos were taken of the conditions, no enforcement action was noted to 
address the insanitary conditions.
9. Did the inspector obtain and document on‐site corrective action for out‐of‐control risk factors at the time of inspection as appropriate to the type of 
violation?
On site corrective actions conducted during the evaluations was inconsistent.  Some inspectors were noted identifying out of control risk factors and obtaining 
immediate corrective actions.  Some inspectors were noted identifying the out of control risk factors but did  not require immediate corrective actions.  Some 
inspectors were not able to identify all the out of control risk factors that were occurring during the inspection in order to take appropriate follow‐up actions.
10. Did the inspector document that options for the long term control of risk factors were discussed with establishment managers when the same out‐of‐
control risk factor occurred on consecutive inspections?  Options may include but are not limited to risk control plans, standard operating procedures, 
equipment and/or facility modification, menu modification, buyer specifications, remedial training, or HACCP Plans.
The majority of the evaluator markings in this category were determined to be Not Applicable (NA) due to lack of identification of risk factors requiring long term 
control measures on the part of the inspector.
11. Did the inspector verify that the establishment is in the proper risk category and that the required inspection frequency is being met?
All facilities where evaluations occurred were noted to be conducting high risk processing activities.  The inspectors properly documented this on Risk Factor forms.  
Some inspections frequencies were not being met, but majority of the inspectors were aware. 
12. Does the inspector file reports and other documents in a timely manner?
Majority of the inspectors fell within guidelines provided by their agency.  However, the timelines identified by the agency may not have been approved by MDA.  
Additionally, some inspection reports were not provided to MDA until 8 days or more after the inspection was conducted.
13. Does the inspector provide accurate information relating to HACCP provisions for a specialized process?
Significant deficiencies relating to communication of HACCP provisions as well as specialized processing inspections were noted during the evaluations.  This section 
applied to 6 out of 11 inspections based on activities conducted at the facilities or by communications from the inspector (2 of which were scheduled specifically for 
the evaluation of specialized processing inspections).  Adequate competency was displayed in only one of the applicable inspections.  Deficiencies included not 
identifying specialized processes that were occurring during the inspection and taking adequate actions on site to determine safety of practices or food products as 
well as not adequately review HACCP materials that were available and determining operator conformance with plans.  Additionally, no submitted or reviewed 
HACCP plans were noted for any facilities during the inspection evaluation process. 




                                                                               Page 3 of 5
  Minnesota Department of Agriculture Field Evaluation Results                  Delegated Agency:  City of Saint Paul
                                             Date: June 5, 2013

                                     FACILITY EVALUATION TRENDS
                                               Significant deficiencies were noted in regards to the delegated inspector's 
Do the inspection documents and interviewing  ability to identify compliance with risk factors ‐  most notably for CFM/PIC. 
process reflect Active Managerial Controls in  Additionally, the documentation necessary to support risk factors that 
place (item #1‐3 on form)?                     were determined to be IN compliance by the delegated inspect was 
                                               missing.
                                               One high risk facility was identified as not needing a CFM.  Another facility 
Determination of CFM status was 
                                               was identified as being IN compliance with only course completion rather 
inadequate/inaccurate
                                               than MDH Certification documentation.
                                               There was little documentation of PIC review or Demonstration of 
PIC and Demonstration of Knowledge 
                                               Knowledge performance in file reports for facility evaluations to indicate it 
documentation deficiencies
                                               had been conducted. 
                                               Identification of compliance status seemed to be correct for GRP items, 
                                               however the strength of the documentation to support the non‐
Overall GRP Determination Differences
                                               compliances was not as significant as that identified by MDA inspection 
                                               staff during facility reviews.
                                               Multiple facilities reviewed had significant labeling violations that included 
                                               undeclared allergens. 
Food labeling deficiencies for packaged foods  An issue was noted with a facility offering unpasteurized juice that was 
are consistently not identified                prepared on site as packaged product without a warning label.  It was not 
                                               able to be determined if the activity was occurring during the last 
                                               delegated inspection. 
                                               The idea of "grandfathering" in for approval or waiting until equipment 
Non‐NSF food equipment issues were identified  malfunctioned was noted.  However, in these cases equipment was noted 
but not properly addressed                     in poor condition or the facility had been licensed after the 
                                               implementation of the NSF‐equipment requirements.
Additional Pertinent Comments:
                                                   One facility was in fact currently a low risk operation due to an ownership 
                                                  change in 2011.  The facility HAD been licensed and inspection since the 
                                                  ownership change.  This same facility had significant lack of facilities 
Some risk levels were incorrect.                  including warewashing sink and hand sink necessary for use of beverage 
                                                  dispensing equipment.
                                                  One facility was in fact a medium risk facility.  They are planning to do 
                                                  additional operations in the future that would move it to a high risk 
                                                  operation, however those activities are not occurring.
                                                  Thirty‐day compliance dates were noted for issues such as temperature 
                                                  violations as well filling hand soap dispensers.
Compliance Dates are inappropriate
                                                  Compliance dates from previous years were noted in reports, such as 2005 
                                                  and 2006 in reports conducted in 2011. 
                                                  Food temperature orders were noted issued with observations of cooler 
Some orders were issued without proper 
                                                  temperatures only rather than food temperatures.  Some of these 
evidence of violations
                                                  included further regulatory action including fines issued to the facility.
                                                  Bare hand contact was not addressed in regards to non‐employees 
                                                  including reporters.
                                                  Numerous complaints regarding poor hygiene, sometimes more than one 
Complaint follow‐up inappropriate/inadequate
                                                  for a single facility, were noted.  These facilities did not always get orders 
                                                  issued and MDA inspectors then noted additional violations during the 
                                                  facility evaluations.




                                                        Page 4 of 5
Inspector ID

          t          1
                     2            Evaluator Time Per Travel Per
                     3                             14
                     4                             16         3
                     5                            7.5         3
                                                   11
                                                    7         2   TOTAL    AVG
Establishment ID                                 55.5         8     63.5   12.7
                          1
                          2
                          3
                          4
                          5
                          6
                          7
                          8
                          9
                         10
                         11


Facility Eval List
                              1
                              2
                              3
                              4
                              5
                              6
                              7
                              8
                              9

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:7/10/2013
language:English
pages:45