Non Disclosure Agreement for China by myl47151

VIEWS: 184 PAGES: 34

More Info

How China           Breakout Session
Secures its Tests    Alex TONG, ATA
 How Serious is ‘Test Cheating’ in China

 Common Fraud in testing administration in

 Underlying reasons for the popularity of
  cheating and administration ‘fraud’

 What actions has China authorities taken for
  the last few years

 Ministry of Justice Licensure exam - a case
             How serious is cheating in China?

 An online survey in June 2008 conducted to
  graduates on exam-cheating in Universities has
  alarming results:

   Have you cheated in your exams?
     Yes – 51%; With pride – 11%
     No – 49%; Would Stop others – 11%

   Agrees to ‘Cheating rather than Failing’:
     Agree – 74%;
     Disagree – 9%;

Common ‘Fraud’ in
Test administrations in China
                     Common fraud found in
                       Test Administrations


Leakage of exam items and forms

Onsite organized cheating/assistance

Test results integrity
                         Common fraud found in
                           Test Administrations
 Impersonation
     Organized proxy
     Fake ID easily
     Administrator not
      trained to verify fake
     Insignificant legal
                              Common fraud found in
                         Test Administrations – Cont.

Items/forms leakage
   Brain-dump sites acquiring/selling items/forms

   Candidates upload test items they have taken
    to share in forums

   System fraud
     • Insecure item formats (unified test with single form,
       Paper based, etc…)
     • Too many human interventions in items/forms
                         Common fraud found in
                    Test Administrations – Cont.

Onsite cheating/assistance behavior
  ‘Consulting’ neighboring seats
  ‘Bringing’ in reference materials
  Communications for external assistance
                              Common fraud found in
                         Test Administrations – Cont.

Test Results Security
   Scorer fraud – altering individual scores
   Local results – insecure storage media:
     • paper score sheet,
     • removable storage devices
   Database being hacked to alter test results
     • In 2008 Ministry of Health identified and uncovered
       a group of 200+ individuals assisted in altering the
       exam result databases
Underlying reasons for the popularity
of cheating and administration ‘fraud’
              Underlying Reasons for the Popularity
              of Cheating and Administration Fraud

 Fierce competition in job market

 High dependency of qualifications

 Too many human interventions in test
 administration process

 Cultural reasons and lacking of legal
            Underlying Reasons for the Popularity
            of Cheating and Administration Fraud

Fierce competition in job market

   Resulting from the reforms in China education
    system, there is an increasing number of
    university graduates in China year on year:
    5.7 mil (2008) to 6.2 mil in 2009
   According to the Ministry of Labor there are
    24 million people seeking for jobs in 2009
    while there will be 12 million vacancies
              Underlying Reasons for the Popularity
              of Cheating and Administration Fraud

High dependency of qualifications
  Large number of job applicants for any job;
  Tradition of certification as an entrance ticket
   for anything

Test administration process
   Unified test administration using single form
   Too many processes involving human
               Underlying Reasons for the Popularity
               of Cheating and Administration Fraud

Cultural reasons and lacking of legal
   ‘Loosing Face’ if failed, cheating as an insurance;

   ‘helping each other’ is a traditional Chinese virtue

   ‘Cost of Failing’ too high for existing practitioners

   Maintaining ‘GuanXi’ (relationship) through providing
    ‘special assistance’ i.e. fraud in administrations

   Insignificant legal consequences until late 2008
Actions taken by Chinese authorities
in the last few years
                Actions Toward Secured Testing

 Legislation – Testing Act being drafted in 2008
   Standardize testing procedures
   Disciplinary actions on cheating behavior (max. of 7
    years of imprisonment)
   Covers all aspects of testing, from test publishing,
    item security, test proctoring, test result evaluation,
    through remediation for the penalized
 Create and maintain large item banks
   The National Item Bank of Ministry of Education
   The National Item Bank of Ministry of Personnel
   The National Item Bank of MOF and MOE
             Actions Toward Secured Testing

 Leveraging technology-based measures for
  supervision, authentication and monitoring of
  test administration processes

 Minimize human factors and exposure to test

 Converting paper-based exams into computer-
  based test administration
            Technology-based Test Administration

Online-offline-online test mode

Secured Data Transfer

Secured Content Display

On-site demographic Verification
                 Security built throughout the process

128-bit symmetric encryption deployed throughout
 operation processes from pre-test, test delivery
              and post-test stages

      Pre Test             Testing              Post Test
       Online              Off-line             Online

                 Secured Test Delivery System
             Secured Data Transfer

  Form/                  Station
           Exam Starts                           Admin.


              Pre Test         Testing        Post Test
                      Secured Content Display

 Randomized item display

 Randomized sequence of choices

 Performance based items

 Single item displayed on screen at a time

                          Pre Test   Testing   Post Test
                      Onsite Demographics Creation

On site photo-identity verification

    Candidates          Onsite                     Photo-
     arriving         Photo taking              demographics


                                     Pre Test        Testing   Post Test
                    Minimize Human Factors and
                         Exposure to Test Items

Test development/production Processes

 Personnel involved in test development/production sign
  legal binding Non-Disclosure Agreement which held
  them with legal responsibilities to protect information
  they are working on;

 Separation of teams who work with test contents, no
  individuals would have access to and/or able to
  associate their works on hand with any particular test
  titles or the entire item bank

 Role-based Security Control (Access Level)
                     Minimize Human Factors and
                          Exposure to Test Items
Form Assembly

 Randomized items packaging from item bank based on
  blueprint of exam

 Standardized test item types are stored in centralized
  application servers, and will only be downloaded
  dynamically before actual test time.

 Items being downloaded from the application server will
  compose the test form ‘live’ according to the ‘blue-print’
  of the exam provided on the server
                                  Conversion of PPT to CBT

 China Securities & Regulatory Commission once-a-year paper
  based test turned to CBT in 2001.

 China Future’s practitioner Test converted to CBT in 2004

 Ministry of Education revolutionized its National IT Level Exam from
  paper based into simulation-based performance test (2006)

 China Banker’s Test opted for CBT in 2006.

 College English Tests (11 million tests/year) to be converted to CBT
  in 2009

 And more to come…
Ministry of Justice – A Case Study

                                   Ms Xia Lu,
        Chief Exam office, Ministry of Justice
        MOJ Licensure Exam - A Case Study

Background of MOJ exams

Common fraud found in MOJ exams

Actions being implemented in MOJ exams

What’s next?
                   Background of MOJ Exams

Launched in 2002 for China lawyers,
 judges and prosecutors qualifications

Administered across the country once a
 year with a unified A/B form

Extremely high stake qualification leading
 to prestige social status

 Currently 300K examinees each year
   4 exams all administered through paper forms
   Total of 7 hours of test conducted in one day
              Common fraud found in MOJ exams

Handled over 1,000 verified fraud and
 complaints in the exam administration
 since 2004

     Reference Materials/ ext. communication 42.6%
     On-site Communications during exam       36.2%
     Identical set of wrong-answers (copying) 14.5%
     Impersonation                             5.3%
     Others                                    2.3%
                                   Actions Taken

 Create and fine tune Regulations and Guidelines
  for legal actions against exam fraud (Guidelines
  and Actions for Exam Indiscipline behaviors,

 Aggravate disciplinary actions against exam fraud

 Leverage technologies to monitor, avoid and
  prevent misbehaviors in exam administrations

 Communicate and Educate public of the
  importance of a fair exam and the consequences
  of exam fraud
          A Legal Case against Exam Fraud

In Sep 2007, a group of 22 test takers
 were caught and admitted to buying a set
 of live 2007 MOJ exam questions
A MOJ officers were put behind bars for
 leaking the exam questions to his old
 friend who has a daughter who failed MOJ
 exam twice
The boyfriend of the daughter was also
 put behind bars for selling the items to 22
 examinees at price of $100K each
   Two Ministries invested 100million in building test item banks to ensure
  security in exams
  《Exam Law》 Would this Stop cheating in Exam
   Ministry of Education: National Exam item bank launched, pushing for the
  legislation of Exam Law
    University Entrance Exam: To build large national item bank to replace
  ad hoc items/forms building
    Ministry of Education built National Item bank - Creating fairness to the
  exam takers
    Ministry of Justice put three ‘cheaters’ behind bars

To top