TRIAL OF NEW GENERATION AUTOMATIC NUMBER PLATE RECOGNITION TECHNOLOGY July 2009 National Motor Vehicle Theft Reduction Council Trial of New Generation Automatic Number Plate Recognition Technology ISBN 978 1 876704 98 8 REPORT OUTLINE Date: July 2009 ISBN: 978 1 876704 98 8 TITLE: Trial of New Generation Automatic Number Plate Recognition Technology ADDRESS: National Motor Vehicle Theft Reduction Council Suite 1, 50-52 Howard Street NORTH MELBOURNE VIC 3051 E-mail: email@example.com Type of report: Evaluation Working Paper Objectives: To provide an in-field assessment of a low cost, high- performance automatic number plate recognition system developed by SenSen Networks in a vehicle crime environment. Program: Enforcement (Improving Local Police Responses) Key Milestones: Completed Abstract: In 2007 the NMVTRC formed a strategic partnership with technology developer SenSen Networks to trial a new low-cost Automatic Number Plate Recognition (ANPR) system in a vehicle crime reduction setting with the co- operation of Victoria Police. The field trials confirmed the technical proficiency of the SenSen system indicating very high levels of read accuracy compared to the currently most commonly deployed system and fewer false alarms against vehicles of interest data. By using off-the-shelf cameras, the SenSen system represents a significant financial saving over current industry offerings. Purpose: Encouraging an improved strategic police response to vehicle crime in local areas is a core objective of the NMVTRC’s work program. Key words: Automatic Number Plate Recognition, ANPR, police responses, vehicle crime. DISCLAIMER The observations contained in this report are provided in good faith and solely in the interests of promoting innovative responses to detecting and deterring vehicle crime. The NMVTRC acknowledges that the design of its trial has some limitations. Readers should take this into account in their assessment of its findings. FOREWORD Encouraging an improved strategic police response to vehicle crime in local areas is recognised as an important part of the NMVTRC’s work program. Automated Number Plate Recognition (ANPR) systems use a camera and optical character recognition software to capture an image of a vehicle, locate the number plate within the image and convert the number plate to a string of text and numerals. The text may then be compared with databases containing vehicles of interest to enforcement agencies to generate appropriate alerts for on-road interdiction or point-in-time tracking. In 2007 the NMVTRC formed a strategic partnership with technology developer SenSen Networks to trial a new low-cost Automatic Number Plate Recognition (ANPR) system in a vehicle crime reduction setting with the co- operation of police in New South Wales and Victoria. The principal claimed benefits of the SenSen system over existing technologies was that it would be much lower in cost and provide more operational flexibility. The total cost of the SenSen MANPR system is designed to deliver highest accuracies at one-third of the cost of the currently most commonly deployed ANPR systems—with the capacity to simultaneously detect cars up to 3 adjoining lanes. The field trials confirmed the technical proficiency of the SenSen system indicating very high levels of read accuracy compared to the currently most commonly deployed system and fewer false alarms against vehicles of interest data. While the wider deployment of a nationally or regionally networked ANPR capability for Australia will be determined by national and state/territory governments, the SenSen trial demonstrates the undeniable value of a low- cost, high-performance system to detecting criminal activity involving the use of a motor vehicle, including vehicle theft. BACKGROUND Forms of ANPR technology have been used in Australia since the mid-1990s. The first significant example was Safe-T-Cam developed for the New South Wales (NSW) Roads and Traffic Authority (RTA) by the CSIRO. The system was designed as a heavy vehicle compliance tool—checking speed (including average speed between camera locations), driving hours and route access control. Based on video optical character recognition the system now has an extensive network of cameras in NSW and South Australia. However, it is dependent upon expensive camera hardware and site protection (against malicious damage) and is subject to high error rates in certain environmental conditions. More recently tolling systems, which use video capture as a back up to electronic tag detection, have become increasingly common as part of new road developments in eastern Australia. However, they too are prone to relatively high error rates due to the duplication of number plate character sets between jurisdictions and their inability to successfully read the state/territory identifier. Finally, fixed and mobile ANPR technology is increasingly being used by police and transport agencies to detect unregistered and other vehicles of interest but have again been limited in their application by their generally narrow field of view and other practical considerations. Police in NSW have deployed an extensive network of cameras across Sydney for anti-terrorist purposes. In late 2006 the NMVTRC was approached by SenSen Networks—a commercialisation venture originally based at the University of Technology Sydney in relation to a new system it had developed. SenSen indicated it was looking for new ‘application spaces’ in which to refine its system which utilised off-the-shelf analogue/internet protocol cameras and processor hardware supported by sophisticated probability and statistical inference based software—dubbed by UTS as Data Fusion. SenSen claimed the principal benefits of its system over existing technology was that it offered greater operational flexibility in— using off-the-shelf CCTV cameras that function effectively under ambient street lighting with or without the need for infra-red lighting and special optics; and managing character image aspect ratios (which means that the plate will be detected anywhere in the camera’s field of view rather than having to track through a narrow ‘channel’ in order to be detected). This report provides a high level introduction to SenSen Networks’ ANPR System including its capabilities, advantages and limitations. As part of the NMVTRC trial the system was subject to extensive in-field testing via collaboration between the NMVTRC and Victoria Police1. 1 A Memorandum of Understanding between the NMVTRC and Victoria Police set out the terms and conditions of the collaboration. The NMVTRC maintained sole responsibility for the contracting and payment of any fees for services provided by SenSen. Victoria Police contributed by assigning While the SenSen system can be deployed for a number of applications, this report highlights the key features in the context of a Mobile ANPR application (MANPR)—purpose built by SenSen in response to expert feedback from experienced field officers from Victoria Police’s (VicPol) Region 3 Vehicle Theft Task Force based in Melbourne’s outer north-west. THE SENSEN SYSTEM SenSen was to develop a portable, camera-independent and easy to deploy MANPR system that would— operate from a stationary or moving vehicle using low cost off-the-shelf cameras; and detect and generate alerts for stolen, unregistered and other vehicles of interest, in real-time. The system is designed to be used with any standard, low cost, off-the-shelf cameras and can support up to two concurrent cameras on board a stationary or mobile vehicle. The SenSen System employs patented image analysis techniques combined with temporal data fusion, to provide a usable image with standard off-the- shelf cameras. As no infrared (IR) light sources are required for plate illumination, SenSen is able to provide a sound technological solution at a fraction of the price of the current ‘gold standard’ alternatives. The non- reliance on IR also enhances its use in covert applications. The key innovations and technical differences from alternative conventional products are summarised in the following table: SenSen MNAPR System Conventional ANPR System 1 Uses Patented, Temporal Data Fusion; Uses a single image frame in which the fuses information from all image frames in vehicle is visible. which vehicles are visible. 2 Enables highly accurate readings out of Needs crisp, non-noisy data where noisy data from low cost, low resolution nothing other than the license plate is cameras. visible. 3 Enables Off-the-Shelf Cameras to Provide Needs specialist, custom built automated High Levels of Intelligence. cameras that are both expensive and difficult to use. 4 Can deal with much higher levels of Can only deal with well specified, rotation and skew in plate images. predefined angles and skew. 5 Intelligence is in the software with Intelligence is in the Camera hardware flexibility to work with any camera. with tightly coupled, relatively simple software. experienced personnel to field test the system in a variety of settings and providing feedback to the NMVTRC and SenSen on how the system could be optimised for efficient operational performance. The total cost of the SenSen MANPR system is designed to deliver highest accuracies at one-third of the cost of the currently most commonly deployed ANPR systems—with the capacity to simultaneously detect cars up to 3 adjoining lanes. The SenSen system comes with simple plug and play features. An operator needs to know about only the hardware buttons/connectors and GUI (graphical user interface) features to operate the system. It comprises the following hardware and software components— 1. Hardware Camera - Off the shelf (e.g., Sony Handy Cam) or Infra-red A purpose built Computer with a DSP PCI card 10” Mini Monitor with Touch screen features Car Battery (in addition to the main car battery) Accessories such as mounts, adaptors & cables 2. Software Redhat Fedora 7; Linux Operating System SenSen MANPR Core Engine Java based Front end GUI PostgreSQL Database The hardware components may be pre-installed in the vehicle except for the camera. The user only needs to configure the camera according to his or her requirements. The software is also pre-installed. Complete details on how to operate the software are provided in the user manual. The MANPR software developed by SenSen is easy to install and highly responsive. The GUI provides user-friendly features which are very easy to operate and navigate through. Sufficient care has been taken in building the GUI features based on constant inputs from VICPOL and any additional features can be provided with quick turnaround. Figure 1: SenSen MANPR GUI The control options can be used to easily configure the SenSen MANPR system. The 'Vehicle of Interest' or 'Alert' data can be efficiently uploaded in the system. For example, a typical data comprising of 300,000 records could be easily uploaded in less than 3 minutes by means of a simple USB. This data is used for matching the License Plate Characters. The resultant alert data generated by SenSen MANPR is stored in the local data base. The search and drill-down feature is provided to access the stored vehicle and alert data. The Export feature in the GUI is provided to export or upload the 'detected alert data' in CSV & PDF format. It can be customized to any other desired format (such as CSV) to a storage device or can even be sent through an e- mail (provided the system has access to the internet). Figure 2: Exporting an Alert Report from the GUI The SenSen MANPR System uses real time video processing to detect the intelligent information from the video frames. Other system features include— 1. Secure login 2. Remote operation meaning that the system can be left unmanned and the intelligence gathered in a session can be viewed, stored and transmitted over the internet from anywhere in the world. 3. Real-time alerts of 'Vehicle of Interest'. 4. Hardware and connectors suited to withstand the typical outdoor environment and in-car conditions. 5. Low power consumption. 6. Fast processing capability that can detect registration plates of vehicles travelling at up to 200km/h. The Software has been rigorously tested for 24/7 availability. At this time, there are no known system crashes, software memory leaks or any other failure issues. The SenSen MANPR system can be easily deployed at any location or simply moved on to any other vehicle in minutes. As the system is built with off-the shelf computers, cameras and accessories ongoing management and maintenance costs are expected to be low. SYSTEM ACCURACY In field tests with Victoria Police the system proved to be highly accurate. The following comparative results are drawn from extensive testing and measurement carried out by experienced Victoria Police personnel using knowledge gained from the use of a number of commercially available alternative systems in a variety of lighting conditions and operating environments, including major arterial roads, freeways and car parks. They compare the performance of the SenSen system using a Sony Handy Cam with an industry standard IR camera. Category Feature Accuracy/False Alarms Sony REG IR Handy Cam Camera Basic License Plate characters displayed as ‘Last 95% 92% Accuracy Cars’ in the MANPR Graphical User Interface Provides Provides (GUI) – This is the best possible result obtained much wider a very by data fusion of multiple image frames field of view narrow captured by the camera. It is one of the three field of ranks of results available to the user. view Accuracy As there are two more lower ranked results 98% 95% For Search available in the database which can be queried Application* at any time, there is a high chance of finding cars of interest when queried. False Alarms Due to occasional misreads of MANPR system, 2% 4% there is a chance to get an alert that matches a misread leading to false alarms.