Enhanced AIRNow Data Collection Using XML and “Internet by bns26590

VIEWS: 4 PAGES: 28

									Laboratory


  Enhanced AIRNow Data Collection
Using XML and “Internet” Technologies


                     Jeff Smith
          Air Quality Monitoring Manager
                 State of Oregon
        Department of Environmental Quality

                        &

                 Andrew C. Montz
                      Owner
                     DR DAS
    Laboratory



•    Background
•    Motivation for Change
•    Specifications/Requirements
•    Solution
•    Implementation Experiences
•    Costs
•    Future Plans
    Laboratory


Background - Old PC & Datalogger System
•   Personal Computer as central host
•   Dataloggers at all sites
•   Modem Dial-up access, pulling data
•   Not real time, data retrieved twice a day
•   Sites accessed serially, one at a time Recorder
•   DOS based, in service since 1991
•   No Web support                         Monitor


      Host PC
                                          Datalogger


                             Phone line

          Modem                                        Modem
                 Laboratory


                 Real-Time Monitoring Stations in Oregon




• 35- 45 Sites
• ~150 parameters
 Laboratory


Motivation for Change
• Wildfires
  – Big fires is 2001-2003
  – Big demand for data, internal and external
  – Cooperative monitoring projects with USFS
      • 9 station Forest Health Network
 Laboratory


Motivation for Change

• Air Quality Index
   – Hourly Updates
   – Local Media
   – Clean Air Action Days

• AirNOW Program
   – Need for Real-Time data
   – National mapping and forecasting
   – EPA Funding
 Laboratory


Motivation for Change

• Public Access to Data
   – Public Education
   – Health Advisories
   – Visible Product (tough budget times)
• Internet Widely Available
   – Means to provide the data
• Age of existing system
    Laboratory


    Specifications/Requirements
• Push data to Central Database
     – Data delivered from all sites at top of each hour
     – Send 5 minute & hourly averages, calibration
       data
     – Work in rural/remote areas
•    Work with current dataloggers
•    Use modern database (MS SQL Server)
•    Easy to get data to Web pages
•    Allow continued remote access to samplers
•    Low cost (both to purchase and operate)
 Laboratory


Solution
• Use Internet to transmit data
   – Oregon On-Line Services for ISP
   – Local access points, higher speed
• XML Pusher software from DRDAS on local PC
   –   Custom driver for Odessa dataloggers
   –   Pushes data, calibration results, IP address
   –   Synchronizes time with central server
   –   Pass-through feature to access DL directly
• ENVIEW database for central storage (DRDAS)
   – Running on MS SQL Server
   – Already available of agency server
• Use existing IT resources for Web development
   Laboratory


  Solution
• New Data System consists of 3 components
   – Remote Station
      • XML Pusher running on PC
      • Collects data from existing datalogger
      • Local access to Internet using modem
   – DMZ Computer
      • Located outside agency firewall
      • Receives XML data packets from remote systems
      • Transfers data to database server
   – Central Database Server
      • ENVIEW database running on SQL Server
      • Behind Agency Firewall
                  Laboratory

                                                                    Monitor
                  Block Diagram
                                                                 Datalogger


                                                        XML Pusher PC

                                                                                             Recorder
             Inner Firewall          Outer Firewall


                                                        Remote Station                         Modem

                                                                                          Local Phone
                                                                                          Access
                                                      Internet
                                                                                  ISP
                                                                         Oregon On-Line Services
Database Server        DMZ PC hosting
 with MS SQL            IIS & the XML
   Server &              template files
   ENVIEW
   database
    Laboratory


Remote Station

• Surplus PC running Windows 2000/XP
• IIS, MSDE, and XML for SQL Tools installed
      – IIS = Internet Info Services, MSDE = SQL Desktop Engine
•    Internal modem (56K)
•    XML Pusher software
•    Configuration Utility (web based – .asp)
•    Data display software (web based –.asp)
•    Remote control software (VNC)
•    UPS for power conditioning and interrupts
•    System configured to recover for power
     interrupt
    Laboratory

DMZ Computer

• Surplus PC running Windows 2000/XP
• IIS and XML for SQL Tools installed
• Fixed IP address, outside Agency firewall
• Data from remote stations posted to specific
  web page on DMZ PC
• Templates to map data from remote stations
  into SQL database on server inside firewall
      – Template for each type of data
      – Using stored procedures in ENVIEW database
 Laboratory


Central Database Server

• Existing Agency server running MS SQL Server
  2000
• ENVIEW database copied to server and
  attached
   – Tables in place for data, calibration results, and
     IP addresses
   – Stored procedures for data uploads
• Database user needs Read/Write/Execute
  permissions
                  Laboratory

                                                                    Monitor
                  Block Diagram
                                                                 Datalogger


                                                        XML Pusher PC

                                                                                             Recorder
             Inner Firewall          Outer Firewall


                                                        Remote Station                         Modem

                                                                                          Local Phone
                                                                                          Access
                                                      Internet
                                                                                  ISP
                                                                         Oregon On-Line Services
Database Server        DMZ PC hosting
 with MS SQL            IIS & the XML
   Server &              template files
   ENVIEW
   database
 Laboratory


Implementation - Testing New System
• 3 Phases of Testing
   – Basic system operating within Lab network
      • No dial-up, ISP
      • No firewall
   – Single test station using dial-up and ISP
      • Using outside phone line
      • DMZ PC outside agency firewall
   – Deployed at 3 real stations in Portland area
      • Real-world test
      • Provide real data for Web and software
        development
 Laboratory


Testing New System

• Basic system running on internal Lab network
   –   Software installed on remote PC
   –   DMZ PC setup
   –   Loaded ENVIEW database on local server
   –   Had little trouble getting this system to work
        •   Took a couple of software updates to fix bugs
        •   SQL Server needs to be using TCPIP protocol
        •   Issues with oldest dataloggers
        •   Timeout problems caused by power settings
   – Data transfer very quick
 Laboratory

Testing New System

• Test station running on external (ISP) network
  through Firewall
   – Installed internal modem
   – Subscribed to Internet Service Provider (ISP)
      • Using Oregon State On-Line Services
          – Local access numbers for all sites
          – Central trouble shooting and administration
   – Configured Windows Dial-up Networking
      • Using inactivity setting to end connection
   – Moved DMZ PC to Agency HQ and within
     Firewall
      • First DMZ PC implemented by Agency
      • Initial steep learning curve for IT staff
 Laboratory


Testing New System

• Had serious problems with Firewall and access
  to database server
   – DMZ PC could not talk to database server at Lab
      • Firewall would not allow connection
      • Considered getting outside help (Cisco)
   – Moved ENVIEW database to HQ server as test
      •   Connection to database now worked
      •   Blessing in disguise
      •   Hardware/software maintenance done by others
      •   Easier access to Agency Web servers
Laboratory


Testing New System
  – Still had lots of problems transferring data
     • Lots of timeouts
     • Odd pattern of failure
         – Transfer would eventually complete, but initial
           attempts would always fail
         – Discovered problem to be in “route” tables in outer
           firewall, too many entries
  – Discovered phone billing problem
     • Unexpected large phone bills
     • Older accounts set up for “metered” service
 Laboratory


Deploying systems to real sites in Network

• Test station working well
• Added more user accounts with ISP
• Installed at 3 sites for final testing
   – Cloning PCs to simplify deployment
• Space issues at most sites
   – Not arranged for a computer system
• Other considerations
   – At the mercy of phone companies and ISP
   – Central server operational schedule
       • Late night backups caused problems
       • Shutdowns for upgrades and maintenance
• Now installed at 30 sites
  Laboratory


 Software Development
Utility to transfer raw data to older system for
   processing & data management
   – New data uploaded to old system every morning
   – All editing, validation, and reporting done with old
     system
• AQI calculations and Web displays updated
  every hour from new system
• AirNOW uploads every hour from new system
• Custom data displays for agency staff
   – 5 minute data and graphs
   – Calibration results, IP addresses
 Laboratory

Data and AQI Display
 Laboratory


Costs

Item                                            Cost
Hardware
      PCs for remote sites (12 new)                    $12,000
      Flat panel displays (24)                          $6,000
      UPS, Modems, cables                               $4,800
Software - XML Pusher from DRDAS                        $7,500
Internet Access – 40 sites (per year)
         Oregon On-Line Services                        $8,000
Web Pages & Data Utility Development
      Using internal Lab and IT staff                  $32,000
                                        Total          $70,300
  Laboratory


 Communications Costs
• Comparison of phone line costs per year
   – Old system dialing twice a day, no internet
      • Total = ~$1,500
   – Old system dialing every hour
      • 1 minute per call, $0.06 per minute
      • 40 sites, 30 are long distance calls
      • Total = ~ $16,000
   – New system pushing data every hour
      •   40 sites
      •   All local calls
      •   ISP charges = $200/year
      •   Total = $8,000
  Laboratory


 Future Plans
• Retire old data system
   – Do all data management with new database
• Upgrade from dial-up to high speed internet
   – Still not available in most remote locations
   – More expensive
   – Would allow transfer of current 5 minute data
• Transfer other data from sites
   – FRM PM2.5 data records
   – Site and instrument records, photos
• Transfer data directly to AirNOW
 Laboratory


Future Plans

• Replace old dataloggers
• Serial data interfaces
• Rethink monitoring station layout
   – Centered around computer
 Laboratory


Questions


• http://www.deq.state.or.us/aq

								
To top