Emerging networked sensing and actuation technologies end-to-end

Document Sample
Emerging networked sensing and actuation technologies end-to-end Powered By Docstoc
					       Sensing and Actuation:
    End-to-end systems design for
      safety critical applications

                   Dr. Elena Gaura, Reader in Pervasive Computing
                Director of Cogent Computing Applied Research Centre,
                                  Coventry University,

           Dr. James Brusey, Senior Lecturer, j.brusey@coventry.ac.uk

Gaura, Brusey
Cogent Staff and PhD students

                                                                                         Tessa Daniel
                                                                                         danielt@coventry.ac.uk    Tony Mo                  John Kemp
Dr Elena Gaura                                                       Michael Richards    Expertise:
                                  Dr James Brusey                                                                  tony.mo@coventry.        kempj@coventry.ac.uk
e.gaura@coventry.ac.uk                                               richardsm@coventr   Applicative Query
                                  j.brusey@coventry.ac.uk                                                          ac.uk                    Expertise:
Expertise:                                                           y.ac.uk             Mechanisms; Information
                                  Expertise:                                                                       Expertise:               Advanced Sensing;
Advanced Sensing; Advanced                                           Expertise:          Extraction in Wireless
                                  Industrial Robotics and                                                          Wireless sensing for     Sensing Visualisation
Measurement Systems;                                                 3D CFD Modelling    Sensor Networks.
                                  Automation; Machine                                                              gas turbine engines      Systems.
Ambient Intelligence; Design
                                  Learning; RFID; Sensing
and Deployment of Wireless
                                  Visualisation Systems.
Sensor Networks; Distributed
Embedded Sensing; Intelligent
Sensors; Mapping Services for
Wireless Sensor Networks;
MEMS Sensors

                                                                     Costa Mtagbe        Mike Allen                Ramona Rednic            Dan Goldsmith
                                                                                         allenm@coventry.ac.uk     rednicr@coventry.ac.uk   goldsmitd@coventry
                                                                     Expertise:          Expertise:                Expertise:               .ac.uk
                                                                     Environmental       Design and Deployment     Body sensor networks,    Expertise:
 Dr. Fotis Liarokapis                                                                    of Wireless Sensor
 f.liarokapis@coventry.ac.uk       Dr. James Shuttleworth            monitoring                                    Posture                  Middleware design
                                   j.shuttleworth@coventry.ac.uk                         Networks; Distributed
 Expertise:                                                                                                                                 and test-beds for
                                   Expertise:                                            Embedded Sensing.
 Mixed reality systems; mobile                                                                                                              WSNs
 computing, virtual reality for    3D Graphics; data fusion and
 entertainment and education       feature extraction, information

       Gaura, Brusey
                Talk Scope

                    • development cycle for a multi-
                      modal wearable instrument
                    • system design decisions
                    • embedding actuation and its
                    • hurdles encountered….

Gaura, Brusey
• Timeliness: BSNs and WSNs are becoming
  commercial in their simpler forms; also coming
  out of research labs in elaborate versions;
- Task Difficulty: Designing such systems needs
  teams of applications specialists, electronics
  engineers (most often) and definitely Computer
- Usefulness: proven, but, apart from being very
  useful, BSNs are a lot of fun to develop!

Gaura, Brusey
                Talk Structure
• Part 1: Introduction and overview of the
• Part 2 : The deployment environment - a
  physiological perspective
• Part 3 : System design
• Part 4 : Enabling actuation - on-body processing
• Part 5 : Implementation - software and hardware
• Part 6: Results analysis and evaluation
Gaura, Brusey
                Part 1: Introduction
                and overview of the

Gaura, Brusey
WSNs: research motivation
Start point:
-Smart Dust (1998) – Pister
 ($35,000) vision of “millions of tiny wireless
sensors (motes) which would fit on the head of a

-sharing “intelligent” systems features (self –x)
pushed to XLscale – millions of synchronized,
networked, collaborative components

-Dust Networks - $30 mil venture (2006);
-TinyOS – the choice for 10000 developers
-make the news and popular press
- fashion accessory & easy lobbying
- big spenders have committed already (BP,
Honeywell, IBM, HP)                                  Attention!
-technologies matured (digital, wireless, sensors)   Your spatio-temporal
-first working prototypes;                           activities are recoded
-getting towards “out of the lab”                    and analyzed by the
    Gaura, Brusey                                    20000 sensors wide
-social scientists are getting ready!
                                                     campus net
                   WSNs –reality
Market forecast:
2014- $50bil. , $7bil in 2010 (2004)
2014- $5-7 bil. sales (conservative)      Infineon tyre sensor
2011-$1.6 bil. smart metering/ demand response

Industrial Markets-    old and new; mostly wired
   replacements; generally continuous monitoring         Connecting 466 foil strain gages to a
   systems with “data-made-easy” features and internet   wing box
Prompted by regulations and drive towards process                        Invensys asked a
   efficiency or else…                                                   Nabisco executive
                                                                         what was the most
the “cement motes” from Xsilogy come with 30 min
    warranty!                                                            important thing he
                                                                         wanted to know. The
                                                                         reply came without a
Research: mainly newly enabled                                           moment's delay: "I'd
applications; “macroscopes”/                                             like to know the
                                                                         moisture content at the
“microscopes” ; adventurous money
                                                                         centre of the cookie
savings ideas
    Gaura, Brusey            ISWC, Pittsburgh, 01/10/2008                when it reaches the
                                                                         middle of the oven."
            WSNs - pushing the frontiers
             The motivational square
                          …forget about throwing
Practical, application       them from the back of             Visions
                             that plane!...
oriented research and
                            Making the most
                            out of a bad
                                                     Research space

 Commercial                      Research space
                         Research/Adoption roadblocks

Internet able
Microclimate, soil
moisture, disease                                             Largest part of community
                                                               Theoretical research for
  Industrial needs                                               large scale networks
      Gaura, Brusey
    Why is it all so hard?
…the WSN design space (Ray Komer, ETH, 2004)
   cost, size, resources and energy   Highly theoretical
   communications modality
   network topology
   network size                       deployments
   other QoS requirements

Gaura, Brusey
            WSN challenges

• Application specific (deployment, size, weight,
• System specific – the network is the SENSOR
     –   Distributed processing- system infrastructure
     –   Information extraction
     –   Scalability
     –   Robustness
• Node specific – hardware

Gaura, Brusey
    WSN – challenges
•  Physical environment is dynamic and unpredictable (Hw&Sw)
•  Small wireless nodes have stringent energy, storage, communication
   constraints (Hw mainly)
In-network processing of data close to sensor source provides (Sw, systems
    – Scalability for densely deployed sensors
    – Low-latency for in situ triggering and adaptation

•   Embedded nodes collaborate to report interesting spatio-temporal events
    (Sytems design)

     Embeddable       Portable Adaptive
     Low cost         Robust    Self healing
     Self configuring      Globally query-able
Gaura, Brusey
         Application related
• User requirements definition – novel
   technology hence this is hard
• Capability/expectations mitigation
• Lack of comparison measure at end-to-
   end systems level
Don’t underestimate the role of cyclic
Gaura, Brusey
 Data acquisition phase
• Sensors availability – MEMS technologies are
  just maturing - many physical sensors available
• Digital or analogue output - Digitization required
• Sensors compatibility with other systems
  FAULTS- Mostly uncalibrated, but…very cheap
• Integration sometimes a problem

Gaura, Brusey
   Processing and comms
• Nodes size, weight, energy resources and
  processing capabilities – contrary
  constrains which need mitigating
• Unreliability of wireless communications
• Lack of debugging tools and wireless
  technology immaturity
• Off-the-shelf comms encapsulation;
  unlexible protocols
• Processing with little on much data
Gaura, Brusey
         Processors and Motes
                Sensor device   Processor       Memory   Communications        Form factor
Mote                interface

Renee           Mezzanine       Atmel 8 bit 4   49 kB    916MHz,    software   484 mm2
                   card            MHz                      modulation         rectangle

Mica 2          Mezzanine       Atmel 8 bit 8   644 kB   916/433MHz            1800 mm2
                   card (4         MHz                   hardware modulation   rectangle
                   sensors)‫‏‬                             19.2 kbps
Mica2Dot        Single sensor   Atmel 8 bit 4   644 kB   916/433MHz            255 mm2
                Analog             MHz                   hardware modulation   disc
MicaZ           Mezzanine       Atmel 8 bit 8   644 kB   2.4GHz                1800 mm2
                   card (4         MHz                   ZigBee                rectangle
Intel mote      Digital         ARM 32-bit      586kB    2.4GHz                900
                    interface      12 MHz                Bluetooth             mm2

Gaura, Brusey
     Information extraction
•   Timeliness of acquired data
•   Time synchronization
•   Data storage
•   Information extraction at source
•   Co-opertive behaviour
•   Global vs local treatment of the challenge
•   Mitigating energy vs quality/detail vs
    timeliness vs system cost, size, etc
Gaura, Brusey
        Information delivery
• Raw data is too much saying too little
• Huge range of user requirements motivated by –
  conservativeness of some engineering fields
  (ref- Energy sector, aerospace, defence)
• Ease of interpretation by human in the loop –
  hard to accommodate with limited resources
• Range of useful options continuously growing

Gaura, Brusey
       Actuation enablers
• Are still in its infancy
• Much to be gained from any
  breakthroughs here

   Enabling actuation has serious
   consequences in the overall system

Gaura, Brusey
          User satisfaction
• Usually unknown/unpredictable till the
  development ends
• Trail and error as the favourite methods
• Huge range of reported work which failed
  to satisfy for all possible resons
• Unreliability of the put-together systems is
  damaging to the filed
Gaura, Brusey
       The Grand WSN challenge
Facilitating the migration of pervasive sensing from
  future potential to present success
                                     Design space

                                     •Care for the un-expert user –
                          “The       “beyond data collection systems”
                          network is
 VLS networks as          the sensor” •Robustness, fault tolerance
 Scientific instruments
                                     •Long life – across system layers
  Permanent monitoring               and system components- in
 fixtures                            network processing &distribution

                                     •Maintenance free systems –
                                     scalability, remote programming
                                     &generic components/
   Gaura, Brusey
   Software - design features
• designing for information visualization

• designing for robustness and long life - Fault Detection
  and management

• designing for practical applications

• designing for robust services support

• designing for information extraction- Complex Querying

Gaura, Brusey
     Designing for practical
                                                      The problems:
                End-to-end system
BSN             design approach
                                                      •Robustness of deployment
                                                      •Technologies Integration
                                                      •Fitness for purpose
                                                      •Non-experts will use it!!!

Gaura, Brusey          ISWC, Pittsburgh, 01/10/2008
Matching application requirements
 with available technology in a
    safety critical application

Gaura, Brusey
                Project history
• Commissioned late 2005
• Externally funded
• Client: NP Aerospace Plc - protective
  clothing manufacturer for Defence - mostly
  for bomb disposal missions, de-mining, etc
• PhD student project

Gaura, Brusey
           Project aim: Increased
             safety of missions
              through remote

Gaura, Brusey
   The problem: the suit
• Increased heat production and
  reduced ability to remove heat
  results in storage
• Thermoregulatory system
  becomes unable to correctly
  regulate core temperature
• This may result in physical and
  psychological impairment
• Increased risk of making an
  avoidable error and jeopardising
  the mission
Gaura, Brusey
                 Possible solutions
Manufacturer solution: add a cooling system to the suit
a) Inefficient use due to human factors
b) Distraction

a) in-suit instrumentation and continuous monitoring
b) automated cooling actuation based on state

 Gaura, Brusey
• Sense-model-decide-act
• Two control loops
  – Rapid feedback to
    autonomously adjust
  – Support for modifications
    to mission plans and
    investigation into the
    construction of the suit.
  Gaura, Brusey
• provide detailed physiological measurement - better insight
  into what is happening

• support on-line and real-time thermal sensation estimates

• report of useful information (rather than data) to a remote
  station and the operative

• enable rapid assessment of hazardous situations

• allow the provision of thermal remedial measures through
  control and actuation

Gaura, Brusey
Part 2 : The deployment environment - a
 physiological perspective

Gaura, Brusey
     UHS and Suit Trials
• UHS- the thermoregulatory system is unable to defend against
  increases in core body temperature

• UHS - associated with significant physical and psychological

• Trials activity regime -four 16:30 min:sec cycles
    –   treadmill walking
    –   unloading and loading weights from a kit bag
    –   crawling and searching
    –   arm cranking
    –   standing rest
    –   seated physical rest

Gaura, Brusey
        Experimental data
• Measurands- wired instrumentation
     – Heart rate
     – rectal temperature
     – skin temperatures (arm, chest, thigh and calf )
• Assessment
     – Subjective thermal sensation – twice per cycle, per segment and
     – Comfort – as above
• Measurands - wireless
     – Skin temperature - 12 sites (symetrical + neck +abdomen)
     – Acceleration - 3D - 9 sites
     – Pulse oximetry, heart rate, CO2, galvanic

Gaura, Brusey
        Experimental data

 Figure 5. Core temperature responses (n=4; error bars are omitted for
    clarity) FS-NC=full suit, no cooling; NS= no suit

Gaura, Brusey
        Experimental data

                                                                                      Figure 6. Skin and rectal temperature over time for a subject wearing the full
   Figure 3. Typical heart rate response to EOD activity simulation (based on a
                                                                                             suit with no cooling. Note how core temperature rises with thigh
          single subject trial). FS-NC=full suit, no cooling; NO-S=no suit;
          W=walking; U=unloadin/loading weights; C=crawling and searching;                   temperature after the two merge. This experiment needed to be
          A= arm exercise; R= seated rest. NB. Two of four subjects were not                 terminated as the subject could not continue.
          able to complete four activity cycles.

      Figure 4. Mean skin temperature responses (averaged over 4 subjects; error        Figure 7.Self-assessed thermal sensation compared with chest skin
             bars are omitted for clarity). FS-NC=full suit, no cooling; NS=no suit            temperature for subject 1.

Gaura, Brusey
          Part 3: System design

Gaura, Brusey
 Constraints and design
       choices- I
Suit related
     –   Mix of wired and wireless
     –   Multiple sensors to each node
     –   Wires in suit
     –   Size, power and weight a concern
Suit modularity accounted for – multi-node BSN
Three tiers of comms     Two separate systems for:-
     Sensors to node            posture monitoring
     Node to node               Physiological ???
     Node to base station

Gaura, Brusey
 Constraints and design
       choices- II
Application related
     Intermittent comms - jammers, obstacles
     Maintaining autonomous operation - key
Two modes of wireless comms
     In-suit, on body - short range, near field
     External to mission control - long range
     Buffering - avoid overflow
     Priority transmission
     Information extraction in-suit
Gaura, Brusey
 Constraints and design
Safety critical
     – Cooling actuation
     – Operative alerts
     – Mission alerts
     – Hardware redundancy
Information extraction in-network - major
  design implications
Fault isolation and management

Gaura, Brusey
 Constraints and design
Instrument scope-dual
     – In field
     – In the lab - for physiological research and manufacturer research

User led choice of operation
In field
     max infromation output - thermal sensation, cooling status, trends,
        alerts x2
     Data on demand - temperature and other selected
In the lab
     Data output - continuous - all including accel
     Information output - continuous

Gaura, Brusey
                Part 4: In-network

Gaura, Brusey
Gaura, Brusey
• Basic filtering performed on sensor node
     – Allows rejection of invalid data and generation of alarms
• Additional filtering using a Kalman filter on the
  processing nodes
     – Smoothes data as well as providing estimates of error
• Posture estimation from acceleration data
• Modelling of thermal sensation
• Operative alerts (threshold based using data /
• Mission control alerts

Gaura, Brusey
                           Bremen, February 2009.
Temperature, Filters and
Fusion – Kalman Filtering
• Why filter?
     – Basic measurements may be too noisy
     – Can’t estimate gradient meaningfully
• Why fuse measurements?
     – Two measurements are more reliable than
     – Allow for / detect faulty sensors

Gaura, Brusey
       Thermal sensation
• Takes skin temperature (and optionally core temperature)
  readings as input
• Provides an estimation of thermal sensation, both per body
  segment and globally, as output
• The main part of the model is a logistic function based on two
  main parameters:
   – the difference between the local skin temperature and its
     “set” point (the point at which the local sensation is neutral)
   – the difference between the overall skin temperature and
     the overall set point
• Thermal sensation is given in the range −4 to 4, with −4 being
  very cold and 4 being very hot

Gaura, Brusey
            Zhang’s model

                           QuickTime™ and a
                  TIFF (Uncompressed) decompre ssor
                    are neede d to see this picture.

                               QuickTime™ and a
                     TIFF (Uncompressed) decompressor
                        are neede d to see this picture.

Gaura, Brusey
 Zhang’s model evaluation

                                                                                            Figure 9.Overall thermal sensation over time during the activity
Figure 8. Overall thermal sensation over time during the activity                           regime with the full suit and with no cooling.
regime with no suit.

                                                 Figure 10.Overall thermal sensation over time for a habituated
                                                 subject with the full protective suit and no cooling.

Gaura, Brusey
  Heart rate and CO2
• Some outlier rejection needed but
  otherwise processing simple
• Basic approach is to use thresholds
• Well researched limits for human comfort
  and safety in terms of heart rate and CO2
• They are used both as individual alerts
  and ideally will be part of a global state
      Posture monitoring
• Dual aim
    – Direct activity information to mission control for
         • Supervision of mission - health hazards/colapse/restrains
         • Technical assessment - problems - controller expertise
         • Inferrence of abstract info by controllers
    – Parameter for thermal state prediction
• 8 postures required: stand, walk, crawl, sitting,
  lying down (up, down, side x2)

Gaura, Brusey
  Posture processing
• J48 Decision Tree (like C4.5) approach
  used to identify posture with minimal
  feature extraction - raw accelerometer
  data fed into the tree
• Further work led to using RMS measures
  for each accel over a 5 second interval
  together with raw as input to the tree
                Part 5: Prototype

Gaura, Brusey
Gaura, Brusey
   Platform and sensors

Gaura, Brusey
Platform and sensors -
Gaura, Brusey
                Bremen, February 2009.

• Wireless links between actuation / processing nodes
• Wireless link between actuation node and remote
  monitoring point
• Data/information buffered in case of link failure - may
  be uploaded at future point
Gaura, Brusey
                      Bremen, February 2009.
     Sample System Data Flow -
       temperature / posture

Gaura, Brusey
                 Bremen, February 2009.
                                                                             QuickTime™ and a
                                                                    TIFF (Uncompressed) decompressor
                                                                       are neede d to see this picture.

         QuickTime™ and a
TIFF (Uncompressed) decompressor
   are need ed to see this picture.

                                                    QuickTime™ and a
                                           TIFF (Uncompressed) decompre ssor
                                              are neede d to see this picture.
Gaura, Brusey
                Bremen, February 2009.
      Remote Monitoring

Gaura, Brusey
                Bremen, February 2009.

Gaura, Brusey
                   Bremen, February 2009.
• Reinforcement Learning algorithms (such as
  SARSA()) can be used to develop a “policy” for
  controlling the cooling fan based on the “state”
  of the user
• Action is to turn fan on or off and regulate
• Utility is based on maintaining good comfort
  levels over time
• Takes account of battery depletion, likely mission
  duration, posture, as well as current thermal
Gaura, Brusey      Bremen, February 2009.
           Operative alerts
• Framework in place
• Data and information processing flows readily
  available (piggy back on mission control)
• Avoid false alarms - link to robustness and fault
• Sound considered at this stage but tactile
  sounds good too
• Research into HCI issues badly needed

Gaura, Brusey
                    Bremen, February 2009.
  Evaluation and results

• “Research” level instrument has been created
• System so far is not at “product” level though
  and some way yet to go

Gaura, Brusey   Bremen, February 2009.
                Review of talk

• WSN – lack of design reuse or code reuse is increasing
  development time and delays productization
• The road from theoretical advances to practical implementations is
  still to be travelled

• BSN- neither large nor widely distributed but there are a number of
  fundamental requirements
    – the size of the nodes, wearability of the instrumentation, robustness,
      reliability and fault-tolerance, etc
    - they dictate the majority of the design and implementation

• Pursuing application driven design processes will enable the
  development of industrially strong systems which will increase
  confidence in the technology and contribute to its adoption in near
                        Bremen, February 2009.
Gaura, Brusey
         Challenges- cont.
The problems:
-point measurements reporting often outside
the scope of deployment
                                                         Design for re-use
-time-space link implied as crucial

-user needs global and/or change/event driven
information as deployment outcome                     Don’t re-invent the wheel

                              Possible solutions:
Design “big” to               -In-network information interpretation
successfully go
                              -Robustness of information - cross-layer
                              design & top down, integration,
Hang on to the                distribution
deployment expertise
                              -Optimized query-able systems

Gaura, Brusey
                             Bremen, February 2009.