Costas

Document Sample
Costas Powered By Docstoc
					     Multiagent Negotiation for Constraint
                 Satisfaction

                          Costas Tsatsoulis

          Dept. of Electrical Engineering and Computer Science
                         The University of Kansas




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
             Problem Characterization

  • Distributed constraint satisfaction (resource allocation)
    problems which are either over-constrained (no optimal
    solution exists) or time bound (optimal solution cannot be
    found in allocated time) or both
  • Highly decentralized environments, where problem solving
    is performed on the local level making maximum use of
    local information
  • Both resources and their consumers are subject to change,
    and have significant individual constraints on action.


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                   Proposed Solution

  • Bottom-up formation of problem solving coalitions using a
    multiagent system

  • Allocation of resources done in a collaborative fashion

  • Collaboration managed by negotiation

  • Goal is to achieve “just-in-time” or “good-enough, soon-
    enough” solutions

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
      Brief Introduction to Negotiation for
                   Reasoning
  • We use argument-based negotiation
  • Assuming benign and collaborative agents, each agent
    attempts to convince others that it has a more urgent need
    for a resource than they do
  • To do so an agent must present arguments to its
    negotiation partners
  • If the arguments are believed and are stronger than the
    reasons an agent is using a resource, then it will free the
    resource of a percentage of it to the negotiating partner
  • Resources may be surrendered completely, partially, or
    conditionally
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                Negotiation Strategies

  • The parameters of a negotiation are its “strategy.”
  • Parameters may be the types of information to exchange,
    the time spent negotiating, the importance an agent places
    on the tasks it is performing and the perceived need for
    resources, the number of agents it is willing to negotiate,
    with, and so on.
  • Our innovation is the use of adaptive, dynamic negotiation
    strategies, which situates the negotiation in the current
    state of the world


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                       World descript ion
                                       Task , goals




     Ref lect iv e
                                       Case-Based
     Programming                                                               Ranked
                                       Negot iat ion
     Model                                                                     Negot iat ion
                                                                               St rat egies
                                   Selec t best st rat egies
  The r ef lect iv e programming
  model coll ec t s and
  dist ribut es sys t em- lev el
  inf ormat i on                        Ut ilit y
                                        Evaluat ion


                                   Us e c rit er ia and const raint s         Illocut ionary
                                   t o ev aluat e and rank
                                   possible negot iat ion
                                                                              Ac t
                                   st rat egies                               Int erf ac e

                                                                        Formulat e t he speec h
                                                                        act s t o communicat e
                                                                        and neg ot iat e wi t h ot her
                                                                        agent s




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                  Domain of Application



  • Multiple target - multiple
    shooter problem




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                       Send messages

                                     Receive messages


                                     Initiate Negotiations

                                    Respond to Negotiations

                                     Read sensor data

                                     Control the sensor

                                     Read system data

                                        Read self data

                                   Request system resources

                                    Perform problem solving



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                          Agent

                                     Agent polls sensor for data
                                     at regular intervals



                               Vehicle appears and is sensed




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                             Send messages

                           Receive messages


                           Initiate Negotiations

                          Respond to Negotiations

                           Read sensor data

                           Control the sensor

                            Read system data
                            Read system data

                             Read self data
                            Read self data

                          Request system resources
                         Request system resources

                         Perform problem solving


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                Send messages

              Receive messages


               Initiate Negotiations
              Initiate Negotiations

             Respond to Negotiations

              Read sensor data

              Control the sensor
               Control the sensor      Perform the task given the
                                              sensor data
              Read system data
                                            Decide if negotiation
                                                 is needed
                 Read self data

            Request system resources

             Perform problem solving



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                               Instantiate profile
                                       Establish negotiation
                                       Instantiate negotiation
                                          Instantiate profile
                                      Instantiate negotiation neighborhood
                                         Initiate negotiations
                                              parameters
                                             neighborhood
                                       Establish negotiation parameters
                                            Initiate negotiations
               Send messages

             Receive messages


             Initiate Negotiations

            Respond to Negotiations

             Read sensor data

             Control the sensor

             Read system data

                Read self data

           Request system resources

            Perform problem solving

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                           Send messages
                          Send messages

                        Receive messages


                        Initiate Negotiations

                       Respond to Negotiations

                        Read sensor data

                        Control the sensor

                        Read system data

                           Read self data

                      Request system resources

                      Perform problem solving



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                          Send messages
                          Send messages

                        Receive messages


                        Initiate Negotiations

                        Respond to negotiations
                       Respond to Negotiations

                         Read sensor data
                        Read sensor data

                        Control the sensor

                        Read system data
                        Read system data

                          Read self data
                          Read self data

                       Request system resources
                      Request system resources

                      Perform problem solving



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
          Reflective Programming Model

  • Provide executing agents with a sophisticated, adaptable
    view of system state, resources, and competing
    computations
  • Enable agents to monitor and control their own progress
      – Agents can potentially select their own computational model
  • Agents may adapt future behavior in response to past
    behavior on a system-level
  • System-level resources become explicit and enter into the
    negotiation
  • “Soon-enough” results are achieved

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
            System Software Architecture


          Agent SW                                     Agent SW
          RTSS Lib         RTSS Scheduler   TAO      Sch Ping CPU
                              Object


   User
   KURT-Linux
                               RT           Timers         DSKI
                             Schedule


                           RT Scheduler


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                              Scheduling
  • RTSS Library API wraps RTSS Scheduler Object interface
      – Ace ORB (TAO) Object based access
  • Scheduler object
      – Evaluates agent requests and responds
      – Combines agent allocations into a new schedule
  • Agents adjust their own scheduling constraints
      – Currently CPU use pattern
      – Other system resources in the future
      – Negotiate with Scheduling object




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                       Ping
  • Enables agents to relate computational and temporal progress for each
    thread
      – Proportional progress model (e.g. .2,.4,.6,.8)
  • Agent code uses ping interface to
      – Establish progress update points
      – Query progress state
  • Supports per-thread model with per process POSIX timers
      – Per threads timers preferred




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                             CPU Use

  • Provides resource use information for all registered RT
    threads
      – Gives each thread an agent and global view
      – Can be used by threads in deciding how to request or adjust
        resource use
  • Data Stream Kernel Interface based
  • Extensible to other resources




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                          Agent Profiles

  • Agent has preference over aggregates of goods
  • Agent has preference over satisfaction of goals
  • Agent has preference over negotiation partners
  • Parameterized preference profiles used to select among
    goals and goods
  • Situation- and goal-dependent preferences
      – Situation instantiates profiles, “persuasion threshold,” evaluation
        criteria
  • Helps define which offers to accept, decline, or modify in
    negotiation

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                   Instantiate Profile

  • Profiles are parameterized structures that describe the
    preferences of an agent
  • They are instantiated based on the world information the
    agent has, i.e. what it senses, what it knows about its own
    state, and what it knows about the system state
  • The agent profile may indicate preferences of certain
    resources over others, preferences of negotiation partners,
    and preferences over task satisfaction




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                                          Profile

    Profile Header           Agent A Profile

    Profile Body
                   Resource If ?task=recognition then be willing to spend more power
                   Preference
                                 If ?target-speed>200 then ask system for smaller and
                                 more frequent periods of execution
                                 If ?power<10 then avoid ON/OFF switching

                   Negotiation
                   Partner
                                 If ?time-of-day=night then prefer negotiating with an IR
                   Preference    sensor
                                 If ?target=type-A and ?prob<80 then prefer negotiating
                                 with an RF sensor




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                            Negotiating :
                             I. Basics
• An agent may negotiate with multiple partners concurrently

• An agent may be an initiator, a responder or both

• An agent estimates the future location of a target and when a target will enter
  another sensor‟s lobe(s)

• This establishes who to negotiate with, when to start negotiating, and how
  long to negotiate

• Negotiation is argumentation-based. Argumentation is modeled as
  production rules in CLIPS

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                       Negotiating :
          II. Initiating a Negotiation Request
• Retrieve the best case to obtain a negotiation strategy

• Use utility theory to sort potential negotiation partners

• The utility is a weighted sum of three sets of attributes:
    – the time that the target will hit the coverage of a potential partner or the time that the target
      will leave the coverage of a potential partner
    – the past experience of negotiations between the agent and a potential partner („have I been
      helpful to you?‟ „how many times have we had successful negotiations?‟)
    – the current relationship between the agent and a potential partner („are we already
      negotiating about some other things?‟ „how much coverage overlap do we have between
      your sensor sectors and mine?‟ „have I already initiated a negotiation request to you?‟)


• For each potential partner, we obtain the most useful sector (highest utility)
  and sort the potential partners based on their most useful sectors.


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                    Negotiating :
   II. Initiating a Negotiation Request (cont.)

  • Create negotiation tasks

  • Upload relevant data for each negotiation task

  • Activate a negotiation thread for each negotiation task

  • A request to negotiate consists of:
         (Initiating-Agent Message-ID Request)




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                       Negotiating:
               III. Responding to a Request
  • A request to negotiate consists of:
          (Initiating-Agent Message-ID Request)
  • If Request is to turn on a sector that is already on and measuring, then
    the agent agrees
  • If there are no idle negotiation threads, then the agent declines to
    negotiate
  • Otherwise, the agent decides to negotiate
      –   creates a negotiation task
      –   retrieves the best case to obtain a set of negotiation parameters
      –   uploads relevant data about self, other sensors and target
      –   activates a negotiation thread
      –   if the activation fails, the agent declines the request to negotiate


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                      Case-Based Negotiation:
                      I. Situation Description
   •   tasks
        – descriptions of what an agent is doing.
   •   partners
        – describes the agents with whom an agent negotiates with.
   •   target model
        – a description of the target which is being tracked or identified.
   •   constraints
        – the limiting factors determining if a task can be accomplished.
   •   request
        – the type of request: TurnOnSectorAmpliture, TurnOnSectorFrequency,
          TurnOnSectorBoth, GiveUpCpuResource




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                     Case-Based Negotiation:
                     II. Negotiation Strategy
   •   Initiator:
        –   ranking of classes of information to send as part of the argument
        –   total time willing to spend
        –   total negotiation steps willing to engage in
        –   total cpu resources willing to allocate to the negotiation
   •   Responder:
        – total time willing to spend
        – total negotiation steps willing to engage in
        – cpu resources willing to allocate to the negotiation
        – power required to satisfy the request
        – function used in certain negotiations (exponential, linear)
        – function parameters (kappa and beta)
        – ranking of classes of information that it would send back to the initiator during
          negotiation/argumentation (so it defends its use of its resources)
        – persuasion threshold used to determine if convinced to release or share a resource


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                Case-Based Negotiation:
                     III. Retrieval


    • Simple weighted similarity measures on the case features
    • Best case selected - If more than one match with the exact same value
      we select the one with the best outcome
    • Case “solution” contains the negotiation strategy
    • Case also contains outcome of old negotiation (success, aborted,
      out-of-resources, rejected, channel-jammed, out-of-time)




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                 Case-Based Negotiation:
                     IV. Adaptation
  • Retrieved strategy is adapted to reflect differences in
    situations
  • Two adaptation strategies:
      1. difference driven
       TimeAllowed and StepsAllowed modified based on the % difference of
        TargetSpeed
       if old request was TurnOnSectorBoth and new request is
        TurnOnSectorAmplitude, lower persuasion threshold by 10%


      2. outcome driven
       if old outcome was Rejected then decrease TimeAllowed and
        StepsAllowed by 10%
       if old outcome was OutOfTime then increase TimeAllowed and
        StepsAllowed by 15%



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
               Case-Based Negotiation:
                    V. Learning

   • New negotiation strategies and their results are learned
   • Before learning we cluster the case base to see if the new
     case should add new information and should be learned
   • We have noticed that learned cases are used often in
     negotiation
   • We have not yet studied the advantages -if any- of learning




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                   Negotiating:
         IV. Negotiating by Argumentation
  • Supply arguments to responder to “convince” (i.e. push
    argument above persuasion threshold)
  • Use CLIPS production rules
  (defglobal ?*evidenceSupport* = 0.0)

  (deftemplate self (slot _posX) (slot _posY)(slot _numOfTasks) …)

  (defrule self-num-of-tasks
     (self (_numOfTasks ?x))
     =>
     (if (= ?x 1)then
        (bind ?*evidenceSupport* (+ 0.1 ?*evidenceSupport*))
        else
        (if (= ?x 2)then
           (bind ?*evidenceSupport* (+ 0.2 ?*evidenceSupport*)))))



INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                 Negotiating:
   IV. Negotiating by Argumentation (cont)
  • If initial argument is not sufficient, responding agent sends
    the initiator a MORE_INFO message
  • Initiator sends the next ranked class(es) of arguments
  • Process stops when:
      –   persuasion threshold is exceeded SUCCESS
      –   total time or steps are exceeded by any participant OUT-OF-TIME
      –   no response is received to any message CHANNEL-JAMMED
      –   no more information can be supplied REJECTED
      –   low cpu resources OUT-OF-RESOURCES
      –   other catastrophic failure ABORTED


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                  Negotiating:
       V. Using the RTSS for Negotiation
  • During negotiation the agents need to:
      – know the exact time elapsed
      – know the exact cpu % used by various processes and threads
      – be able to request new cpu allocation as the result of negotiation
        (give up or get cpu resources)


  • The RTSS gives the agents this information




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                       RESULTS




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                  EXPERIMENTAL SET-UP


                           50 ft




                             70ft
                          speed=0.5ft/sec




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                      Attempted and Successful Negotiations Involving Each Agent

 12 0


 10 0


  80

                                                                                              # attempts
  60                                                                                          # of succe ss

  40


  20


   0
        1->2   1->3     1->4   2->1   2->3   2->4   3->1   3->2   3->4   4->1   4->2   4->3




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
            Percentage of Total Negotiations Involving Each Agent

          100%
           90%
           80%
           70%
                                                                    A4
           60%
                                                                    A3
           50%
                                                                    A2
           40%
           30%                                                      A1
           20%
           10%
            0%
                      A1           A2           A3           A4




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                  Tracking Accuracy




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                 Comparisons:
         Negotiation vs. No Negotiation
         Case-Based vs. Static Strategy




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                      Experiments

  • No negotiation: Each sensor senses and tracks
    independently. There is no communication between sensor
    agents
  • Static negotiation strategy: Agents negotiate using a pre-
    defined, static, “good” negotiation strategy




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
              Messages sent vs. types of agent behavior II

45

40                                                        no n ego ti atio ns
                                                          sta ti c ne gotia ti on stra teg y
35                                                        ca se -b ase d strategy
30

25

20

15

10

 5

 0
             average l en gth                                   co st
                                Agent beha v ior type s




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
                  Some Conclusions

  • The “No negotiation” strategy sends almost 50% more
    messages to the Tracker and almost 20% more total
    messages, yet has almost 50% worse tracking accuracy
    than negotiation strategies
  • The static negotiation strategy sends almost 10% fewer
    messages, but has a higher message cost than CBR because
    of excessive length of messages
  • The static negotiation strategy has approximately 35%
    worse tracking accuracy than CBR


INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER
       Go to

       http://www.ittc.ukans.edu/ANTS

       for details on software, methodology, presentations, etc.




INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER

				
DOCUMENT INFO