Automatic Agent GUI Generation by hcj


									          Automatic GUI Generation for Device Agents
                                            Kyle Neumeier, Joe Hoag
                                   CSCE Department, University of Arkansas
                          340 B Eng Hall, University of Arkansas, Fayetteville, AR 72701

Abstract – The Ubiquity framework is an open source          However, the theory behind generating GUIs for
toolkit that enables developers to wrap devices with         devices is not restricted to Ubiquity and the java swing
communicative classes – resulting in smart devices.          package. Furthermore, there are large classes of device
When adding agents to a Ubiquity agent system,               agents for which this method will work; a simple
creating the graphical interfaces for controlling the        example is given in this paper.
agents is a tedious task that steals development time
from creation of the agent itself. Currently, there is no
way of capitalizing on the fact that a large number of                           BACKGROUND
agents have interfaces that contain similar kinds of
controls that interact with similar kinds of fields within   Ubiquity Background
the agent. This paper introduces ―properties,‖ which         A brief overview of the Ubiquity architecture [1] will
are XML descriptors of certain key fields in an agent.       be given in order to provide some context for the
A GUI-Builder agent then uses these properties to            concept of automatic GUI generation. As mentioned
construct the interface for the agent. This alleviates the   above, Ubiquity is a framework for developing
developer from having to spend time building and             communicative agents that can wrap devices. Thus, it
debugging graphical interfaces.                              is mainly focused on system level activities such as the
                                                             actual passing of messages.        Ubiquity will be
                                                             overviewed by listing and explaining a few of its most
                   INTRODUCTION                              important classes.
Problem                                                      Agent — The agent class represents the “actor” in the
Despite the fact that many agents added to a Ubiquity-       system; it is what actually wraps the device. The agent
based agent system share interfaces that are nearly the      class provides some basic functionality such as the
same, a new interface must be constructed for each kind      ability to register to listen for certain messages and the
of agent. Because GUI programming requires a                 ability to send messages to other agents. It is important
specific skill (familiarity with the Java Swing package)     to note that the only difference between agents is the
that many programmers may not possess, GUI                   kind of messages they listen for and the actions they
development may be a disproportionate cost in terms of       take when they receive a message. [1, 2]
development time. Still, the GUI, however simple it
may be, is a necessity for usability. Thus, the agent        Message — The Message class contains two parts. The
                                                             first is a header with information such as source and
developer with little Swing experience is forced to
                                                             destination IP addresses and port numbers. The more
spend a large amount of time writing and debugging
                                                             interesting portion of the Message class is the
graphical interfaces that will turn out to be mediocre at
                                                             MessageBody, which is a class which a developer
                                                             extends to create a new message type. For example, if
Objective                                                    a developer would like to create a “power” message
                                                             telling a light switch agent to turn on or turn off, he
The objective of this project is to define descriptors of    would extend the MessageBody class. In this newly
common data types used in Ubiquity agents and to             created class he might add a boolean state variable
develop a mechanism that automatically builds                called “power” that would tell the light to turn on or off.
graphical user interfaces based on these descriptors         [1]
Scope                                                        XMLTranslator — One important aspect of Ubiquity
Because Ubiquity is written in java, the automatic           messages is that all messages are sent between agents in
interface generator is also implemented in java.             XML. However, when an agent receives a message, it
                                                             can turn the message back into an object. The
XMLTranslator class provides the parsing required to         controls. For example, a GUI control for a hashtable is
do this task. [1]                                            not needed because hashtables are not normally used by
                                                             most agents.
GUIAgent — The GUIAgent is a system level agent
(just like any other agent, it simply extends the Agent      Finally, each data type used by Ubiquity agents to
class) that facilitates GUI support. It builds a list of     capture states can be represented by a small number of
connected agents in a tree on a panel on the left. When      graphical control types. For example, a boolean can be
a particular agent is selected from this tree, its control   effectively represented by a two-option radio button
panel is displayed on the pane on the right. Typically it    control. An integer can be represented by a slider.
is this control panel that must be built by hand.            Because it is relatively simple to represent simple data
However, the automatic GUI Building mechanism                types with the same simple controls, it is possible to
seeks to automate this construction. Once the panel is       limit the number of cases and controls to deal with to a
built, it is simply added to the GUIAgent like traditional   small number, thus enabling a simple algorithm.
                                                             Related Work
                                                             A closely related project is the Personal Universal
                                                             Controller (PUC) portion of the PEBBLES project at
                                                             the Carnegie Mellon University. In this project, a XML
                                                             specification of functionality of devices is used to
                                                             automatically generate graphical user interfaces to
                                                             devices on a personal computer. The PUC project is
                                                             similar to this one in that the number of data types and
                                                             graphical controls is limited in order to simplify the
                                                             algorithm for creating the devices [3]. The projects
                                                             differ in that this project is intended to be built into an
                                                             existing agent framework, whereas PUC focuses only
            Figure 1 A GUIAgent example                      on how such interfaces can be built.

Key Concepts
Before delving deeper into the design of the automatic                             APPROACH
GUI generator, it is important to understand three key
                                                             At the heart of this project is a class called Property that
insights into Ubiquity agents that make this project
                                                             extends the MessageBody class. It allows developers to
possible. Firstly, for a large class of agents (at least
                                                             create new “properties” inside of their agents. The
those of all kinds of physical devices like stereos,
                                                             properties have different types (such as int, string, bool,
microwaves, and thermostats), the state of an agent at a
                                                             etc.) and represent a user controlled parameter to that
given time can be represented as the set of user-
                                                             agent. The GUIAgent, when it is instantiated, sends a
specified parameter values. While it is true that some
                                                             PropertyRequest Message to each of its connected
complex and abstract agents, such as database agents,
                                                             agent. At that time, the agent will send back all of its
may contain other kinds of data that affect their states,
                                                             properties. The GUIAgent will then use this list of
there are many kinds of simple agents whose states can
                                                             properties to generate the GUI. When the user changes
be completely controlled by user input. As a simple
                                                             the value of a component on the GUI, the GUI sends
example, the state of a dimmer light switch agent can
                                                             the respective agent a message that tells the agent
be represented by two user-specified parameters: an
                                                             which property was changed and how it was changed.
on/off Boolean and an intensity integer. This fact
                                                             The agent then decides what to do with this information
allows useful GUIs to be built in the first place for a
                                                             and broadcasts the update to all interested GUI panels.
class of agents that includes physical devices.
                                                             One of the design goals of this project is to keep GUI
A second observation that provides a foundation for this
                                                             specific information out of the properties messages in
project is that a vast number of agent states can be
                                                             order to prevent unnecessary complexity. The idea is
captured using only a small number of data types.
                                                             that the properties message should reflect the data that
Furthermore, the data types that are used are normally
                                                             is being exposed (and some of its attributes such as the
primitives. For example, an RFID reader agent (which
                                                             domain), but leave all GUI decisions to the GUI
is somewhat more complex than a light switch agent)
uses only booleans, strings and integers. This fact has
the effect of simplifying the set of needed graphical
Another design decision was to maintain backward             the type is boolean, it adds a radio button with two
compatibility with previous versions of the Ubiquity         choices to the GUI.
framework by having the GUIAgent first check for
traditional, hand-built panels before constructing an        Perhaps the most complex functionality of the
AutoPanel from properties. If the traditional panel          AutoPanel is keeping track of which properties it has
exists, then it ignores the properties.                      added and how to respond to changes to these
                                                             properties. Because it is possible to have more than one
Property — The property class contains several               GUI panel per agent, it must also listen for
important attributes. The property has a name that           PropertyUpdated messages in case a change to the
allows the agent to programmatically control it. It also     property has taken place in another GUI.
has a string called value that represents the state of the
property, the semantics (i.e. whether it is an int or a
string) of which depend on the type attribute. The type
can be one of several choices:

  Bool – A bool property represents a boolean data
   type (Implemented)
  Int – An int property represents an integer data type
   (partially implemented)
  String – A string property represents an String data
   type (partially implemented)
  Real – A real property represents a Real data type
   (not implemented)
  Date – A date property represents a date (without a
   time)(not implemented)
  Time – A time property represents a time (without
   a date)(not implemented)
  Timestamp – A timestamp property represents a
   date and a time together (not implemented)
  Choice – A choice property represents when a user
   has a choice between several objects (not
  Collection – The most complex property. It
   represents a collection of several properties which
   may or may not be of the same type. It may be
   used to create tabbed panels (a similar method can
   be found in the PUC project) that represent
   different “modes” of the device (such a “on” and
   “off” or “play”, “rewind” and “stop”)(not
Another important attribute of the Property class is the
Annotation string. It is a list of name-value pairs that
provide extra information which can be used by the
GUI builder. For example, an int property may provide
the GUI builder with a “min” and a “max” integer. The
Property class also contains functionality to add new           Figure 2 UML Diagram of the Property class
name-value pairs to the annotation string and to parse it.

AutoPanel —The AutoPanel class is contains the                    LIGHT SWITCH AGENT: AN EXAMPLE
functionality for actually creating the new GUI controls
                                                             In order to better explain how the Property and
from an agent’s properties and adding them to the
                                                             AutoPanel classes enable rapid agent construction
GUIAgent. It listens for AvailableProperties messages
                                                             through the elimination of hand-build GUIs, an
from its agent.          When it receives a new
                                                             example will be discussed. The example, a light switch
AvailableProperties message, it decides how to create a
                                                             agent with a dimmer capability, will demonstrate how
control for it depending on the type. For example, if
                                                             properties work, despite its simplicity.
Class LightSwitch extends Agent{                                               CONCLUSION
Boolean state;
Int intensity;                                              The initial implementation results have been promising.
Public LightSwitch(){                                       A GUI for a dimmer light switch with two controls (a
  Property power = new Property();                          boolean and an int) has been created. It seems to work”Power”;                                       fine in single GUI environments, but has not been
  power.label=”Power”;                                      tested when more than one GUI can control the agent.
  power.type=Property.TYPE_BOOL;                            Although the GUI is simple, it is usable and could
  power.value=”false”;                                      possibly help a developer, especially in rapid;                               prototyping an agent.

   Property dimmer = new Property();                        This work could definitely produce some interesting”Dimmer”;                                    results, especially because not much work in automatic
   dimmer.label= “Dimmer”;                                  GUI generation for agents has been done. It appears
   dimmer.type=Property.TYPE_INT;                           that because device agents are similar enough in terms
   dimmer.value=0;                                          of functionality, generating the GUI for them is a
   dimmer.addAnnotation(“min”, “0”);                        feasible task. However, the work is far from being
   dimmer.addAnnotation(“max”, “100”);                      completed. Much more work must be put into creating;                             the AutoPanel’s building of controls based on
}                                                           properties. As seen above, most of the defined property
void processChangePropertyMessage(                          types do not have an implementation. Also, in order for
ChangeProperty cp){                                         the GUI to be usable from the user’s point of view and
  if(cp.Name.equals(“Power”){                               general enough for developers to be able to use them in
    //Code to handle power change                           various situations, the GUI controls must be constructed
  }                                                         well.     This will require a lot of detail-oriented
  if(cp.Name.equals(“Dimmer”){                              programming.
    //Code to handle dimmer change
}                                                                              REFERENCES
In the above code for a light switch agent, two             [1] Joe Hoag, Craig Thompson, “Architecting RFID
properties are created, initialized, and added to the       Middleware,” Architectural Perspectives column, IEEE
agent’s list of available properties. Then, when the        Internet Computing, September-October, 2006.
GUIAgent is created, it will poll all available agents to
ask for their available properties. At that time,           [2] J. Hoag; “Ubiquity Agent System Framework”;
LightSwitch will send GUIAgent its properties.    
GUIAgent will create a new AutoPanel and will hand          CSCE-490-590--RFID-Agent-
off the properties to it. AutoPanel will construct the      Middleware/DOCS/2006-08--Ubiquity-
GUI, register the LightSwitch agent for future message      Architecture.doc
passing (so it knows who to send messages to when a
user changes something on the GUI), and add the GUI         [3] Jeffrey Nichols, Brad A. Myers, Michael Higgins,
to GUIAgent.                                                Joe Hughes, Thomas K. Harris, Roni Rosenfeld,
                                                            Mathilde Pignol. “Generating Remote Control
                                                            Interfaces for Complex Appliances.” CHI Letters:
                                                            ACM Symposium on User Interface Software and
                                                            Technology, UIST'02, 27-30 Oct. 2002, Paris, France.

  Figure 3 Screenshot of a GUI for a light switch

To top