Get documents about "Functional Unification Approach to Visualization Design
Document Sample
Functional Unification Approach to Automated Visualization Design
Stephan Kerpedjiev, Steven F. Roth, Joe Mattis
Carnegie Mellon University
Pittsbrugh, PA 15213
{kerpedjiev,roth,jam@cs.cmu.edu}
Abstract Uniformity - the same formalism can be used for all
stages of the design process.
A unification-based approach to visualization design Usability - the system provides convenient means for
provides a uniform way of representing user require- users to express their design knowledge.
ments, design knowledge, and graphic designs as well as
Reusability and extensibility - the system can be
algorithms for synthesizing graphic presentations. We
extended with new design elements, constraints, and
demonstrate this approach on two types of requirements
knowledge by reusing existing grammars.
– structural in the form of sketches and functional in the
form of tasks. With this approach we aim to achieve the We chose functional unification as a formalism to
following system design goals: expressiveness (the for- achieve our goals because it has the following proper-
malism can express the visualization design problem ties:
and its problem-solving algorithms), uniformity (the It is a constraint-satisfaction method, which fits per-
same formalism can be applied to different generation fectly with the nature of the design task.
tasks), usability (the formalism is convenient for people It is fairly simple: functional unification grammars
to express their knowledge), efficiency (graphics can be (FUGs) have only one type of data structure, the
designed in a reasonable amount of time), and extensi- functional description, and one type of operation,
bility (the system can be extended with new types of unification (Shieber, 1986).
requirements, design elements and design knowledge by It provides a common representation for multimedia
providing reusable grammars). generation, where the natural language and graphics
can be coordinated by sharing common data struc-
The visualization design problem tures.
The visualization design problem is to synthesize a
Prior work in natural language generation has already
graphical presentation that expresses a set of data to
produced programming environments for unification
satisfy given requirements. The requirements may come
(Elhadad, 1992), which we can use for graphic design.
in different forms. For example, there might be no re-
quirements besides presenting the data. Or, the user Our graphic design system employs a wealth of
might sketch some elements of the visualization (e.g., a knowledge gained in prior work on automated graphic
chart with an interval bar), and even how some of the design. This knowledge includes the definition of ex-
data attributes map to graphical properties. Another type pressive and effective graphical languages by Mackinlay
of requirement consists of the tasks that the user needs (1986), task-driven graphic design by Casner (1990),
to perform. By addressing different design requirements data characterization by Roth and Mattis (1990), and
we go beyond Mackinley's (1986) graphical presentation visualization tasks by Zhou and Feiner (1998). Later
problem, which did not consider any requirements other work on multimedia explanation showed how commu-
than the data and Casner's (1990) work, which consid- nicative goals can be mapped to user interpretation
ered only tasks. With the challenge of designing visuali- tasks, which in turn can be used as requirements to the
zations that respond to various types of requirements, graphic designer (Kerpedjiev and Roth, 2000). Work on
we turn our attention to using a formalism that can ex- user interfaces for graphic design brought a different
press the variety of requirements in a uniform way. We perspective to the automated design problem. Interfaces
propose a unification-based approach to graphic design such as SageBrush (Roth et al., 1994) and SageBook
to achieve the following system design goals: (Chuah et al., 1995) let the user express elements of the
design in an intuitive visual way. This gives the user
Expressiveness - the formalism can capture various more control but also poses a greater challenge to the
design requirements and the knowledge needed to designer - SageBrush sketches need to be reconciled
generate graphics that satisfy those requirements. with designs generated automatically by Sage. This
Efficiency - the system can produce designs in a challenge was a major contributing factor for choosing
matter of seconds. functional unification - decisions informed by different
sources such as data characteristics and user sketches
1 08/08/12
need to be unified in order to produce desirable presen- attribute-value pairs, where each value is either an atom,
tations. another FD, or a path (pointer) to another value. For
This paper extends prior work on automated graphic example, in the sample FD below, the value of a is the
design in three ways. First, this paper demonstrates how atom x, the value of b is another FD with two attribute-
a functional unification approach can provide a common value pairs, and the value of c is a pointer to another
representation throughout all stages of the design pro- value obtained by following the path {b m}. Each com-
cess. This provides significant support when engaging ponent of a FD is "addressed" by the path that leads
the larger problem of automatically designing multime- from the root to that component by following the attrib-
dia presentations with coordinated natural language and utes in the path. For example, the address of the whole
graphics. FD is the empty path, which is denoted {}. The address
of the value of attribute a is the path {a}. A pointer indi-
Second, this paper contains previously unpublished de-
cates shared representation. In the sample FD, the com-
tails about the internal representation of graphical design
ponents {c} and {b m} should always have a common
requirements, search strategies, and completed designs,
which can facilitate the implementation of new automat- value. Therefore, if a constraint is imposed on {c}, it
ed design systems. will be imposed on {b m} as well and vice versa. Com-
ponents can be accessed using relative paths as well.
Third, and most significantly, this paper introduces the Instead of an attribute, a relative path begins with the
concept of applying design strategies, which is a logical special character “^”. Each such character indicates go-
and systematic approach that produces more coherent ing one level up the FD structure. For example, we
graphical designs, more quickly, than previous automat- could modify the sample FD into an equivalent one by
ed design systems. In particular, both Mackinlay's APT replacing the {c} component with the atom v, and the {b
system (1986) and prior Sage work (Roth et al., 1994) m} component with the relative path {^ ^ c}, which
functioned by determining a maximally "effective" set means go up two levels and then follow attribute c.
of perceptual techniques (e.g., position, size, color) and
then by exhaustively searching for a means of compos- Sample FD: Sample grammar: Enriched FD:
ing those techniques into a coherent design. Converse- ((a x) ((c v) ((a x)
(b ((m v) (alt (((a y) (b ((m v)
ly, design strategies take a top-down approach to the (n w))) (p s)) (n w)))
graphical design problem, which improves system re- (c {b m})) ((a x) (c {b m})
sponsiveness, makes extensibility more tractable, and (p t))))) (p t))
produces more coherent graphical designs. The attribute-value pairs in an FD are conjunctive, i.e.
the object being described by the FD must possess all
Functional unification the properties specified in it. Grammars, however, use
Functional unification (Kay, 1979) is an approach to disjunction as well - for a string to be accepted by a
natural language processing that assumes the functional grammar, only one of its rules need to be satisfied. To
perspective to language rather than the more common accommodate disjunction in FUGs, the formalism was
structural perspective. The functional perspective re- extended with the alternation construct specified by the
flects the role of the constituents of a message while the attribute "alt" followed by a list of alternative FDs. For
structural perspective cares exclusively about the well- example, the sample grammar above consists of the pair
formedness of the messages. In generation, the function- (c v) in conjunction with one of two alternatives: the
al perspective is crucial as it deals with the proper map- pairs (a y) and (p s) or the pairs (a x) and (p t).
ping from communicative goals to components of the The only operation defined on FDs is unification. The
message. We intentionally replaced the terms "sentence" unification of two FDs either produces a new FD that is
and "words" used by the proponents of functional unifi- compatible with and more specific than the input FDs or
cation with the more general concepts "message" and fails. The unification fails if at least one attribute has
"components" to include graphical communication into incompatible values in the input FDs. Two values are
the discussion of functional unification. Indeed, just like incompatible, if one is atomic and the other is an FD, if
language achieves communicative goals using finite they are two different atoms, or recursively, if they are
discrete messages in the form of sentences and words, two incompatible FDs. Compatible FDs are merged into
graphical languages use discrete "messages" in the form one FD much like set union except that it is recursive. In
of charts with their own constituents such as graphemes the case of alternation, only one of the alternative FDs
to achieve communicative goals. needs to unify with the other FD.
Unification is a simple formalism. It requires just one Here is how the unification of the sample FD with the
type of data structure called functional description (FD) sample grammar would proceed. The first pair of the
for representing both the input (the requirements for a grammar is (c v). The value of attribute c in the input is
sentence or a graphic) and the grammar (the knowledge accessed following the path {b m}, which yields the
that guides generation). An FD describes an object via atom v. The two values are equal and therefore the
2 08/08/12
matching is successful. The next construct of the gram- A formal representation of the graphic design facilitates
mar is a set of alternatives, the first one being the FD the proper communication between the system modules
((a y) (p s)). The first pair of this FD is (a y), and supports the reasoning of the designer. It captures
which does not unify with the pair (a x) of the input the graphical elements, their relationships, and the map-
FD. Therefore, this alternative fails and the next one is ping of data objects to graphical objects. Our design
tried. The first pair (a x) of that alternative unifies with representation (explained in the next section) is based on
the pair (a x) of the input. The second pair, (p t), does prior work in the Sage group (Roth and Mattis, 1990,
not have a counterpart attribute in the input and there- Roth et al., 1994, Chuah et al., 1995).
fore is added to the result. Finally, the pair (b ((m v) The design knowledge is modularized into the following
(n w))) has no counterpart in the grammar and there- sub-grammars applied to the input in this order:
fore is added to the result. The unification produces the
Mapping design requirements into constraints on
enriched FD given above.
the design (grammars SKETCH-MAPPINGS and TASK-
Like in other grammar formalisms, the output of unifica- MAPPINGS).
tion consists of terminal and non-terminal symbols. The Creating the skeleton of the symbols that express
non-terminal symbols, called constituents, need to be data elements (grammar DESIGN-STRATEGIES).
unified with the grammar again. They are specified us- Merging different designs (grammar COMPOSITION).
ing the special attribute “cset” followed by a list of the
Checking the completeness and consistency of the
attributes that identify the constituents of the FD. By
design (grammar COMPLETE-DESIGN).
tradition, the type of a non-terminal symbol is given as a
value of the “cat” (category) attribute. The general uni- Design representation
fication procedure consists of the following steps: The design representation fully specifies the visualiza-
1. Unify the input with the grammar tion so that other components of the system can use it to
2. Identify the constituents of the resulting FD perform other tasks such as rendering, explaining or
3. Recursively unify each constituent with the grammar. supporting interaction. Figure 1 shows a sample graphic,
Having presented the basic concepts of functional unifi- which consists of three horizontally aligned spaces. Fig-
cation, we turn to the heart of the problem - analysis of ure 2 (from Chuah et al., 1995) decomposes it into de-
the design problem, identifying its ingredients, and rep- sign elements. The main design components are spaces
resenting them by FDs, grammars and unification pro- (charts, maps, networks, tables), encoders (axes, color
cedures. keys), symbol sets, graphemes (marks, bars, lines) and
their properties.
Ingredients of visualization design The space is a container for symbols and imposes a lay-
At the highest level, the visualization design problem out discipline via its encoders (e.g., X and Y axes for
deals with three types of objects: requirements, design charts). Each space is represented by an FD, with attrib-
knowledge, and graphic designs. Roughly speaking, the utes for its type and one or more positional encoders.
analogs of those objects in natural language processing For example, a space of type chart would be represented
are semantic (or logical) forms to be communicated, as follows:
natural language grammars that map semantic to syntac- ((space1 ((cat space)
tic form, and syntactic specification of sentences. (type chart)
(x-axis ((g-type x-position)
Our system deals with three types of requirements: (data-type date)))
(y-axis ((g-type y-position)
Sketches. The user imagines what kinds of graphics (data-type address))))))
they need and sketches key design elements using a
specialized drawing editor called SageBrush (Roth An encoder maps values from a data type such as date to
et al, 1994). Those sketches are then parsed and graphical values such as x-position. Hence, their repre-
represented as constraints on the design. sentation consists of those two elements.
Tasks. The users specify their data exploration or A symbol provides an integrated view to a data object
communicative goal, which gets decomposed into a by presenting several of its attributes via the graphical
sequence of tasks. Our task language is based on properties of one or more graphemes, all of which are
work by Casner (1990) and Roth and Mattis (1990), co-located in space. The main components of the sym-
as well as some recent work in our group on multi- bol description are its determinant - the attributes that
media generation (Kerpedjiev et al., 1997). determine the position of the symbol, and a pointer to
Data characteristics. The data manager specifies the space in which the symbol resides. A sample FD of a
properties of the data that are important for the se- symbol is given below:
lection of graphical techniques (based on prior work ((cat symbol)
by Roth and Mattis (1990) and Mackinlay (1986)). (det ((y address)
(x1 date-on-market)
3 08/08/12
Figure 1. A sample graphics consisting of three aligned charts.
(y-pos ((attr address)
(encoder ((data-type home-address)
(g-type y-position)))))
(size ((attr lot-size)
(encoder ((data-type square-feet)
(g-type size))))))
The definitions of graphemes and symbols is applied to
a whole set of graphical objects in a space, with each
graphical object created by applying the design to an
individual data object (or record). Thus, it is more accu-
rate to call them symbol sets and grapheme sets if we
refer to the sets of graphical objects, and symbol class
and grapheme class if we refer to the definition of the
prototype graphical objects. For simplicity we will use
the terms symbol and grapheme and assume that the
meaning can be inferred from the context.
To summarize, the design is an interlinked collection of
spaces, symbols, graphemes, and encoders. Each graph-
Figure 2. The design elements of a complex graphic eme is a part of exactly one symbol, each symbol can
reside in exactly one space, and each space imposes a
(x2 date-sold))) layout discipline by a combination of positional encod-
(of-space {chart1}))
ers.
where {chart1} is the path to a space description like
the one given above. A symbol with this description Design strategies
occupies a horizontal interval location within the chart Design strategies represent possible high-level organiza-
pointed to by the of-space attribute. This location is de- tions of graphics. The basis for a design strategy is the
termined by the triple of attributes: address, date-on- mapping of data attributes to the positional properties of
market, and date-sold. symbols. In general, a design strategy prescribes how
The grapheme is an atomic graphical object such as the graphic will use the space to structure the infor-
mark, bar, line or text that conveys information through mation and leaves out any other details such as how
its graphical properties (e.g., x-position and color). Each many and what kinds of graphemes will constitute the
grapheme is described by its type, a pointer to the sym- symbol, or if and what retinal properties (e.g., color and
bol it is part of, and a number of graphical properties shape) will be used for encoding data attributes. For
that encode data attributes. For example, the mark de- example, at least the following two strategies could be
scribed below conveys information via its x-position, y- adopted for presenting data about four attributes of
position, and size: house sales: the date the house was put on the market,
((cat grapheme) the date it was sold, address, and selling price.
(type mark)
(of-symbol {symbol1})
(strat1) By symbols that mark the intervals each house
(x-pos ((attr date-sold) was on the market. External to the strategy might be a
(encoder ((data-type date) text annotation of price.
(g-type x-position)))))
4 08/08/12
(strat2) By symbols whose spatial distribution conveys egy the designer makes an important decision about
the correlation of the date on which a house was put on what the main view to the data will be and will stick to
the market and the date on which it was sold. External to this view as long as there is no evidence that the user
the strategy might be the size of a mark for price and a needs a different one. For example, if at a certain point
text annotation for address. of the design process, the designer decides to organize
The strategies are represented in the grammar by FDs the graphic around location, it would try to realize all
with two components: strategy type and determinant. subsequent constraints in the framework of the LOC
The strategy type (explained below) is a convenient ab- strategy unless this proves impossible or ineffective.
straction that is used throughout the grammars for vari- The reason for this is that selecting a different strategy
ous types of decision making. However, the key compo- means selecting a different view to the data, which in-
nent of any strategy is its determinant. It specifies the troduces a need to establish a link between the two
attributes that determine the location of the symbols views to make the presentation expressive.
designed by this strategy. For example, the functional How do design strategies work? When a grapheme is
descriptions of strategy strat1, which is of type disjoint instantiated as the result of satisfying some constraint,
interval (DI), and strategy strat2, which is of type corre- that grapheme is unified with a grammar called DESIGN-
lation (CORR), are illustrated below: STRATEGIES. This unification instantiates a symbol and
((type CORR) coordinates the positional properties of the grapheme
(det ((y date-on-market) with the determinant of the symbol. Grammar DESIGN-
(x date-sold))))
STRATEGIES has the following structure:
((type DI) ((grapheme
(det ((y address) ((of-symbol
(x1 date-on-market) ((of-strategy
(x2 date-sold)))) ((alt <design-strategies>)))
(det {^ of-strategy det})))
The strategy type, defined solely on the basis of charac- (alt (((x NONE) (y NONE)
teristics of the data, asserts some positional features of (x1 NONE) (x2 NONE)
the symbols designed by the corresponding strategy. (y1 NONE) (y2 NONE))
((type 1D-POINT)
The following strategy types are used: (y {^ of-symbol det y}))
Functionally-independent attribute (FIA) - the x or y ((type 2D-POINT)
attribute of the strategy functionally determines all other (x {^ of-symbol det x})
(y {^ of-symbol det y}))
attributes of the data. This type of strategy guarantees a ((type HORIZONTAL-INTERVAL)
unique strip (horizontal or vertical) for each symbol in (y {^ of-symbol det y})
the symbol set designed by this strategy. (x1 {^ of-symbol det x1})
(x2 {^ of-symbol det x2}))
Relation (REL) - the x and y components of the determi- ((type VERTICAL-INTERVAL)
nant together functionally determine all attributes in the (x {^ of-symbol det x})
data. This type guarantees a unique point location for (y1 {^ of-symbol det y1})
(y2 {^ of-symbol det y2}))
each symbol in a chart-like space. ((type LINE)
Location (LOC) - the pair of attributes bound to the x and (x1 {^ of-symbol det x1})
(x2 {^ of-symbol det x2})
y components of the determinant form a geographic lo- (y1 {^ of-symbol det y1})
cation. The symbols are shown on maps. (y2 {^ of-symbol det y2})))))))
Disjoint interval (DI) - the triple of attributes bound to Alternation <design strategies> consists of the FDs rep-
the x, y1, and y2 components of the determinant are resenting all potential design strategies. This list is gen-
characterized to form a disjoint interval. This strategy erated on the fly by analyzing the characteristics of the
guarantees a unique interval location for each symbol. data, which include functional dependencies, data types
Correlation of two, three or four attributes (CORR, COOR3, (e.g. nominal vs. ordinal vs. quantitative), and compo-
CORR4, resp.) - there are no functional dependencies site data types such as location and interval. First, the
from the attributes in the determinant to the other attrib- grammar prescribes that the symbol’s determinant
utes in the data. Strategies of type CORR are realized by should unify with the determinant of the strategy. The
symbols that occupy a point in the space, type CORR3 second alternation coordinates the positional properties
strategies are realized by horizontal or vertical interval of the grapheme with the determinant of the symbol. For
bars, and CORR4 strategies are realized by lines. None of example, the first alternative corresponds to the case
those strategies guarantees uniqueness of the position of when the grapheme does not use its own positional
any symbol in the set, i.e. the symbols may overlap. properties but instead is a “satellite” that annotates an-
other grapheme in the symbol. The other alternatives
Why use design strategies? The positions of the symbols
describe the cases when the grapheme represents a point
determine how the space is utilized. By selecting a strat-
5 08/08/12
A free (unbound) attribute - realized by a symbol de-
signed by any strategy applicable to the data.
The description of the four types of constraints summa-
rizes the SKETCH-MAPPINGS grammar. This grammar
translates the informational constraints of the sketch into
the common language of the design representation so
that the constraint could be unified with one of the strat-
egies and subsequently passed through the COMPOSITION
and COMPLETE-DESIGN grammars.
To illustrate the grammar, below we show the FD of the
grapheme-in-space constraint from Figure 3 and a frag-
Figure 3. A sketch created in SageBrush ment of the grammar.
((cat grapheme-in-space)
in a 1D-space (e.g. a table), a point in a 2D-space (e.g., a (grapheme ((type horizontal-interval-bar)
chart or a map), a horizontal interval in a 2D space, a (id GRA-10000)
vertical interval in a 2D space, or a line in a 2D space. (y ((attr team)
(x1 ((attr start-time)))
Sketches (color ((attr vehicle-type)))))
(space ((type chart)
SageBrush (Roth et al., 1994) is an interface by which (id CHART-1000))))
users drag primitive design elements from palettes and
arrange them into sketches. The primitive elements are ((cat grapheme-in-space)
(grapheme
spaces, graphemes and data attributes. The graphemes ((of-symbol
are placed within spaces while the data attributes are ((of-space {^ ^ ^ space})))))
mapped to grapheme properties such as position and (alt (((space ((type chart)))
(grapheme
color, to space encoders such as X and Y axes, or just left ((alt
unbound. Figure 3 shows a sketch with a chart drawn (((type mark)
from the palette on the left side of the interface and an ...)
interval bar dragged from the top palette. Three attrib- ((type horizontal-interval-bar)
(of-symbol
utes are mapped to the properties of the bar (its y, x1- ((of-strategy
positions, and color), and three attributes, end time, du- ((alt (((type DI))
ration and cargo-weight, are not bound. ((type CORR3))))))))))
((type vertical-interval-bar)
Analysis of the sketches shows that four types of con- ...)
straints can represent each sketch: ...)))))
((space ((type map)))
An empty space - created for each space in the sketch ...)
that has no graphemes in it and no attributes mapped to ...))))
its axes. The type of the space imposes a constraint on Let us see how the two grammars, SKETCH-MAPPINGS
the type of the strategy. For example, a map can be sat- and DESIGN-STRATEGIES, would unify with this con-
isfied only by a symbol designed by a strategy of type straint. Grammar SKETCH-MAPPINGS hooks the graph-
LOC, a table - by a strategy of type FIA, and a chart - by
eme's symbol to the space in which the grapheme was
any type of strategy but LOC. placed. Next, for the combination of chart (space type)
An attribute on an axis - created for any attribute and horizontal-interval-bar (grapheme type) the gram-
dropped on the axis of a space. This constraint can be mar constrains the strategy to types DI and CORR3. Then
satisfied by symbols whose determinant has the attribute the FD is unified with grammar DESIGN-STRATEGIES.
dropped on the axis as a value of the corresponding The constraints imposed so far make possible the unifi-
symbol position. The space type imposes a constraint on cation only with strategies of types DI and CORR3 and
the type of the strategy. whose y and x1 determinants are bound to the attributes
A grapheme in a space - created for each grapheme team and start-time, respectively. Once a strategy is se-
placed within a space. The grapheme and space types lected, it imposes additional constraints: a fixed strategy
impose constraints on the strategy type. For example, an type and a mapping of all positional properties of the
interval bar can only realize strategies of type DI and grapheme to concrete data attributes. Similar rules guide
CORR3. Any mappings of attributes to positional proper- the realization of all other types of sketch constraints.
ties of the grapheme impose constraints on the determi-
nant of the symbol. Tasks
Tasks are the other type of design requirement that we
explored in the context of the unification-based ap-
6 08/08/12
proach. Tasks are specified as the operations that the Correlate a number of attributes. All attributes must be
user should perform on some, yet undefined, representa- mapped to properties of the same symbol; the properties
tion of the data to achieve a given data exploration or a have to be positional or retinal.
communicative goal. An example of a data exploration Tasks have a hierarchical structure obtained by decom-
goal is "find the addresses of all houses that were on the posing the goal into data manipulation and value access-
market within a given time interval." For such a goal, ing subtasks. The value accessing subtasks are dominat-
the user will need to search the set of houses by inspect- ed by the entity manipulation tasks in the sense that the
ing their date-on-market and date-sold attributes to find former are performed as part of the latter. For example,
a house that was on the market in the specified interval, in the example above, the search task for houses domi-
and then look up its address attribute. These are the con- nates the attribute access task on the selling price attrib-
ceptual tasks that the users will need to perform in order ute. In addition, there are dependency relations between
to accomplish the goal. By designing a graphic, the con- the entity manipulation tasks. For example, before look-
ceptual tasks will be realized as specific perceptual and ing up the attribute of a house of interest, the user will
cognitive operations. The designer's goal is to make have to find this house. In this case, the lookup task de-
those operations maximally effective. pends on the search task. Those types of relations are
Our analyses revealed that tasks are composed of opera- captured by the following three organizational opera-
tions (or subtasks) at two levels: value accessing and tors:
entity manipulation; a distinction not evident in Sequence. Each subtask of a sequence operator depends
Casner’s work. on the previous one and therefore the subtasks have to
Each value-accessing task produces a value in one of be executed in the order given in the sequence.
three possible ways: Disjoint subtasks. The subtasks are independent of each
Evaluate a constant (e.g., evaluate the date April 16, other and can be executed in any order.
1999); Conjoin subtasks. The execution of each subtask de-
Access the value of an attribute (e.g., evaluate the date- pends on the execution of the other subtasks and there-
on-market of a given house) fore all subtasks have to be executed in parallel. An ex-
Compute a value by applying some arithmetic operator ample of mutually dependent subtasks is a pair of search
such as total and max to the values accessed by other tasks for the same object by two different attributes.
operations (e.g., compute the difference between asking- We believe our treatment of the structure of tasks is
price and selling-price). more principled than Casner’s (1990) vector representa-
The entity manipulation tasks work at the level of ob- tion based solely on the co-occurrence of objects and
jects (as opposed to values) and result either in identify- attributes in different tasks.
ing an object or a set of objects by conditions imposed The input FD for each task includes its subtasks and any
on them (the SEARCH task) or in asserting some predicate relevant data characteristics. Grammar TASK-MAPPINGS
about objects that are already identified (the LOOKUP, creates and uses attribute-value pairs that represent the
COMPARE and CORRELATE tasks). state of the interpretation process. Those pairs reflect
Search for an object by conditions on some of its attrib- concepts known as topic and focus in the field of natural
utes. The attributes should be mapped to properties that language processing. The topic of a sentence is the in-
allow direct access from the data value to a narrow formation that has already been introduced in the dis-
space where the object's symbol is located. Positional course and is included in the current sentence to glue the
attributes are most suitable but retinal properties that are past discourse with the introduction of new information
processed pre-attentively such as color are also good (the focus). In our design, the topic is represented by an
candidates; text labels are ineffective for search. FD, which contains the constraints imposed on objects
by the processing of previous tasks, i.e. tasks that pre-
Lookup the attribute of an object. The attribute should
cede the current one in a SEQUENCE operator. The focus
be mapped to a property that allows easy mapping from
contains the constraints imposed on the same object by
graphical values to data values. Labels and positional
processing the current task. The mapping rules use the
properties are good candidates as well as color and
topic and the focus to make sure that the design deci-
shape for attributes of enumerated types, i.e. data types
sions that satisfies the current task are consistent with
that have a small number of values such as sex or race.
design decisions made for earlier tasks. Thus, while en-
Compare attributes of objects. Two conditions should be tity manipulation tasks are realized by selecting graph-
satisfied: the attributes are mapped to graphical proper- ical techniques, the organizational tasks primarily estab-
ties using the same encoding rule such as a common lish relations between states of their constituent subtasks
axis or common retinal property; and the graphical (cf. the explanation of the sequence and conjoin sub-
property allows effective comparison (e.g. by position). tasks below).
7 08/08/12
The sample task from the beginning of this section is subtask is realized by a FIA strategy whose determinant
represented by the following FD: is the same functionally-independent attribute. The third
((cat sequence) alternative is the least efficient way of connecting the
(sub1 ((cat conjoin) two symbols because it requires that the users look up
(sub1 ((cat search)
(op1 ((cat attr-value)
the label in the topic symbol, and then using its value
(attr date-on-market) find the strip that contains the focus symbol.
(object ?house)
(op2 ((cat value))))) The conjoin task (subtask-1 of the sequence task) im-
(sub2 ((cat search) poses the constraint that the common objects in the foci
(op1 ((cat attr-value) of its subtasks should be realized by the same symbol.
(attr date-sold)
(object ?house)))
This constraint stems from the fact that tasks can be
(op2 ((cat value))))))) performed in parallel only when their operands are sim-
(sub2 ultaneously in the user's focus.
((cat lookup)
(op ((cat attr-value) The grammar for search tasks imposes a common en-
(attr address) coder on the realization of the operands, which are value
(object ?house)))))) accessing subtasks, as well as preferential constraints on
Since we don’t have space to show grammar fragments, that encoder. Positional (x or y) and retinal pre-attentive
we will give just a hint of how the tasks are mapped to properties (e.g. color) are preferred to other retinal prop-
design constraints. The sequence task (the outmost task erties. Textual properties are only a last resort.
of our example) requires that the user is able to effec- The attribute-value subtask requires that there is a
tively connect the symbol realizing the focus of each grapheme as part of a symbol for that object and one of
subtask with the topic of the following subtask. This is the grapheme properties encodes that attribute. The ob-
possible in one of the following three ways. (1) The two ject of an attribute-value task is linked to the focus of
symbols are identical. (2) The two symbols are different the super-ordinate entity manipulation task, thus ensur-
but realized by the same FIA strategy (in this case the ing that all constraints imposed by any of the subtasks
user will be able to match the symbols by virtue of the will be consistent. For example, if the search task selects
fact that they lie in the same strip). (3) The topic symbol an X encoder, the attribute could only be encoded by an
has a label for the functionally-independent attribute of X position of the grapheme.
the data-set and the symbol for the topic of the second As in the case of sketches, each instantiated grapheme is
unified with grammar DESIGN-STRATEGIES. Given the
grammars described so far, at least the following realiza-
tions are possible:
One symbol with one grapheme of type horizontal-
interval-bar: the y-position encodes address, x1 and
x2 encode date-on-market and date-sold.
One symbol with two graphemes: a mark whose x
and y positions encode date-on-market and date-
sold, and a label, which encodes address.
One symbol with two graphemes: a mark whose x
and y positions encode date-on-market and address,
and a label, which encodes date-sold.
Figure 4. The tasks are realized by an interval bar The first design (Figure 4) is definitely the most effec-
chart. tive one. It employs only one grapheme and exploits a
DI strategy, which allocates a unique interval location
for each data object. The second one might also be ef-
fective but the CORR strategy does not guarantee unique
locations for the symbols, which in the case of poor data
distribution may cause the labels to overlap. The third
one (Figure 5) is ineffective because it employs text to
encode the attribute of a search task.
Composition
Composition merges elements of the design instantiated
in response to different requirements. It makes visuali-
Figure 5. The tasks are realized by a labeled mark. zations more compact, coherent, and effective. Four
8 08/08/12
((cat vertical-alignment)
(grapheme-1
((of-symbol
((of-space
((x-encoder
{^ ^ ^ ^ grapheme-2
of-symbol
of-space
x-encoder}))))))))
Similarly, horizontal alignment unifies the y-encoders of
the two spaces.
Figures 6 and 7 illustrate grapheme composition. In the
sample sketch in Figure 6, the attribute street-number is
Figure 6. A sketch that results in two constraints: dropped on the y-axis of a chart. There is also a mark in
an attribute on encoder and a grapheme in space. the same chart with the selling price mapped to its x-
position and neighborhood mapped to its color. Clearly
this sketch is represented by two constraints: attribute-
on-encoder and grapheme-in-space. The two graphemes
that satisfy those constraints can be merged into one to
produce the graphic in Figure 6.
Completing the design
The final step in the design process is running all graph-
emes through the COMPLETE-DESIGN grammar. This
grammar organized by grapheme type checks whether
all required properties have values and if the values are
consistent. It also realizes all unbound attributes to reti-
nal or textual properties.
Figure 7. The two instantiated graphemes are
merged into one. Discussion
types of composition are described below along with Using functional unification for graphic design offers
their FUGs. The grammars apply to two graphemes, some clear benefits, the most important one being that it
grapheme-1 and grapheme-2, where grapheme-1 is the supports thinking about the design in a systematic way.
grapheme just instantiated and grapheme-2 varies Every factor that contributes to the selection of graph-
among previously instantiated graphemes. ical techniques is considered from the perspective of
imposing constraints on some design elements. These
Merging graphemes. The two graphemes can be merged
constraints are expressed declaratively as FUGs. Our
into one. Merging graphemes is expressed by the fol-
approach was enabled by careful analysis of the re-
lowing grammar:
quirements and the elements of graphic designs. Our
((cat merging-graphemes)
(grapheme-1 {^ grapheme-2})) design representation is informed by Bertin's (1983)
semiological analysis of graphics, Mackinlay's (1986)
Clustering. The symbols of two distinct graphemes can
relational approach, and the long-term research and de-
be unified (e.g. a mark and a text annotation to it):
velopment effort of the Sage project (Roth et al., 1997).
((cat cluster-composition)
(grapheme-1 ((of-symbol {^ ^ grapheme-2 We looked at two radically different ways of expressing
of-symbol})))) user needs - sketches and tasks. Sketches, based on
Double axis composition. The distinct symbols of two SageBrush work (Roth et al., 1994), convey the user
graphemes can be placed in the same space: needs in the form of graphical elements and relations.
((cat double-axis-composition) Although those constraints are in a graphical language,
(grapheme-1 the designer still needs to reason about proper and con-
((of-symbol sistent mapping of attributes that are not bound to con-
((of-space {^ ^ ^ grapheme-2
of-symbol
crete graphical properties. On the other hand, tasks,
of-space})))))) based on Casner's work (1990), are goal and process
Alignment. The distinct spaces of two graphemes can oriented rather than graphics oriented. The designer
share a common positional encoder. The alignment can needs to reason about what graphical techniques would
be horizontal (shared y-axis, cf. Figure 1) and vertical support the tasks and the relations between them.
(shared x-axis):
9 08/08/12
The functional unification approach described in this Bertin, J. 1983. Semiology of Graphics: Diagrams, Net-
paper has been employed in the development of two works, Maps. Madison, Wisconsin: The University of
systems that include automated graphic design. Sage Wisconsin Press.
automatically generates graphics that satisfy usrer‘s Casner, S.M. 1991. A Task-Analytic Approach to the
sketches. Sample visualizations designed by Sage can be Automated Design of Information Graphic Presenta-
found at http://www.cs.cmu.edu/~sage/sample.html. tions. ACM Transactions on Graphics, 10(2), 111-151.
AutoBrief is an automated multimedia explanation sys-
Chuah, M., Roth, S., Kolojejchick, J., Mattis, J., and
tem (Kerpedjiev et al., 1997). It employs communicative
Juarez, O. 1995. SageBook: Searching Data-Graphics by
planning, media allocation, text and graphic
Content. In: Proceedings SIGCHI '95, Denver, CO, pp.
microplanning, text realization, and graphic design. The
338-345.
graphic microplanner maps communicative goals allo-
cated to graphics into conceptual tasks (Kerpedjiev and Elhadad, M. 1992. Using Argumentation to Control Lex-
Roth, 2000). The graphic designer generates a presenta- ical Choice: A Functional Unification Implementation.
tion as part of a text and graphics explanation that satis- Ph.D. dissertation, Computer Science Dept, Columbia
fies those tasks. Sample visualizations designed by University.
AutoBrief are available at Kay, M. 1979. Functional Grammar. In Proceedings of
http://www.cs.cmu.edu/~sage/ab-tour/start.html. We the 5th Meeting of the Berkeley Linguistics Society.
used FUF (Elhadad, 1992) as a functional unification Berkeley Linguistics Society.
engine for both systems. Kerpedjiev, S., Carenini, G., Roth, S. F., and Moore, J..
For future work we plan to explore how context affects 1997. AutoBrief: a multimedia presentation system for
presentations. By context we mean features of the envi- assisting data analysis. Computer Standards and Inter-
ronment that influence the way users interpret graphics. faces, 18, 583-593.
For example, the size of the display or any previous Kerpedjiev, S. and Roth, S. F. 2000. Mapping Commu-
visualization in the current session is a factor that poten- nicative Goals into Conceptual Tasks to Generate
tially might affect the choice of graphical techniques. Graphics in Discourse. In Proc. Int. Conf. on Intelligent
Returning to our design goals, our system development User Interfaces, New Orleans, LA, (in print).
effort confirmed that functional unification is a good Mackinlay, J. 1986. Automating the Design of Graph-
formalism for tackling the visualization design problem. ical Presentations of Relational Information. ACM
In both types of design requirements we were able to Transactions on Graphics, 5(2), 110-141.
formulate the design knowledge in the form of FUGs
Roth, S. F., and Mattis J. 1990. Data Characterization
and both systems generate graphical presentations in
for Intelligent Graphics Presentation. Proc. SIGCHI'90,
about 5-10 seconds. Compared to the older version of
Sage, the unification-based one is able to complete a Seattle, WA, ACM, pp. 193-200.
much larger number of design requests imposed by user Roth, S. F., Kolojejchick, J., Mattis, J., and Goldstein, J.
sketches. In fact, the system very rarely fails to design a 1994. Interactive Graphic Design Using Automatic
graphic from a consistent sketch. Those observations Presentation Knowledge. Proc. SIGCHI'94, Boston,
rate the system pretty well on the scale of expressive- MA, ACM, pp. 112-117.
ness and efficiency. Since all the knowledge employed Roth, S. F., Chuah, M. C., Kerpedjiev, S., Kolojejchick,
by the designer is represented as FUGs, we achieved J. A., and Lucas, P. 1997. Towards an Information Vis-
uniformity. We gained some confidence about the ex- ualization Workspace: Combining Multiple Means of
tensibility of the grammars after members of our group Expression. Human-Computer Interaction Journal, Vol.
requested incorporating pieces of specific design 12, Numbers 1& 2, 131-185.
knowledge and we were able to fulfill those requests in Shieber, S. 1986. An Introduction to Unification-based
half to one hour. However, to better evaluate the exten- Approaches to Grammar. Center for the Study of Lan-
sibility of the grammars, we would like to extend our guage and Information. Stanford, CA, 105 p.
design languages with new types of layout disciplines
(e.g., polar charts) and new types of graphemes (e.g., Zhou, M., and Feiner, S. 1998. Visual Task
tick marks). We cannot really claim high usability of the Charactererization for Automated Visual Discourse Syn-
formalism since only one person familiar with FUF en- thesis. Proc. CHI-98, Los Angelos, CA, 392-399.
coded all the grammars. We hope our future work will
cast additional light on the usability issue.
References
10 08/08/12
Related docs
Other docs by HC12080810742
Prazosin Reduces Nightmares and Other PTSD Symptomns in Combat ... - PowerPoint
Views: 1 | Downloads: 0
