Behavior And Scenario Modeling For Real-Time Virtual ENVIRONMENTS _2000_

BEHAVIOR AND SCENARIO MODELING FOR REAL-TIME VIRTUAL ENVIRONMENTS by Peter Jason Willemsen An Abstract Of a thesis submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree in Computer Science in the Graduate College of The University of Iowa May 2000 Thesis supervisor: Professor Joseph K. Kearney 1 ABSTRACT Virtual environments provide a powerful medium for studying human behavior. To exploit the potential of virtual environments as laboratories for psychological experimentation, we must be able to control the dynamics of complex virtual environments populated with autonomous entities. In experiments, the right things must happen at the right time and place. My research addresses two essential components required to create engaging, realtime, virtual environments: (1) Representations of the physical environment adapted to the needs of autonomous behavior programming in urban settings, and (2) An interpreted scripting language for online programming of scenario control processes. I introduce the Environment Description Framework (EDF) and the Scenario Description Language (SDL) as parts of an integrated system to address intertwined problems of modeling behaviors and scenarios in structured, urban environments. We create autonomous behaviors that model the dynamic entities commonly found in these environments: traffic lights, motorists, bicyclists, and pedestrians. We use the same framework to model scenario directors responsible for orchestrating the actions of multiple, autonomous behaviors. The EDF defines a road and intersection based representation that supports programming of autonomous agents in real-time virtual environments. As its core function, 2 EDF gives structure and meaning to physically-based components while overlaying logical and spatial relationship information on the geometric structure of the environment. Our environment model is distinct in that it embeds behavioral constraints and inter-object relationships into the environment structure. SDL is used for interactive modeling and prototyping of scenario directors in realtime virtual environments. SDL is an interpreted scripting language that translates scenario directives into state machines which are then inserted into the simulation execution framework. SDL advances standard scripting languages by clearly separating statements that describe what happens from statements that determine when things happen. SDL is tightly bound to simulation activity and forms a glue between the EDF, autonomous behaviors, and the scenario directors that influence the environment. Abstract approved: Thesis supervisor Title and department Date BEHAVIOR AND SCENARIO MODELING FOR REAL-TIME VIRTUAL ENVIRONMENTS by Peter Jason Willemsen A thesis submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree in Computer Science in the Graduate College of The University of Iowa May 2000 Thesis supervisor: Professor Joseph K. Kearney Copyright by PETER JASON WILLEMSEN 2000 All Rights Reserved Graduate College The University of Iowa Iowa City, Iowa CERTIFICATE OF APPROVAL PH.D. THESIS This is to certify that the Ph.D. thesis of Peter Jason Willemsen has been approved by the Examining Committee for the thesis requirement for the Doctor of Philosophy degree in Computer Science at the May 2000 graduation. Thesis committee: Thesis supervisor Member Member Member Member The highest reward for a person’s work is not what they get for it, but what they become by it. —John Ruskin ii ACKNOWLEDGEMENTS I am thankful that I have been fortunate enough to be blessed with the intelligence and endurance to make it through this process. It has been a tremendous experience for me. For this success and accomplishment, I acknowledge and thank the Lord! I would like to thank my wife Renee, for overwhelming love and support over these last several years. I could not have made it this far without her companionship, and for this I am deeply indebted. I love you, Renee! I really don’t know how I would have made it through this demanding and trying process without my advisor Joseph Kearney. He has been and continues to be a mentor and friend showing me an almost perfect example of what it truly means to be a professor. Thanks Joe! I would also like to thank my family (Dad, Mom, Matt, Renee, Emily, Jim, Jeanne) for the tremendous love, support, and encouragement they have given to both myself and my wife throughout this process. Additionally, there have been many wonderful people from all over the world that I have been lucky enough to meet and work with in graduate school. I’ve enjoyed all of your company and have learned many things from you. In particular, I’d like to thank Jim Cremer, Paul Henning, Apurv Mathur, and Joan Severson. And finally, I’d like to mention Bruce Bennett, my high school computer science iii teacher who really got me started on this track of discovery. It wasn’t that long ago that I was typing in BASIC programs on Mr. Bennett’s lonely Apple IIe up in the library of my high school. Those were the days. iv ABSTRACT Virtual environments provide a powerful medium for studying human behavior. To exploit the potential of virtual environments as laboratories for psychological experimentation, we must be able to control the dynamics of complex virtual environments populated with autonomous entities. In experiments, the right things must happen at the right time and place. My research addresses two essential components required to create engaging, realtime, virtual environments: (1) Representations of the physical environment adapted to the needs of autonomous behavior programming in urban settings, and (2) An interpreted scripting language for online programming of scenario control processes. I introduce the Environment Description Framework (EDF) and the Scenario Description Language (SDL) as parts of an integrated system to address intertwined problems of modeling behaviors and scenarios in structured, urban environments. We create autonomous behaviors that model the dynamic entities commonly found in these environments: traffic lights, motorists, bicyclists, and pedestrians. We use the same framework to model scenario directors responsible for orchestrating the actions of multiple, autonomous behaviors. The EDF defines a road and intersection based representation that supports programming of autonomous agents in real-time virtual environments. As its core function, v EDF gives structure and meaning to physically-based components while overlaying logical and spatial relationship information on the geometric structure of the environment. Our environment model is distinct in that it embeds behavioral constraints and inter-object relationships into the environment structure. SDL is used for interactive modeling and prototyping of scenario directors in realtime virtual environments. SDL is an interpreted scripting language that translates scenario directives into state machines which are then inserted into the simulation execution framework. SDL advances standard scripting languages by clearly separating statements that describe what happens from statements that determine when things happen. SDL is tightly bound to simulation activity and forms a glue between the EDF, autonomous behaviors, and the scenario directors that influence the environment. vi TABLE OF CONTENTS Page LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii CHAPTER 1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 1.2 1.3 The Day in the Life of Murphy Scenario . . . . . . Motivation . . . . . . . . . . . . . . . . . . . . . . The Trilogy - Environment, Behavior, and Scenario 1.3.1 Environment Representation . . . . . . . . 1.3.2 Behavior Modeling . . . . . . . . . . . . . 1.3.3 Scenario Modeling . . . . . . . . . . . . . Contributions . . . . . . . . . . . . . . . . . . . . Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 4 5 6 7 8 9 1.4 1.5 2 RELATED WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1 Environment Representations . . . . . . . . . . 2.1.1 Scene Graph, Polygonal Representations 2.1.2 Simulated Environment Representations Directable Autonomous Behavior Modeling . . Languages for Animation and Simulation . . . . 2.3.1 Scenario Scripting Languages . . . . . . Interactive Fiction, Story-Telling, and Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 11 13 19 20 21 22 2.2 2.3 2.4 3 DIRECTABLE, SEMI-AUTONOMOUS BEHAVIORS IN HANK . . . . . 24 3.1 3.2 Hank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Modeling Directable, Semi-Autonomous Behaviors . . . . . . . . . . 28 4 ENVIRONMENT DESCRIPTION FRAMEWORK: AN INTERFACE TO THE WORLD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 vii 4.1 4.2 4.3 4.4 4.5 4.6 Curvilinear Coordinates . . . . . . . . . . . . . . . . . . . . . Segments - The Pavement Under Our Feet . . . . . . . . . . . 4.2.1 Compositing - Hierarchical Segment Descriptions . . . 4.2.2 Segments in the EDF . . . . . . . . . . . . . . . . . . Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Interpretation of Curvilinear Coordinates . . . . . . . . 4.3.2 Lanes . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Range Attributes . . . . . . . . . . . . . . . . . . . . 4.3.4 Features . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.5 Additional Queries for Interrogating Road Information Intersections . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Intersection Boundaries - Exterior Geometry . . . . . . 4.4.2 Corridors . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3 Dependencies - Internal Intersection Constraints . . . . 4.4.4 Traffic Control . . . . . . . . . . . . . . . . . . . . . . 4.4.5 Zero Length Intersections . . . . . . . . . . . . . . . . 4.4.6 Hierarchical Intersections - Internal Routing . . . . . . 4.4.7 Intersection and Corridor Behavior Queries . . . . . . Occupancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Occupying Space in the EDF . . . . . . . . . . . . . . 4.5.2 A Witness to Occupancy . . . . . . . . . . . . . . . . 4.5.3 Sophisticated Witnesses . . . . . . . . . . . . . . . . . 4.5.4 Level of Detail Occupancy Queries . . . . . . . . . . . Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Path Augmentation and Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 37 40 41 45 47 48 50 51 53 55 56 56 64 67 68 69 70 72 73 74 76 77 81 82 5 SCENARIO MODELING WITH A SCENARIO SCRIPTING LANGUAGE 84 5.1 5.2 SDL Basics - Iterative, Interpreted Programming . . . 5.1.1 Statement Execution Semantics . . . . . . . . Activities - Instantaneous Commands . . . . . . . . . 5.2.1 Simulation State and EDF Query Access . . . 5.2.2 Dynamic Object Creation and Direction . . . 5.2.3 Conditionals . . . . . . . . . . . . . . . . . . 5.2.4 Iteration . . . . . . . . . . . . . . . . . . . . 5.2.5 Function Definitions . . . . . . . . . . . . . Monitors - Computational Language Constructs . . . 5.3.1 Continuous, Interval-based Actions . . . . . . 5.3.2 Impulsive, Event-based Actions . . . . . . . . 5.3.3 Sensors and the Event Algebra . . . . . . . . 5.3.4 Nesting - Mixing Interval and Event Monitors 5.3.5 Parameterized Scenario Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 86 88 90 91 96 96 97 98 102 105 105 113 120 5.3 viii 6 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.1 6.2 6.3 Environment Models and Representations . . . . . . . . . . . . . . . 127 Scenario Scripting with an Interpreted Language . . . . . . . . . . . . 130 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 APPENDIX A EDF FILE FORMAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 A.1 EDF EBNF Specification . . . . . . . . . . . . . . . . . . . . . . . . 132 B EDF EXAMPLE - BIKETOWN . . . . . . . . . . . . . . . . . . . . . . . . 136 B.1 Biketown Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 C SCENARIO DESCRIPTION LANGUAGE . . . . . . . . . . . . . . . . . . 163 C.1 SDL EBNF Specification . . . . . . . . . . . . . . . . . . . . . . . . 163 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 ix LIST OF TABLES Table 4.1 Page The queries supported by segments in the EDF. All segments must support these queries. Queries are often parameterized by curvilinear coordinates to access exact surface detail. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Various types of lanes defined in the EDF. . . . . . . . . . . . . . . . . . . . . 49 EDF queries that support interrogation of range attributes and features on roads. 53 Additional queries supported by roads in the EDF. . . . . . . . . . . . . . . . . 54 Types of dependencies that can be used in the EDF to describe the inter-object relationships on corridors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Traffic control state. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Intersection queries relate connecting roads and corridors and provide traffic control state for corridors. Corridor queries provide access to dependency information and which intersections (in the case of internal intersections) connect to the corridor termination points. . . . . . . . . . . . . . . . . . . . . . . . . 71 Primitive point-based witness queries provided by the EDF. These queries use point-based witnesses for occupancy computations. . . . . . . . . . . . . . . . 75 Level of detail (LOD) witness queries for EDF roads, intersections, and corridors. LOD occupancy witness queries are computed on an as-needed basis. . . 80 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 Queries provided by paths in the EDF. . . . . . . . . . . . . . . . . . . . . . . 83 5.1 5.2 Operators defined in SDL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 SDL defined mathematical functions accessible through the math object. Additional functions can be bound to the math object and used in the SDL environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 x 5.3 Illustrative SDL commands for specifying locations in the simulation environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 SDL set notation for describing event lines and regions. . . . . . . . . . . . . . 109 The components returned in an event record describe the type of event, who instigated it, the event threshold or region value, and the time the event occurred.110 The LightSequencer scenario creates a succession of LightSequencer scenarios. Every LightSequencer directs the behavior of a single traffic light to synchronize its cycle to the approach of the driver. . . . . . . . . . . . . . . . . . 122 5.4 5.5 5.6 xi LIST OF FIGURES Figure 2.1 Page An image from the VUEMS environment editor illustrating the topology and geometry of roads and intersection. . . . . . . . . . . . . . . . . . . . . . . . . 17 A third person perspective of a scene from the Hank simulator. This scene shows our model of Iowa City and the University of Iowa campus. When subjects interact with the simulator, either by driving a car, or riding a bike, they are presented with a first person view of the world. . . . . . . . . . . . . . 25 A view from the virtual environment. A subject interacts with the simulated vehicles in a simulated rural town. . . . . . . . . . . . . . . . . . . . . . . . . 27 A vehicle navigates over a ribbon-like surface inquiring about the tangent and curvature properties at various points along the surface. . . . . . . . . . . . . . 33 Illustration of a curvilinear coordinate system. Two curvilinear coordinates are shown depicting different locations. . . . . . . . . . . . . . . . . . . . . . . . 35 Illustration of general segment properties. . . . . . . . . . . . . . . . . . . . . 38 An illustration of a hierarchical segment in the EDF. The complete segment is composed of three leaf segments [S4, S5, Segment3]. Segments Segment1 and Segment2 are parent segments. . . . . . . . . . . . . . . . . . . . . . . . . . . 40 A longer and more complex road can be built by compositing segments. In this example, only three different types of segments are used: straight, curve, and spiral. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Road width is subdivided laterally across the width of the road to form lanes. Lanes can be of varying types and in this example sidewalks, vehicle lanes, bicycle lanes, parking lanes, and medians are shown. . . . . . . . . . . . . . . 46 Curvilinear distance is calculated along the reference curve (shown in black). Both coordinates have the same distance parameter ( ), but different offsets ( , ). Dashed, green arcs represent the true arc-length of the curve at the different offsets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 xii   ¤ ¥¡ ¢ £¡ 3.1 3.2 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 Corridors cross intersections and connect incoming lanes to outgoing lanes. The corridors leaving the left and right roads are shown in red. . . . . . . . . . 57 Illustration of fixed and floating junctures placed along the intersection boundary. Red squares denote fixed junctures while green dots represent floating junctures. The corridors between the junctures are shown as well. . . . . . . . 62 4.9 4.10 The road axis connects to the fixed juncture (shown as a red square) and the floating junctures (shown as green circles) adapt to fit the offsets of the matching lanes. This three-part illustration shows the process of connecting a road to an intersection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.11 An illustration of the use of dependencies to negotiate a left turn. Vehicles travand may pose threats to objects traveling on corridor eling on corridor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.12 Example of how a zero-length intersection connects two roads with differing lane definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.13 Example of a hierarchical intersection definition. Note the internal intersections forming the pedestrian sidewalk intersections. . . . . . . . . . . . . . . . 70 4.14 Level of Detail (LOD) occupancy queries provide general solution to different occupancy needs. Witnesses at different LODs provide different detail. . . . . . 78 4.15 Illustration of path use by vehicles. Three paths are shown for three different vehicles. The paths cross into and through the intersection boundary. . . . . . . 82 5.1 Illustration of the two dimensions used to define SDL monitor commands. Monitor statements are predicated by either events or state-based queries and can execute once or periodically. . . . . . . . . . . . . . . . . . . . . . . . . . 100 State machine diagram that illustrates the necessary transitions and activation states for the aslongas monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . 102 State machine diagram for the whenever monitor. . . . . . . . . . . . . . . . . 104 The event algebra is based on how the values of successive state variables , ) relate to the threshold . . . . . . . . . . . . . . . . . . . . . . . . . 106 ( A region being described by the event algebra. . . . . . . . . . . . . . . . . . . 107   ¢    ¤ ¨¦ ¢ §¦ © ¨¦ 5.2 5.3 5.4 5.5 xiii 5.6 5.7 5.8 State machine diagram for the every monitor. . . . . . . . . . . . . . . . . . . 112 State machine diagram for the when monitor. . . . . . . . . . . . . . . . . . . 113 The basic behavior of the traffic light is to cycle between green, yellow, and red states. The behavior responds to a directive to synchronize the cyclic pattern so that at a time in the future ( ) the light will be at a specific point in the cycle (labeled desired cycle phase). In the example shown the cycle duration is lengthened to insure that the light will be at the transition from yellow to red . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 at time When the driver enters the intersection, the current message stream is discontinued, and as the driver leaves the intersection a new agent is created. . . . . . 124  "  $#!    $" !# 5.9 B.1 Illustration of the main Biketown intersection showing junctures, corridors, and internal pedestrian intersections. . . . . . . . . . . . . . . . . . . . . . . . 136 xiv 1 CHAPTER 1 INTRODUCTION 1.1 The Day in the Life of Murphy Scenario Some days, nothing seems to go right. It’s as if Murphy1 is walking with you at every step. As you leave your house, a bird casually uses your shoulder for target practice. When you drive your car onto the street, you notice that traffic has been reduced to a standstill. Once you finally do start moving, you manage to hit every red light on your way to your destination. Even when you turn at intersections, crowds of pedestrians enter the crosswalk, blocking your way. Just as you pull into the parking lot, the one remaining parking spot is being closed to repair a pothole in it. Yes, on certain days, you’re just lucky to make it to anywhere. Fortunately, situations like these don’t occur everyday. However, consider how we might go about fabricating such a situation in a virtual environment. As an initial example, consider the movie, The Truman Show [71]. The Truman Show did an excellent job of illustrating how scenarios could be constructed and maintained in a virtual environment. In this movie, the main character, Truman, has lived out his entire life in a controlled world created by Hollywood lights, cameras, and actors. All the cast members, except Truman, are in on the virtual world experience. The cast members attempt 1 As in Murphy’s Law. 2 to maintain story and plot to provide Truman with a real experience, but react to directions sent from the director of the show when situations merit his intervention. For instance, at the very moment Truman recognizes his long lost father, cast members from Truman’s own virtual world fill the scene and make it nearly impossible for Truman to actually reach his father. By the time the action dies down, Truman’s father has been swept away and removed from the scene by other cast members. The cast members’ actions at that moment were outside the story and plot line, but were required because the director commanded those actions to prevent Truman from realizing the true nature of his world. However, in a computer simulated, virtual world, we require autonomous agents, rather than actors, built to perform the necessary roles that the vehicles, pedestrians, and birds would play. We also require a means to orchestrate, or direct, the agents’ behaviors around the actions of our subject. Moreover, we require a rich representation of the physical environment in which our agents will be situated. The concepts presented in this thesis form parts of an integrated system that provide the structure for developing and investigating virtual environments in which scenarios and situations can be formed around a human subject. The long term objectives of this research are to capture experiences and create them with predictability in real-time, virtual environments. 1.2 Motivation My work is motivated by the use of virtual environments as a medium for investigating human behavior. Virtual environments present an exciting new medium for the 3 study of human behavior that combines the rigor of controlled laboratory experiments with the ecological validity of natural experiments. Virtual environments immerse subjects in worlds that appear physically real, but where conditions can be controlled. Moreover, virtual environments can realistically simulate dangerous circumstances without risking injury to subjects [19, 30, 36, 45]. To exploit the potential of virtual environments as laboratories for psychological experimentation, we must be able to control the dynamics of complex virtual environments populated with vehicles, pedestrians, bicyclists, and traffic lights. In experiments, the right things must happen at the right time and place. In addition, the activity should maintain an appearance of spontaneity; subjects should feel that they have complete freedom of action and that other entities in the environment are behaving normally. Events should be embedded in a natural flow of action and appear to be coincidental. Consider, for example, a study of the influence of cell phone usage on reaction time in crash avoidance situations. The study calls for the subject to drive down a busy urban street through a series of intersections regulated by traffic lights. At one intersection, an autonomous vehicle is to be poised on a crossing street, facing a red light. As the subject approaches the intersection (with the traffic light green in the subject’s direction), the scenario vehicle is to unexpectedly dart in front of the subject driven vehicle. Scenarios like these place strong constraints on the locations of autonomous agents and the timing of events in the simulation. The traffic light must be green as the subject approaches the intersection, a vehicle must be positioned on the crossing road prior to the 4 subject’s arrival at the intersection, and the scenario vehicle’s motion must be synchronized with the motion of the subject’s vehicle. In addition to these explicit constraints, there are implicit constraints on the surrounding traffic. For example, there must be a gap in front of the subject that provides space for the scenario vehicle to enter the intersection. However, because subjects drive at different rates, it is not possible to explicitly pre-program or script the paths of specific vehicles; the scenario must be created online, adaptively responding to subject behavior and the evolving simulation state. My research addresses two essential components required to create these types of engaging virtual environments: (1) Representations of the physical environment adapted to the needs of autonomous behavior programming in urban settings, and (2) An interpreted scripting language for online programming of scenario control processes. 1.3 The Trilogy - Environment, Behavior, and Scenario Our work is founded in reactive behavior modeling for virtual environments. We program autonomous behaviors to query the state and structure of the world and use that information to make decisions that will influence the agent’s movements and actions in the virtual world. For instance, a vehicle behavior for tracking a lane on a road may query the road to determine both the speed limit and the curvature at a particular point in the road. The vehicle behavior can then use that information to determine a steering angle and acceleration to keep the vehicle on the lane. The relationships between environment representation, behavior modeling, and scenario control are tightly interwoven. My thesis presents an integrated system in which the 5 environment representation, behavior models, and scenario scripting modules are used in conjunction with one another to program engaging and interactive virtual environments. 1.3.1 Environment Representation One aspect of my research focuses on how to efficiently model the geometric, topological, and logical information in urban environments. This information is required by programs that code autonomous pedestrian and vehicle behaviors, as well as by scenario control programs that orchestrate behaviors to create predictable experiences for participants. In the real world, driving is highly regulated; lanes channel traffic into parallel streams, signs posted alongside the road signify important changes to behavior, and lines painted on the surface dictate whether lateral movement is prohibited, or not. Road networks are formed where roads connect to intersections. Entrance into and movement through intersections is regulated by traffic control mechanisms and right of way conventions. Intersections route incoming lanes to outgoing lanes via natural corridors. All in all, movement over roads and intersections in everyday life is controlled by well-defined structures, societal rules, and automated regulation mechanisms to minimize congestion and accidents. Moreover, as humans, our perceptual apparatus easily sense the nearby objects on the roadways, and we use that information to avoid collisions and unnecessary contact with those objects. We understand the complex relationships between objects on roadways and within intersections. 6 My research encapsulates these environment characteristics into a well-defined and structured environment modeling framework that supports autonomous behavior and scenario programming through a set of run-time queries, structures, and inter-object relationships. 1.3.2 Behavior Modeling Autonomous agents and characters form the heart of an engaging, interactive virtual environment experience. Realistic agent behavior contributes to suspension of disbelief on the subject’s behalf and brings the subject into the virtual environment [51, 39]. The autonomous characters and behavior models we create for individual agents are instilled with a basic competence to perform required actions and with the agility to engage in realistic interactions with unpredictable subjects. To accept direction, we program agents with public interfaces through which they can receive messages from directors. For example, a vehicle can be instructed to turn right at the next intersection, to increase its driving speed, or to brake suddenly. Our directable behaviors are truly semi-autonomous because they combine a responsiveness to guidance with their normal autonomous activity. We model semi-autonomous agents that navigate urban environments, such as motorists, pedestrians, and bicyclists. These behaviors require rich access to environment information and state. For instance, a vehicle needs to know where the road is, how long it is, how the road path changes over its length, and where the road connects to an intersection. Moreover, our behaviors require sufficient knowledge about other agents in the environment. To program following behaviors, our agents need to be able to query the 7 environment to determine if other objects are on the road ahead of it. The environment model is tightly intertwined with the decision making processes of our autonomous behavior models. 1.3.3 Scenario Modeling To orchestrate activities, we devise director agents that monitor the progress of the simulation and manipulate the environment by sending instructions to other agents. Directors are pure behavior objects that have no visual embodiment in the virtual environment. They subtly orchestrate the scenario by guiding the actions of the autonomous agents who participate in events. Scenario directors also need rich access to the environment state and data. Directors must be able to monitor agents in the environment, locate where they are and where they’re going. Directors must also detect when specific events have occurred and conditions become true. We initially programmed scenario directors using our C++ based state machine language bindings. However, with time, we found our C++ based descriptions to be cumbersome and unnecessarily complicated for director programming. The scenario directors we were building were inherently more simple than our complicated vehicle and pedestrian behaviors. The code for our multi-level state transition networks had poor locality because activity code (what the state machine did) was separated from the state transition set and transition predicates (when the machine did it). The resulting programs obscured temporal relationships and event sequences. Moreover, we also found that the edit, compile, run cy- 8 cle was time consuming and inhibited quick and dirty experimentation with new direction techniques. We felt we needed a more script-like language, but wanted to retain the execution semantics of our state machine framework. In particular, we wanted to be able to easily create processes that conditionally execute actions that manage the behaviors of agents. When circumstances are right, these processes would, for a period of time run continuously in the simulation creating and destroying objects as needed, and sending directions to guide the behaviors of agents. Our goal is to support highly distributed control strategies. We envision simulations having many directors running at a time, each responsible for some aspect of the scenario, activated by simulation events, and continuously adapting agent behavior to the subject’s actions. My research bundles these concepts into a scripting language designed for scenario modeling. Our scripting language works directly with and is bound to our environment model, thus providing full support for behavior programming and scenario modeling in a virtual environment. 1.4 Contributions My work introduces the Environment Description Framework (EDF) and the Scenario Description Language (SDL) as parts of an integrated system to address intertwined problems of modeling behaviors and scenarios in structured, urban environments. The EDF is distinct in that it embeds behavioral constraints and inter-object relationships into 9 the environment structure. These constraints and relationships make it simpler to program autonomous behaviors. SDL advances standard scripting languages by clearly separating statements that describe what happens from statements that determine when things happen. SDL is tightly bound to simulation activity and forms a glue between the EDF, autonomous behaviors, and the scenario directors that influence the environment. 1.5 Organization Chapter 2 discusses the broad spectrum of related work that has shaped and influenced the direction of my research. Chapter 3 provides an brief overview of our simulation environment Hank for which this work is intended, and explains the execution semantics behind our behavior and scenario programming framework (HCSM). Chapter 4 describes the Environment Description Framework (EDF) for modeling urban environments. Chapter 5 introduces the Scenario Description Language (SDL) which forms the glue between EDF, HCSM, and Hank resulting in a powerful simulation environment. This thesis is wrapped up in Chapter 6 with a discussion of design decisions and directions for future research in environment representations and scenario languages. 10 CHAPTER 2 RELATED WORK This chapter reviews the related research that provides a foundation for our work. The work described in this thesis focuses on two areas: (1) urban environment representations for behavior modeling, and (2) scripting language support for scenario development in virtual environments. This work builds on research in autonomous agent architectures, believable character modeling, driving simulation, virtual environments, robotics, animation, interactive fiction, and computer gaming. In my master’s thesis, I investigated vehicle behavior models for real-time simulation environments [74]. I coded vehicle behaviors for lane tracking, lead vehicle following, intersection negotiation, lane changing, and passing. Controlling a vehicle’s physical movements along a road surface populated with other vehicles required that the behavioral model have access to detailed geometric and topologic information about the road network. Moreover, for following behaviors, the behavior programs needed fast access to the neighboring vehicles, whether they were in front, behind, or in the lanes to the left and right of the vehicle. Much of my work in coding a single vehicle behavior resulted in developing a set of run-time environment queries and structures that expressed the information necessary for vehicle behavior programs. It quickly became clear that programs modeling urban object behaviors (e.g. vehi- 11 cles, pedestrians, trains, bicyclists, etc. ) could benefit from a rich, run-time environment representation describing the geometry, topology, and inter-object relationships of an urban environment. A richly detailed and efficient representation benefits behavior programs in two ways: (1) behavior programs can be coded with less effort, and (2) frequently used computations can be processed by the environment representation rather than by each behavior model. 2.1 Environment Representations After my master’s thesis, my focus has been to simplify the effort required to program behavior and scenario models in real-time virtual environments. One way to achieve this is to provide a robust and explicit representation of the physical environment with data structures and queries useful for programming autonomous behaviors. Three dimensional scene layout representations are also important in artificial intelligence, 3D computer graphics, and immersive virtual environments. 2.1.1 Scene Graph, Polygonal Representations In real-time computer generated 3D environments and visualizations, geometry is typically represented as lists of triangles optimized for low level graphics hardware. Several application programming interfaces (APIs) have been built to abstract the difficulties involved with programming computer graphics at such low levels. APIs such as IRIS Inventor [72], IRIS Performer [59], Java3D [61], and VRML [69] are built on similar hierarchical scene graph representations. A scene graph is a directed acyclic graph structure 12 in which geometry, material properties, transformations, and behaviors are represented as distinct leaf nodes in the graph. Grouping nodes bundle related nodes together. The scene graph representation allows a programmer to concisely construct 3D scenes by grouping related object properties into distinct sub-trees. For instance, a group node could be placed in a scene graph to represent an entire stretch of a road model. Under the group node, the material representing the road along with all the triangles that make up the geometry of the road could be placed into material and geometry nodes, respectively. Likewise, a dynamic object, such as a simulation vehicle, can be placed into the tree under its own group node. Vehicle geometry, as well as position and orientation transforms can be stored under this group node. Scene graphs are designed to optimize the graphics rendering pipeline, and especially the culling procedures. As such, scene graphs do not immediately lend themselves to providing the detailed behavior-centric representations that we need to model and orchestrate behavior in urban environments. More generally, scene graphs are an inefficient way to represent spatial relationships that are tied to the environment structure. Nonetheless, perception-based behavior models may use the scene graph representations for synthetic vision calculations. In perception-based behavior models, computer vision techniques interpret how the geometry described in the scene graph occupies the space surrounding the autonomous characters. These calculations influence character motion [9] (e.g to avoid collisions). In our environment description framework, we chose to explicitly encode the structure and logic of the environment to simplify perception and concentrate more on the 13 actions performed by our behaviors. 2.1.2 Simulated Environment Representations Many virtual environment researchers choose to explicitly represent physical structures to fit the needs of their research and applications. For instance, in driving simulation environments, it is extremely natural to model roads, lanes, and intersections. In fact, there is an abundance of research on urban environment representation in the driving simulation and virtual environments literature. The detail and type of environment representation differs with research focus. In driving simulation research, the amount of information stored about the road network, as well as the types of representational data structures differ greatly depending on which aspect of driving simulation we discuss. For instance, vehicle dynamics computations require height and other surface characteristics, but have little use for how many vehicles are on the road, or where the next intersection is located. On the other hand, vehicle behavior models do need access to this information. To support high-frequency, real-time, vehicle dynamics terrain queries, Artz developed an analytic road terrain model that supplies fast and efficient access to surface attribute data, such as elevation and curvature. In this research, the terrain queries supply a single, human-driven subject vehicle with surface elevation on the road for each tire. Artz proposed four analytic segment types (straight, curve, transition spiral, and intersection) which, when connected together form smoothly adjoining road surface descriptions. The road descriptions produced by Artz correlate with civil engineering road design criteria([23]) and provide height change as well as superelevation modeling. In addition, 14 Artz utilized local curvilinear coordinate systems to access the surface attribute data on each segment [3]. The results from this research serve as the basis for the segment descriptions in the EDF. For my research, I am interested in simulators that model traffic and vehicles at the microscopic, or vehicle-by-vehicle, level. These types of driving simulators take on one of two forms: statistical simulation and agent-based simulation. Research with statistical vehicle models rely heavily on road density values and road flow rates to model the movement of vehicles on roadways. These models primarily represent the environment for statistical purposes only, modeling roads and intersections in terms of input and output density and flow characteristics using queuing theory to simulate the vehicles on the roadways. Statistical vehicle simulation models are also used in macroscopic traffic simulations [54, 37, 44]. In agent-based driving simulation, the focus is on modeling the decision making processes and detailed motion control of autonomous vehicles. In these simulators, roads, lanes, and intersections are modeled explicitly to provide autonomous behaviors with detailed information about the road geometry, interconnections, and the relative positions of objects on the road networks. Many driving simulation groups working with autonomous agents model road networks as a graph structure. Edges represent roads with one or more lanes and nodes represent intersections. Each road maintains internal lists of lanes to preserve lane adjacency orderings and also facilitate simple categorizations of vehicle occupancy in lanes [15, 50, 55, 29]. 15 In Sukthankar’s research on intelligent vehicle algorithms for tactical behavior in highway driving situations, only roads are modeled. Intersections are not represented as separate entities. An interesting design feature of this work is that roads are modeled as lane sets that allow connections to other lane sets. Thus, roads connect directly to other roads on a lane-by-lane basis. While urban intersections were not modeled, implied intersections existed between connecting lanes sets [60]. Most simulators explicitly model the geometry and topology of interconnections between lanes in an intersection. These researchers represent intersections as distinquished regions where roads meet. The pathways through the intersections that connect incoming to outgoing lanes provide explicit guidance data to vehicles [6, 25, 1, 49]. This representation is beneficial for a couple of reasons: (1) agents have a direct path to their destination, and (2) non-standard routes through intersections can more easily be modeled. However, the power of having complete control over all these routes comes at a high cost in modeling effort. Each connection between an incoming and outgoing lane has to be modeled separately. Bayarri automatically generates lane interconnections, thus eliminating the modeling cost for describing the connection between lanes [7]. Similar constructions are made in the Leeds simulator where researchers automatically construct complete road representations from the hierarchical road geometry descriptions provided by 3D road modeling programs. Their software generates a junction at the place where two roads meet. More than one junction can exist when multiple roads interconnect. Junctions serve both as the intersection and as the route between the roads’ lanes [4]. In both these simulators, vehicle behaviors 16 are the primary behavior modeled in the simulation. While automatic generation of routes is useful, we sometimes wish to restrict connections. For instance, consider routing vehicle lanes into pedestrian sidewalks, or vice-versa. Some researchers combine direct querying of environment state with perceptionbased behaviors as a hybrid approach to autonomous behavior modeling. The work by Donikian et al. represents the environment through use of explicitly modeled components. Their system, called VUEMS, uses a cubic spline representation to model the axial line (centerline) of a road in 2D. Roads are composed of different sections. Sections specify the number, type, and width of lanes for a particular section of roadway. Each section may characterize the road differently. For instance, a section may increase the number and width of lanes, while another section can decrease the number and width of lanes. Sections overlay onto the axial line of the road and abut lengthwise with each other. Sections allow road width to change over the length of the road. Where axial lines meet, automatic intersection boundary descriptions are formed based on road widths at the connection. Intersections are typed, for use by behaviors, as either being Y-intersections, T-intersections, or crossing intersections. In their simulations, behavioral information is encoded in a large semantic network that spans the entire simulation structure and merges behavioral information with geometric scene graph representations. An important aspect of their work is that occupancy on roads and intersections is determined by synthetic vision algorithms [20, 21]. Figure 2.1 shows a picture of a VUEMS editing session in which a four-way intersection has been created automatically 17 Figure 2.1: An image from the VUEMS environment editor illustrating the topology and geometry of roads and intersection. 18 from the intersection of two axial road lines and lane profiles. Road signs, crosswalks, and traffic lights are also represented as attributes of the road. The pink square on the left represents a building while the green polygon on the right represents a park1 . In her work on environment modeling for pedestrian behavior, Thomas augmented the VUEMS system by incorporating detailed connectivity and accessibility graphs for pedestrians. This work provides pedestrian behaviors greater freedom of movement within the simulation environment. For instance, if a pedestrian needs to cross the street with some urgency, rather than walking down the street to the crosswalk, it may have to jay-walk. Jaywalking represents the shortest path between where the pedestrian is and where it wants to go. Alternative paths exist, such as the path that would take the pedestrian down the street across the crosswalk and then to their destination. These decisions are parameterized by laziness and caution pedestrian attributes [63, 64]. Thalmann et al. utilize a scene-graph-like semantic network for representing the structure of the urban environment dividing the scene into quadrants, and then decomposing quadrants into blocks, streets, and junctions. This structure is augmented with smart objects. Smart objects encode actions and rules that describe what can be accomplished with the objects, as well as how the objects should be interacted with. This is a powerful way to encode behavior into objects prior to running a simulation [24]. My research relates directly to the discussed body of work and uses many of the same principles and guidelines for constructing urban environment representations. However, where our work in environment representation differs from related research is in how 1 VUEMS image included with permission from authors. 19 we infuse behavioral constraints and inter-object relationships into the environment structures. Our chief goal is to make behavior programs simpler to code resulting in more robust behavior. Moreover, we believe that behavior and database can exist in tight synergy, each working with the other. We achieve our goal by providing structure, logic, and relational information in an environment package that make information retrieval fast, efficient, and accessible to behaviors situated in the environment. 2.2 Directable Autonomous Behavior Modeling An important issue realized by both driving simulator researchers and autonomous agent researchers is that it is important to orchestrate autonomous behaviors in the environment to create controlled experiences. This is true for autonomous characters, driving simulation scenario control, interactive fiction, and gaming. Research involved with directable autonomous behaviors originated out of work in autonomous agent architectures for robotics and animation [58, 14, 42, 41, 39, 57, 38]. Blumberg and Galyean suggested a means for directing autonomous agents by sending directives to the agents and having them react to those commands [10, 8]. At about the same time, driving simulation researchers were working on creating autonomous driving behaviors that could be manipulated over the course of a simulation. The goal was to create replicable experiences adaptable to different human subjects in driving simulation experiments. This work resulted in the initial development of scenario control strategies and methodologies for real-time driving environments [13, 67]. Our initial work in scenario control resulted with us programming the behaviors of 20 scenario directors with languages like C, or C++, and then compiling them into the simulation system. This was time consuming and prohibited experimentation and play of different scenario control strategies. This motivated my research in developing an interpreted, scenario scripting language as a means to glue together our environment representation, autonomous behaviors, and runtime simulation system thereby providing a powerful means to interact with the running simulation. 2.3 Languages for Animation and Simulation Specialized languages provide abstractions to ease programming of complicated systems whether they are reactive physical systems, rigid-body dynamics systems, or autonomous agent behavior architectures [33, 32, 17, 31]. Perlin introduced the Improv scripting language to empower computer animators, allowing them to create behavior-based actors controlled by scripts. Improv is an actionbased scripting language in which well-defined actions can be created and later invoked. Actions are routines that control an object’s degrees of freedom for a period of time. For example, move four steps forward, or turn head ninety degrees to the left. Actions in Improv have definite beginnings and ends. Scripts can be contingent upon the completion of other actions. Also, scripts can run concurrently and compete for access to the actor’s articulated resources [52]. Alice is a programming environment designed to make 3D animation accessible to a large audience. Alice lets users populate a virtual environment with articulated, 3D objects. Objects in Alice have a repetoire of actions they can perform, but objects do not 21 do anything on their own. Alice provides its own scripting language based on the Python language to control and describe the movements of objects in the environment. Alice is primarily designed to be an animation environment allowing users the capability to specify actions that the objects should perform. Just as in Improv, actions in Alice are characterized by the way they control objects’ degrees of freedom. Alice’s scripting language provides users with constructs to create concurrent actions, as well as action sequences [16]. UnrealScript is a scripting language based roughly on a cross between Java and C++ that allows users of the Unreal 3D game to add to the behaviors and actions experienced in the game. UnrealScript is noteworthy because it defines a distinct notion of state and can be used to build autonomous agents [62]. 2.3.1 Scenario Scripting Languages Our goal in developing a scripting language for virtual environments is to bridge the gap between low level behavior programming and interactive scenario programming. Our language is a high-level language that is distinct from other scenario scripting languages in its focus on process generation and more importantly, by how it discriminates the language statements representing what we want to do from language commands that determine when we want to do those things. We are also motivated by the affect that scripting languages such as Perl, Python, Tcl, and JavaScript, have had on other domains, such as web programming, system programming, and rapid interface prototyping. Scripting languages such as these provide increased flexibility and freedom in programming in the environments for which they have 22 been created. They do this by being typeless and interpreted [48, 70, 40, 47, 75]. Moireau has created a behavior programming language based on hierarchical, parallel, transition states (HPTS). The HPTS language is compiled and designed for programming the behaviors used in a simulation, such as a vehicle or pedestrian [46]. HPTS can also be used to program scenario directors that would be compiled into the system. Scenario scripting languages for driving simulation have been proposed by Van Winsum and Donikian. Conceptually, these scripting languages function similarly to Perlin’s Improv system in that simulation objects can be commanded to perform certain actions and those actions have well-defined beginnings and ends. Moreover, actions in both languages can be scheduled to provide complex sequencing and concurrency. Van Winsum’s Scenario Scripting Language (SSL) is compiled and can also be used to program the behaviors of vehicles or pedestrians, and in this sense SSL serves a larger purpose than commanding behaviors to act [66, 68]. Donikian’s language is also compiled and uses Allen relations as the foundation for scheduling scenario directors [22]. The interval scripts paradigm proposed by Pinhanez presents an interesting means to describe complex situations and temporal activities in interactive story systems. Pinhanez’ work is also based on the Allen relations, used to describe the temporal relations of activities in interactive immersive environments [53]. 2.4 Interactive Fiction, Story-Telling, and Plot Although, our immediate research interest is in creating controlled situations for studies of human behavior in driving and bicycle riding simulations we anticipate the 23 techniques will also find application in other areas such as plot control in interactive fiction [5, 34], story-telling [51], training, and game play. The main distinction between scenario control methodologies and interactive stories is the focus of the simulation. In scenario modeling for driving simulation environments, the entire scenario revolves around the actions of the subject. The scenario director’s role adaptively coordinates object behaviors in response to actions of the human subject. In interactive story systems, the focus is on maintaining a specific plot, or feel to the story. The autonomous agents involved with the story do what they can to present a consistent and believable plot to the user. In his work on interactive story systems, Galyean proposed a model called narrative guidance. This model maintains plot and story as the story author intended by manipulating the presentation and flow of the subject’s interaction. The subject is free to interact with the environment, but is ultimately guided towards the story-line set forth by the author [26, 27, 73]. The application of narrative guidance to maintain plot and story is analogous to the role that agent direction plays in evolving a virtual environment scenario in a driving simulation. 24 CHAPTER 3 DIRECTABLE, SEMI-AUTONOMOUS BEHAVIORS IN HANK This chapter provides a brief overview of the Hank simulator architecture as it pertains to behavior and scenario modeling, and my thesis contributions. 3.1 Hank The goal of our work in virtual environments is to make things happen in environments populated with many autonomous characters. To accomplish this, we need four things: (1) a real-time simulator architecture that supports autonomous characters and scenario direction, (2) a behavior programming framework, (3) an environment description, and (4) a scenario component. The simulator architecture we developed to facilitate behavior and scenario modeling is called Hank. We use Hierarchical Concurrent State Machines (HCSM) to model autonomous behavior. The environment description framework and scenario language components will be discussed in Chapters 4 and 5, respectively. Hank is a real-time, ground vehicle simulator framework built to support experimentation with autonomous characters and scenario orchestration techniques. Characters, or simulation objects, as they are denoted in Hank, represent material entities such as vehicles, pedestrians, dogs, traffic lights, or gates. Simulator objects are composed of well-defined computational modules representing important sub-systems for controlling 25 Figure 3.1: A third person perspective of a scene from the Hank simulator. This scene shows our model of Iowa City and the University of Iowa campus. When subjects interact with the simulator, either by driving a car, or riding a bike, they are presented with a first person view of the world. the entity’s appearance, decisions, and actions. For instance, a vehicle can have a behavior module to make decisions about how fast to travel and how to drive on a road; a dynamics module that performs vehicle dynamics code to update the position and orientation of the vehicle geometry; and a geometric module that describes the physical aspects of the vehicle, as well as which parts of the vehicle are articulated. 26 The Hank architecture is designed to be very modular with respect to behaviors, dynamics, and geometry. Modules can be swapped and interchanged with other modules. For instance, a behavior module for controlling a vehicle can be swapped with a different behavior module as the simulation is running. The simulator architecture does not enforce rules about the ways in which modules inter-connect and relate to each other. This powerful feature promotes research and experimentation with autonomous behavior controllers, dynamics systems, geometry controllers, and visual display systems. Sets of simulator objects with the identical dynamics and geometry can be created each with different behaviors to investigate different control strategies. The architecture of Hank allows the simulator to receive input from instrumented vehicles, such as a bike, or car. Thus, human subjects can interact with the simulation through the instrumented components, mouse, or keystrokes. The core simulator software is based upon a hard real-time scheduling kernel that provides task scheduling and deterministic process execution. Process schedules describe the rates and frequencies at which the different simulator tasks run [43]. A visual task outputs high fidelity 3D graphics renderings of the simulated environment onto an immersive, three-screen projection environment that surrounds the subject. Figure 3.2 shows a picture taken from within the projection space of the virtual environment. The picture shows the subject riding the instrumented bicycle down a virtual rural Iowa town. Traffic on the roads react to the movements and position of the subject. Figure 3.1 presents an image from the simulator showing our Iowa City database model. The avatar of the human subject bicyclist 27 Figure 3.2: A view from the virtual environment. A subject interacts with the simulated vehicles in a simulated rural town. is located in the lower left of the image. Autonomous cars and a bus populate the street behind the subject. The intended design and functionality of Hank is to provide a platform with which we can research and experiment with behavior modeling and scenario control strategies and algorithms for human-in-the-loop, real-time, driving simulation environments. Hank utilizes a database component capable of modeling complex scenes with detailed terrain and cultural features. The road network portion of the database can be queried by simulator objects to determine information about important environment state, such as where the roads are, where the roads go, which roads connect to which intersections, or what objects are on the roads and in the intersections. The environment database component is discussed in complete detail in Chapter 4. 28 The behaviors of vehicles, pedestrians, and scenario directors in Hank are modeled with a powerful and flexible mechanism for designing high level, reactive and intentional behavior (HCSM). 3.2 Modeling Directable, Semi-Autonomous Behaviors HCSM stands for Hierarchical Concurrent State Machines (HCSM) [17] and is a programming formalism used to model reactive behavior within a simulation environment. HCSM tightly encapsulates behavior into process-based units called state machines. Each state machine may contain child state machines. The parent is responsible for grouping children according to whether they are concurrent or sequential. Concurrent state machines execute independently and in parallel. In a sequential state machine, only one active state machine is allowed to execute at any given time. Predicates logically connect sequential state machines and dictate when the transitions between active state machines will be fired. Each state machine may also utilize input and output parameters to serve as ways to pass data in and out of the state machine. A state machine’s activity function executes the programmed behavior and may produce output parameters. The parent state machines collect their childrens’ outputs and resolve conflicts between those outputs. Once conflicting outputs have been resolved, the parent sets its own output parameters. Behavioral directives (e.g. buttons and dials) can also be specified at the state machine level. HCSM does not enforce how directives are used. It is the responsibility of each parent to forward both input parameters and directives to child state machines. Behavioral control is passed up the hierarchy from leaf state machines to the top-most state machine 29 (i.e. root) which, in turn, specifies the final control values for the behavior. We use HCSM primarily for two purposes. One use is to program the behaviors of objects in our simulation environment. Examples of complete behaviors include human driving, bicycling and pedestrian locomotion [18]. Each of these behaviors can be composed of many lower-level behaviors; lower-level HCSM are generally responsible for a particular component of behavior. For instance, in the vehicle driving behavior, one HCSM is responsible for tracking lanes in the road while another maneuvers the vehicle across an intersection. Our second use of HCSM is to code scenarios into the simulation. To do this, we first build HCSM directives into our autonomous behaviors that accomplish the highly specific tasks required by scenario directors. In the vehicle behavior, for instance, we have built directives for modifying behavioral aggression, turning at intersections, changing lanes, slowing down, speeding up, running red lights, and stopping. Roughly, the purpose of scenario HCSM in the simulation are to monitor the constantly changing constraints and relationships important to the scenario and trigger behavioral events (i.e. send out directives) at appropriate times. HCSM provides a useful methodology for designing behaviors and programming scenarios. Our goal is to control large numbers of autonomous agents in highly dynamic environments to create predictable experiences. 30 CHAPTER 4 ENVIRONMENT DESCRIPTION FRAMEWORK: AN INTERFACE TO THE WORLD This chapter describes the Environment Description Framework (EDF). The EDF models road networks by defining a road-based representation that supports programming of autonomous agents in real-time virtual environments. As its core function, the EDF gives structure and meaning to physically-based components while overlaying logical and spatial relationship information on the geometric structure of the environment. The purpose of the EDF is to efficiently model the geometric, topological, and logical information required by programs that code autonomous pedestrian and vehicle behaviors. The EDF database is also used by scenario control programs that orchestrate behaviors to create predictable experiences for participants. The structures in the EDF are based on the nature and function of real road networks in urban environs. In the real world, driving is highly regulated; lanes channel traffic into parallel streams, signs posted alongside the road signify important changes to our behavior, and lines painted on the surface dictate whether lateral movement is prohibited, or not. Road networks are formed where roads connect to intersections. Real intersections are not free-for-alls; entrance into and movement through intersections is regulated and controlled. Intersections route incoming lanes to outgoing lanes via natural corridors. As humans, we have exceptional apparatus for perceiving these corridors and understanding the com- 31 plex relationships that stem from using them. Right of way conventions order movement through intersections. Traffic control mechanisms, such as traffic lights, explicitly control entrance to the intersection, often on a lane-by-lane basis. All in all, movement over roads and intersections in everyday life is controlled by well-defined structures, societal rules, and automated regulation mechanisms to minimize congestion and accidents. The organization of the EDF is highly influenced by the behaviors used to control autonomous agents. In this regard, the EDF is behavior-centric. The key to this organization comes from an understanding of what information, in addition to geometry and topology, is needed by the autonomous agents in the environment. Through this organization, the EDF emphasizes a strong coupling between behavior and environmental structure. In essence, behavioral implications are furnished by the environmental structure to assist autonomous agent decision making processes. The coupling between behavior and environment is central to ecological psychology. J.J. Gibson introduced the idea of environmental affordances for behavior. He asserted that humans perceive much more from the environment than basic sensory information. He argued that the environment provides humans with contextual and relational information [28][56]. In a similar sense, most manmade structures, such as roads and intersections, are designed to administer a specific logic and order to our behavior. I believe that these principles and design considerations serve as excellent motivation and reference for constructing both the environmental framework and the interactive, autonomous behaviors within the virtual environment. The EDF defines structural, logical, and relational properties in the database com- 32 ponents to integrate behavior and environment representations. Structural components provide detailed geometric and topologic information. For instance, roads and intersections represent structural components. The road surface is composed of purely geometric representation forming the so-called pavement of the environment. Intersections have shape and define a boundary. Roads connect to intersections along this boundary creating the road network topology. Logical attributes describe the culturally important and socially responsible aspects of the urban environment. For the most part, these properties influence the autonomous agents’ actions within the confines of the EDF’s structural components. Lanes, lane lines, roadside features, intersection corridors, and traffic control mechanisms all represent logical aspects of the EDF. In the EDF, relational components describe spatial and inter-object adjacency information. In addition, relational components are used to link important behavioral relationships with context-dependent behavior. Occupancy information is calculated by the simulation and stored in the EDF to spatially locate the autonomous agents with respect to the structural components of the EDF. Adjacency is defined in terms of which objects are in front of, behind, or next to the nearby autonomous agents. Corridor dependencies mark important intersection relationships on a corridor-by-corridor basis. The treatment of intersections in the EDF illustrates one aspect of how these properties combine to form the EDF’s components. Intersections are notoriously difficult to model, both geometrically and behaviorally, in driving simulations. Roads interconnect in 33 Figure 4.1: A vehicle navigates over a ribbon-like surface inquiring about the tangent and curvature properties at various points along the surface. complicated ways; driving behaviors must understand where they can go, how they can get there, and how their behavior relates to pedestrians and other vehicles whose paths cross their own. Behaviors must also have access to traffic control devices whose time-varying states govern acceptable behavior in the intersection. The EDF database integrates the structural and logical properties of the intersection with information about the relational interdependencies among objects simultaneously traveling through the intersection. 4.1 Curvilinear Coordinates Road-like structures can be modeled as ribbons in space. A roadway is naturally expressed in terms of a space curve with surface orientation defined by surface normals. Objects in the simulation environment follow the curvature of the road. Figure 4.1 illustrates a vehicle following the curvature of a ribbon-like surface. Objects avoid collisions by detecting surrounding objects and taking appropriate actions to avoid contact. For instance, 34 vehicles stay within the lanes of roads and slow down to avoid collisions with objects in front of them. Pedestrians dodge and weave around nearby pedestrians on crowded sidewalks. The driving surface in the EDF is described by a reference curve in 3D space with a surface extending outward, away from the curve. Way points, features, and object positions are all located relative to this surface. Elevation, curvature, tangent, and surface normals are important surface or curve features that are embedded in the surface description. The values of these attributes may change as a function of surface position. For example, curvature at one point on the surface may not be the same as curvature just a small distance away. In general, the information needed by autonomous behaviors is often dependent on structural context and location on the road surface. Road surfaces in the EDF are represented in curvilinear coordinate systems. Curvilinear coordinates provide a natural means for describing positions and relative locations on the road surface. A curvilinear coordinate system is a two-dimensional coordinate system based on a three-dimensional reference curve that acts as the major axis. The minor axis is defined as the vector perpendicular to the major axis and the surface normal. The curvilinear coordinate system used in the EDF defines the first dimension as distance ( ) and the second dimension as offset ( ). Distance is measured as the arc-length along the reference curve. Offset is given by the shortest distance to the reference curve as measured linearly along the minor axis. A single curvilinear coordinate is described by ( )¡ % ¡   the pair & '  . Using this system, points on the surface can be uniquely located with both a 35 reference curve major axis (distance) minor axes (offset) (δ1,ο1) (δ2,ο2) Figure 4.2: Illustration of a curvilinear coordinate system. Two curvilinear coordinates are shown depicting different locations. distance ( ) and an offset ( ) parameter. Coordinates on a curvilinear surface are defined from an origin, which is defined as the starting location of the reference curve. At this location, both distance and offset are zero. Offsets with positive value are located to the left of the reference curve and offsets with negative value are located to the right (when viewed in the positive direction of the major axis). A curvilinear coordinate frame is illustrated in Figure 4.2. The distance and offset measures define ordering relations for objects on the surface that are the basis for determining spatial relationships of objects on roads in the EDF. Curvilinear coordinates are used in the EDF database as a basis representation for ¡   36 all navigable routes. Behavioral objects, such as vehicles and pedestrians, track curvilinear coordinates to follow roads, lanes, and routes through intersections. In Chapter 3, I mentioned that one of the potential responsibilities of the autonomous behaviors is to provide control inputs to physics-based, dynamics models of the objects they control. At times it is beneficial for behaviors to compute control inputs in terms that are consistent with the dynamics models controlling the object. Most dynamics models are expressed in terms of Cartesian coordinates, and thus, produce Cartesian positions, rather than curvilinear positions. Therefore, it is necessary to provide conversions between curvilinear coordinates and Cartesian coordinates. These conversions are an essential part of all curvilinear-based components in the EDF. ( )¡ % Given a be capable of transforming the coordinate into a Cartesian coordinate, as expressed by the EDF query (( DQ¡ & ¥  % P% H F E C A @ 8 6 4 20 IG)DB¥97531 This query takes as input a curvilinear coordinate and returns a Cartesian coordinate in three-space. Once the dynamics process has calculated a new Cartesian position, we must map the position back to curvilinear coordinates. In EDF, the transformation function from Cartesian to curvilinear coordinates is ( 201 % W U E CA @ 8 6 XVTDBSR74 ( Q¡ & ¥  % ( Q¡ % which converts the input parameter into a curvilinear coordinate & ¥  2 1 & '  pair, any curvilinear coordinate system in the EDF database needs to on the curvilinear 37 coordinate system requesting the transformation. The ability to convert coordinates from Cartesian to curvilinear space is important at other times as well. When objects are initially placed in the simulation environment, it is often easier to position the objects according to absolute Cartesian coordinates rather than curvilinear coordinates. Once in the environment, the object’s location can be converted to curvilinear coordinates using the provided queries, as needed. For road-based behaviors, curvilinear coordinates are extremely useful: (1) they naturally represent locations along a road-like surface thus facilitating road and lane tracking behaviors, (2) relative spatial locations are easily defined with respect to the curve which is instrumental for following and obstacle avoidance behaviors, and (3) conversions between curvilinear and Cartesian coordinates afford flexibility in programming autonomous agents as different coordinate systems can be used for different purposes. 4.2 Segments - The Pavement Under Our Feet In the real world, when construction crews build new roadways they first grade the road and then lay pavement where the new roads, or intersections, will exist. At this stage in construction, only the surface geometry is defined. In the EDF, drivable surface geometry is represented by basic structures called segments. EDF segments are analogous to the pavement of real road networks and are used to model fixed width stretches of road. EDF segments are represented with curvilinear coordinate systems to provide geometric and structural information about the surface. While segment width is constant, the extent need not be symmetric about the curvilinear refer- 38 Reference Axis Figure 4.3: Illustration of general segment properties. ence axis. Figure 4.3 illustrates a basic EDF segment. Segments are structural components that support curvilinear coordinate transformations, as well as geometric queries about the curvilinear system and surface, including curvature, tangent, elevation, and surface normal. Using a variety of segments, we are able to create road-like structures in the EDF on which vehicles and pedestrians can drive and walk. For instance, we can construct linear segments for representing straight roads and cubic segments for representing the surface descriptions necessary for curved paths. In general, we can construct many different ribbon-like segment descriptions each for representing different road surfaces. For the most part, the roads we model do not twist violently or turn upside down. However, because the surface is defined by the reference curve’s normal vectors, EDF segments can easily be used to represent surfaces that exhibit extreme twist, such as can be found on roller coaster tracks. Nonetheless, our concentration is on urban driving environments where, for the most part, the road normals point skyward. Because the segment is based on a curvilinear system, surface attributes, such as ` Y left width (positive offset) right width (negative offset) 39 float length() Returns the total arc-length of the segment. float width( [ left or right ] ) Returns the left or right width of the segment. float elevation( ) Calculates surface elevation at ( ) on the segment. float curvature( ) Calculates curvature at ( ) on the segment. tangent( ) Calculates the tangent vector at ( ) on the segment. normal( ) Calculates the surface normal vector at ( ) on the segment. queryXY( ) Converts from Cartesian to curvilinear coordinates. queryDO( ) Converts from curvilinear to Cartesian coordinates. Table 4.1: The queries supported by segments in the EDF. All segments must support these queries. Queries are often parameterized by curvilinear coordinates to access exact surface detail. ¡ & ¥  ¡ ¡ & ¥  & ¥  ¡ & '  ( Q¡ ( )¡ ( )¡ & ¥  201 & ¥  ( Q¡ ( Q¡ % & ¥  % & ¥  % & ¥  % % 6 b4 6 74 6 74 6 74 6 74 2 0 1 6 4 20 7a31 6 4 20 7a31 ( )¡ & ¥  % 6 74 dinates. Such curvilinear data must be well-defined for each segment. Table 4.1 lists and briefly describes the behaviorally important queries provided by EDF segments. Necessary parameters and return types are also noted. These queries are designed to be used in a real-time simulation system. Therefore, it is highly important that the segment queries and the mappings between curvilinear and Cartesian coordinates be efficient, fast, and robust. Many thousands of queries may occur each second. In this context, a segment implementation that requires a complex integration calculation in order to transform from one of the coordinate systems to the other may ( Q¡ % elevation, tangent, curvature, or surface normal can be queried as functions of & ¥  coor- 40 c S4 Segment 2 S4 Figure 4.4: An illustration of a hierarchical segment in the EDF. The complete segment is composed of three leaf segments [S4, S5, Segment3]. Segments Segment1 and Segment2 are parent segments. be too computationally expensive. In any case, it is important that speed, efficiency, and robustness be balanced for segment types used in the EDF. 4.2.1 Compositing - Hierarchical Segment Descriptions It is useful to combine segment definitions to form larger aggregates. Segments in the EDF can be composed of other segments forming hierarchical curvilinear surface descriptions. Parent segments assemble and manage child segments to transparently present a single EDF segment interface. In other words, a single curvilinear coordinate system is mapped across a the leaf segments. Figure 4.4 illustrates the hierarchical structure for a simple segment composed of three leaf segments. Hierarchical segments are continuous at the joins of adjacent segments. With regard to the connection between adjacent segments, we define continuous to mean that (1) connecting segments’ end points be coincident, (2) adjacent segments’ widths are identical on both positive and negative sides, (3) adjacent segments’ reference curves align with at ¢ f ¦ least 1 continuity1, and (4) surface normals at the joining edge of the two segments are ¤ f continuity requires a common tangent at the joining point of the two segments and continuity requires that the curvature be common at the joining point of the two segments. d d S5 S e gm e nt 3 S5 c c S nt 2 egme d e ¢ Segment 1 Segment 1 Segment 3 41 identical. These restrictions ensure that hierarchical segments remain ribbon like in curve and surface definitions. 4.2.2 Segments in the EDF EDF segments are used to model both geo-specific and geo-typical roadways. In the real-world, road designers construct modern highways using standard techniques of civil engineering. However, many older roads are not built to conform to any standard design criteria. Using the EDF, we can model many different types of roadways using civil engineering data, or more free-form open-ended design criteria. To match civil engineering design standards, the EDF defines three segments based on analytic expressions used in roadway design. These are the straight, curve, and spiral segments [3, 23]. These segments are described analytically and model smoothly changing curvature and super elevation required on high-speed roadways. When composited together in the EDF these segments can be used to accurately model many real-world roads. Figure 4.5 illustrates how a curved road would be represented in the EDF using a combination of these segments. Because of their analytic nature, these segments afford fast and efficient queries for surface characteristic information and curvilinear coordinate system transformations. 4.2.2.1 Straight Segments As seen from an orthographic projection onto the ground plane, straight segments in the EDF connect two Cartesian coordinates in space with a line. Surface elevation, and thus, 42 Start of Road End of Road Figure 4.5: A longer and more complex road can be built by compositing segments. In this example, only three different types of segments are used: straight, curve, and spiral. 43 surface normal, is controlled by a separate function of   g that bases height on a parabolic curve. In the EDF, straight segments can be specified with either two, or three, Cartesian coordinates. When two coordinates are defined, the segment surface is flat, with linear g changes in , coordinates. Two-dimensional tangent vectors along a straight segment are constant, while curvature is always zero. 4.2.2.2 Curve Segments g Curve segments define a ribbon-like circular arc with dependence identical to the straight segment. In other words, a curve segment surface has constant circular radius (constant curvature) and sweeps out an arc of constant radius. If only the two end coordinates of the arc are specified than the change in the elevation ( ) is linear with respect to length along the curve. By specifying a third coordinate, we can fit a parabola to the segment describing surface elevation as a function of distance along the curve. Curve segments may also define a constant super-elevation which specifies the amount of banking. Surface elevation at a point is determined as by combining the surface height with the super-elevation. 4.2.2.3 Spiral Segments Spiral segments provide a transition between straight and curve segments. Spiral segments smoothly interpolate between the zero curvature and zero super-elevation of a straight segment to the fixed curvature and super-elevation of a curved segment. The eleva- g g segment’s elevation (height in H F , and coordinates. When specified using three coordinates, the straight ) is determined by fitting a parabola through the three 44 tion of the spiral segment is determined by combining super elevation with surface height. The surface height of the spiral is calculated identically to the straight and curve segments with either a linear interpolated profile or a parabolic elevation profile running the length of the segment. 4.2.2.4 Generalized Cubic Segments At times, it is more convenient and easier to model roads in a free-form manner than by adjoining analytical segments. Free-form road construction is useful for quickly prototyping scenes using intuitive curve drawing techniques. Moreover, many roads in the real world are not constructed according to standard engineering design practices. In the EDF, free-form road modeling is handled by a generalized cubic segment. A cubic segment h is described by a set of interpolating points in To facilitate transformations between curvilinear and Cartesian coordinate systems, we must reparameterize the cubic segment so that it is parameterized by arc-length. Once the segment is parameterized by arc-length, curvilinear coordinates can efficiently be converted to Cartesian coordinates by performing binary searches using curvilinear distance as the key over the sampling points, calculating localized cubic evaluations. Determining curvilinear coordinates from Cartesian coordinate can also be difficult. We use use a small number of steps in a Newton’s method to efficiently handle conversion from Cartesian to curvilinear coordinates. In summary, EDF segments provide the foundation for modeling road surfaces. t¢  D£" 2 1 . ¢ © & q q q p s$rr& p & $i p 45 EDF segments are curvilinear coordinate surfaces that analogous to ribbons of pavement through the environment. Segments provide queries for interrogating the surface information and converting between local curvilinear coordinates and Cartesian space. Using hierarchical composition, segments can be combined to form more elaborate pathways that mimic the construction style of roads found in the real world. 4.3 Roads Once the pavement for a new road is set, construction crews paint lane lines on the road surface that delineate lane structure. Lanes lines also identify regulations on driving activities, such as passing. Signs are placed alongside the road to signify speed limit, no parking, merges, and yields. The lane lines and signage represent logical aspects of driving. Humans can easily drive over pavement, but order, and thus behavior, is determined from the logic applied to the road. In the EDF, roads are modeled very similarly by applying logic and order to a segment representing the pavement. Roads in the EDF are composed of a segment, which may be hierarchical, a lane profile definition, and feature descriptions. The road’s segment definition represents the pavement on which the road logic will be applied. EDF roads connect to intersections at the termination points of the roads. Through the intersection, other roads can be reached. Roads in EDF derive their width from their segment. The road’s width is then laterally subdivided into parallel channels called lanes. Lanes are classified according to behavioral use and run parallel to the curvilinear axis defined by the road’s segment description. Lane definitions are specified outward from the road’s axis to ensure consistent and easily 46 Pedestrian Parking (Parallel) Bicycle Vehicle Vehicle Median Median Vehicle Median Pedestrian Road Reference Curve Figure 4.6: Road width is subdivided laterally across the width of the road to form lanes. Lanes can be of varying types and in this example sidewalks, vehicle lanes, bicycle lanes, parking lanes, and medians are shown. identifiable lane boundary offsets that are based on the lanes’ widths. Lane boundaries are used to define adjacency relationships between neighboring lanes. The treatment of lane definitions in the EDF is depicted in Figure 4.6, which shows a cross-section of a lane definition. As a person drives over the length of a real road, that person’s behavior is apt to change and reflect the features and markers posted or painted along the roadway. Some types of markings, such as speed limit zones, affect large expanses of road while others, BUMP or L ANE E NDS signs for instance, provide more immediate and local information. In the EDF, range attributes describe curvilinear expanses of road in which specific behavioral or societal rules apply. Similarly, the EDF uses features to model localized, situated information that exists at specific distances along the length of the road. 47 (δ,ο2) (δ,ο1) Figure 4.7: Curvilinear distance is calculated along the reference curve (shown in black). Both coordinates have the same distance parameter ( ), but different offsets ( , ). Dashed, green arcs represent the true arc-length of the curve at the different offsets. ¤ '¡ ¢ T¡   During a simulation, the EDF maintains information about the simulation objects located on each road’s surface at any instance of time. This information can be queried providing behaviors with important road occupancy detail. Object location on roads, as well as how those objects’ locations relate to each other, facilitates following and obstacle avoidance behaviors. A complete discussion of road occupancy is provided in Section 4.5. 4.3.1 Interpretation of Curvilinear Coordinates Distance along the length of a road in the EDF is treated similar to mile-markers along an interstate in the real world. A mile-marker marks an invisible perpendicular line that cuts across all lanes of a highway to denote the location of a particular mile, or distance from the beginning of the roadway. Any vehicles crossing that line, no matter what lane they are in, are considered to be located at that mile marker. By our definition, curvilinear distance ( ) is measured along the reference curve; curvilinear offset ( ) is calculated as the shortest perpendicular distance to the reference curve at that distance. This interpretation of a curvilinear coordinate uses a single longitu¡   48 dinal representation of distance, as opposed to an offset based definition of distance over ( ¤ r'¡ % (¢ PT¡ % the curve. Thus, two positions   and reference curve at milemarker . Figure 4.7 illustrates the how the EDF treats curvilinear coordinates in terms of mile markers. The advantage of using this interpretation for the curvilinear coordinate is that important spatial relationships among objects are directly determined. For instance, given a set of of distance parameters on a curvilinear surface, we can define an ordering of those distances over length of the surface. Similarly, using a set of offset parameters, we can determine an ordering that runs laterally across the surface definition. Using these partial orderings, autonomous behaviors can query neighboring objects and compare curvilinear distance coordinates to understand local spatial relationships. 4.3.2 Lanes Lanes partition the road surface into parallel, longitudinal regions. A road in the EDF bundles a set of lanes together. Roads can be composed of one or more lanes, with the lane set being ordered and dense–meaning that there are no gaps between adjacent lanes. Lane widths are constant over the length of the road. As a consequence, the road profile is constant over the length of the road as determined by the extents of the lane offsets. In the EDF, a road’s lanes run parallel to the road’s curvilinear axis and may not straddle that axis. Thus, the road axis is always used as a boundary between two lanes. This restriction is beneficial because it allows for a discrete and simple ordering of lanes. Consequently, a curvilinear offset can quickly be mapped to a particular lane by utilizing & '  & ¥  define a line perpendicular to the road 49 Lane Type VEHICLE BICYCLE SIDEWALK BUS TRAIN TRAM AGRICULTURAL CENTER - TURN MEDIAN PARKWAY ANGLED PARKING PERPENDICULAR PARKING PARALLEL PARKING Description General-purpose vehicles Bicyclists Pedestrians Buses Train tracks Trolley cars Tractors, combines, threshers, wagons Specialized turn lane Lane spacing Area between road pavement and sidewalk Angled parking lanes Perpendicular parking lanes Parallel parking lanes Table 4.2: Various types of lanes defined in the EDF. the lane width and adjacency information. Lanes in the EDF are typed. For instance, a sidewalk is a type of lane dedicated to pedestrians while a center-turn lane allows traffic to make left turns on busy streets without blocking traffic flow. We chose to associate sidewalks with roads because sidewalks, and other similar features, fit nicely with our definition of lanes. Generally, sidewalks accompany roads in the real world, running parallel to the road for the length of the road. Table 4.2 lists and briefly annotates the lane types in EDF. In addition to behavioral type information, lanes designate a preferred direction of flow, oriented with respect to the road coordinate system. Some lanes allow bidirectional movement, so flow direction in a lane may take on one of three values: Positive, Negative, or Both. Each lane can specify a surface-height offset value which is used to raise, or lower, the lane’s elevation relative 50 to the height of the its road. This attribute is useful for medians, parkways, and sidewalks which are sometimes raised above the height of the road to which they are adjacent. 4.3.3 Range Attributes In the real-world, signs and markings are posted alongside and on roads to highlight important information related to driving. For instance, speed limits signs indicate the maximum speed allowed on a section of road. Lanes lines are drawn between lanes to prohibit or allow certain types of behavior, such as passing in a passing zone. The EDF models zone-based, road characteristics as range attributes. Range attributes are bounded curvilinear regions that supply behaviorally important information about that region of the road surface. Range attributes are characterized by a rectangular curvilinear area and data. A range attribute’s area is defined by a pair of curvilinear co( t D( ordinates specifying the lower left and upper right coordinate of road surface. The range attribute’s data is represented by an attribute label, followed by a list of data elements ‡ & q q q rsr$r& ‡ r& y   y r€% @ƒ ‚ ‡ … A r& †'ƒ … A „TC In the case of a speedlimit range attribute, the label is speedlimit and the data associated with it would be the speed limit over that region, as in   €% • ” ‚ ’‚ A – —p ’ ‰$& “#‘… ‡ †A p During the course of a simulation, range attributes can be dynamically controlled by an autonomous behavior. Consider the following, a railroad crossing gate can be rep- ¡ % x & x w ¥w $¥D  (¢  " y  P‰Q'ry ( ¡ % u & uu vu vD  ( ¢ y  ˆ'†y © y  †'ry ¡ % ( ¡ % x & x w ¥w $¥D  & vu vD  i u & uu 51 resented as a dynamically manipulated range attribute. In this example, a region of the road is marked to denote the area of the road where the train tracks cross. The bounds of this region demarcate the location of the gates that guard access to the crossing area. The crossing gate attribute data is specified by ( p @& A  y k j rio¥¡ … h A m†gf ‚  y rgd ¤  y k j rmlS¡  T¡  ¢ d   Vh Q)¡ % A r& r& p @ & A … h A i†gf ‚ ‚ ‚ C˜ C ™TC n   e   r& r& the location of the respective gates along the road surface. The two gates, labeled as PositiveFlowGate and NegativeFlowGate to signify the relative direction of traffic flow they toward which they face, guard access to the railroad track crossing region. The logical state of these gates is either Up or Down to represent the physical state of the gate. By attaching an autonomous behavior to this range attribute, we provide a mechanism to dynamically monitor the crossing train tracks for an oncoming train and set the state of the range attribute gates appropriately if a train is detected. Autonomous behaviors, such as vehicles, navigating the road will query the rrcrossing range attribute and can react to the logical state of the gate, either stopping prior to the gates, or driving over the road normally. When driving on real roads, signs posted along the roadway will, at times, denote immediate and local information specific to that place on the road. For instance, a S PEED BUMP sign often marks the location of a speed bump on the road’s surface. In the EDF, road signs, such as these, that mark a perpendicular reference on a road surface are represented as features. ¤ The curvilinear distances ( ¢   r&   ) given in the range attribute data component determine 4.3.4 Features 52 A feature is a logical EDF component that models localized, cross-sectional information on the road’s surface. Features have a facing direction and can be laterally bounded by two offsets bracketing where the feature resides on the road. Features differ from range attributes in that they do not cover an area on the road; rather, features cover a cross-sectional slice of the road surface. A feature’s placement on the road’s surface is determined by a distance , two offsets bounding the left and right offsets that the feature v    affects ( feature location Feature facing direction determines the direction toward which the feature points. For instance, on real two-way streets, signs for the oncoming lanes face toward the oncoming traffic and do not affect the opposite flow traffic lanes. Features that are P OSITIVE are directed toward traffic traveling toward the end of the road’s curvilinear coordinate system; N EGATIVE facing features are directed toward traffic moving toward the origin of the road’s curvilinear coordinate system. Features can face both directions. The data component of a feature is represented by a feature label followed by a series of data elements. (¢  " y  D£)'ry ¢ y  ˆ'ry © y  Q'ry feature data Like range attributes, feature data may be dynamically controlled by an autonomous behavior over the course of a simulation. The EDF supplies queries for interrogating range attributes and features on roads. Range attributes occur over curvilinear regions of the road, and thus, queries that are pa- ‡ & q q q rs$rr& ( wv  ¡ ¥p  ¡ %  & !‘rx & u ¥  u sq & ‡ r& 6 y  †y #% x A C ‡ … A r& †¥ƒ … ™—@ A 6 ¡ ¥p  ¡  u sq trx & u ), and a facing parameter, . 53 list RangeAttr queryRangeAttributes( ) Returns the list of range attributes that span the given curvilinear coordinate. list Feature queryFeatures( ) Return a list of features between the bounding curvilinear coordinates, given the facing direction. RangeAttr queryRangeAttribute( attribute label, ) Returns the requested range attribute that exists at the given curvilinear coordinate. Feature queryFeature( feature label, ) Returns the requested feature between the bounding curvilinear coordinates, given the facing direction. list RangeAttr queryRangeAttributesByName( attribute label ) Returns a list of range attributes on a road with the specified attribute label. list Feature queryFeaturesByName( feature label, ) Returns a list of features on a road with the specified feature label, given the facing direction. Table 4.3: EDF queries that support interrogation of range attributes and features on roads. v ( Q¡ v ( ¡ % )¡ % ( & { |¥  & '  &{ & & ¥  % v ( ¡ % )¡ % ( & { |¥  & &{ & ¥  ( )¡ % 6 74 6 74 6 74 & ¥  z z 6 b4 z z 6 74 y y y y & ¥  6 74 ( Q¡ coordinates. On the other hand, because features have facing directions and are specified cross-sectionally at exact distances along the road surface, EDF queries restrict the search for features with a bounded curvilinear region and facing direction. Table 4.3 lists the EDF road queries used to query information about range attributes and features. 4.3.5 Roads in EDF use the curvilinear coordinate system provided by its segment. Thus, roads provide the same queries as listed for the segments (Section 4.2). Additional queries provide topologic information and inform behaviors about the intersections to which the % rameterized by return all the range attributes affecting those particular curvilinear Additional Queries for Interrogating Road Information 54 int numberOfLanes() Returns the number of lanes on the road. list Lane laneProfile() Returns the set of lanes that make up the lane profile on the road. Lane laneToLeft( lane reference ) Returns the lane to the left (if it exists) of the lane parameter. Lane laneToRight( lane reference ) Returns the lane to the right (if it exists) of the lane parameter. Intersection startIntersection() Returns a reference to the road’s starting intersection. Intersection endIntersection() Returns a reference to the road’s ending intersection. Intersection nextIntersection( flow direction ) Given a direction of flow on the road, return the forward facing (as determined by flow direction) intersection. float distanceToIntersection( , [ start end ]) Given a curvilinear coordinate distance parameter and an intersection label, return the distance to the intersection. float distanceToNextIntersection( , flow direction ) Given a curvilinear coordinate distance parameter and a flow direction, return the distance to the forward facing intersection. Table 4.4: Additional queries supported by roads in the EDF. }     6 74 6 74 6 74 6 b4 z 6 74 6 74 6 b4 6 b4 y 6 74 55 road connects. Relative queries support queries based on the behavior’s immediate direction of flow with respect to the road’s curvilinear coordinate system. 4.4 Intersections In the real world, intersections serve as interchanges between different roads. Driving behavior is different in intersections than on roads. Roads are simply organized into parallel streams in which human behavior can be tightly regimented, and reasonably predictable. Intersections, on the other hand, are logically complicated, messy, and, in the field of driving simulation, notoriously difficult to model. The channel-like passages connecting incoming to outgoing lanes criss-cross and overlap each other. Spatial relationships become tangled, highly complex, and difficult to identify. In real intersections, societal rules and conventions determine who has right of way and how traffic should proceed. In the EDF, intersections specify input/output relationships between the lanes of the roads connected to the intersection. EDF intersections define a standard interconnection scheme that details how lanes interlink and interrelate in the intersection. Intersections in the EDF have four parts: (1) an exterior polygonal boundary to which roads incident to the intersection connect, (2) sets of individual, lane-like corridors that route traffic through the intersection, (3) lists of dependencies between corridors that describe how objects on different corridors relate, and (4) traffic control devices that regulate entrance into and movement through the intersection. 56 4.4.1 Intersection Boundaries - Exterior Geometry Intersections in the EDF are bounded by a convex polygon. The edges of the polygon are used as landmarks to which the terminating edges of connecting roads can adjoin, forming smooth transitions between road and intersection geometry. As with roads, simulation objects can occupy intersections. The convex geometric description of the intersection polygon allows the EDF to perform efficient object in polygon computations during a simulation. Intersection occupancy semantics will be discussed in Section 4.5. 4.4.2 Corridors In real intersections, the connections between lanes are often implicitly defined. At times however, especially in the presence of adjacent turning lanes, or when it isn’t visually obvious, lanes lines are painted in the intersection to explicitly guide drivers on how they should drive through the intersection. In the EDF, the connections between lanes are explicitly represented and describe the routes between incoming and outgoing lanes of roads that connect to the intersection. These routes are called corridors. Corridors are analogous to single-lane roads. They contain a segment, which may be hierarchical, that describes a curvilinear path between the two lanes incident on the intersection. The corridor’s reference curve connects the termination points of two lane centerlines on the intersection boundary. Thus, the corridor’s reference curve is a lanecenterline to lane-centerline route. Corridor width is determined to be the maximum of the two connecting lanes’ widths. Figure 4.8 illustrates a simple intersection with corridors for 57 Figure 4.8: Corridors cross intersections and connect incoming lanes to outgoing lanes. The corridors leaving the left and right roads are shown in red. the left and right roads drawn in dashed lines. Corridors can represent any route through the intersection. The figure shows commonly used routes for driving straight, turning right, and turning left. Corridors are typed. Corridor type information serves the same purpose as lane type does for EDF roads. Both provide contextual information to behaviors. The EDF places no restrictions on which corridor types can match up with which lane types. This means that logically incompatible connections can be constructed and it is up to the modeler to correctly match compatible types. For instance, a bike lane can reasonably connect to a vehicle lane, but connecting a vehicle lane to a median, sidewalk, or tram lane is not sensible. In the former, both the bike lane and vehicle lanes support drivable vehicles that maneuver on paved surfaces. In the latter, it is not clear how a vehicle would drive on a tram track, let alone a sidewalk, or median. 58 4.4.2.1 Corridor Attributes - Flow, Stoplines, and Curbs At the intersection boundary, incoming lanes connect to corridors leading to lanes on outgoing roads. When more than one corridor branches out from the same incoming lane, multiple pathways exist through the intersection from that lane. Behaviorally, these junctures are noteworthy because they force behaviors to make decisions as to which corridor to use. Often times, random decisions will suffice. Unfortunately though, random decisions produce random behavior that may not be consistent with the design of the environment. From the standpoint of the database modeler, these junctures present an opportune place to encode modeling intentions and design notes that reflect the modeler’s expectations about traffic movement on the lanes and corridors of the environment. Similarly, traffic modeling algorithms can use corridor branches to encode desired traffic flow statistics that will influence turn decisions of autonomous vehicles. Algorithms for managing ambient traffic and maintaining various traffic models attempt to control traffic density and flow on roadways by influencing the routing decisions of traffic in intersections [2, 12, 11]. The EDF encodes a flow percentage attribute in each corridor indicating the desired percentage of traffic entering the intersection on that lane that should take the given corridor. The accumulated percentages for all corridors leaving a lane must sum to one. EDF enforces this constraint by accepting flow percentages between and and then normalizing the set of flow percentages to be consistent over the range [ , ]. For corridors that are  ~  ~ 59 bidirectional, flow percentages are specified at both ends of the corridor. By encoding flow percentage into the corridor, autonomous behaviors, such as vehicles or pedestrians, can make educated, yet random decisions that are consistent with desired traffic distributions. Moreover, database and traffic modelers can prescribe flow percentages that match the design of the road, or scenario being developed without recoding the vehicles that drive in the environment. Flow percentages can be dynamic to evolve to changes in simulation state. For instance, autonomous traffic manager behaviors can modify and tweak the flow percentages at intersections during a simulation to route traffic according to time varying scenario requirements and traffic distribution rules and goals. On roads in the United States, most controlled intersections are marked with stoplines to designate a stopping location for vehicles. The EDF encodes stopline distance for each corridor that is used to enter the intersection. This is primarily useful if the intersection boundary encompasses the crosswalk and stopline markings. Otherwise, stopline can be encoded as a feature along the roadway. Stopline distance is a distance measured along the curvilinear reference curve of the corridor and can be queried by behaviors approaching the intersection to ensure that the object being controlled by the behavior stops at the appropriate place along the corridor. In situations where multiple corridors branch from a single incoming lane, it may be the case that stopline distance will be different for each corridor. Stoplines are primarily for vehicles and mark an important location on the road 60 surface. For pedestrians, curbs act like a stoplines, guarding entrance into the roads. More importantly, curbs often signify abrupt change of elevation. In the EDF, it is important that pedestrian behaviors have access to the location of the curb so that foot falls can accurately, and realistically be placed on the curb [65]. The EDF models the location of curbs by specifying a curb distance attribute on pedestrian corridors. Curb distance is encoded into the corridor as the distance from either the beginning, or the end, of the corridor. As with stopline distance, curb distance on bidirectional corridors is specified at both ends, if required. Flow percentage, stopline distance, and curb distance are specialized attributes built into the corridor description. In addition to these specialized, built-in attributes, corridors, like roads (see Section 4.3), can specify range attributes, and/or features along the curvilinear coordinate system used to define the corridor. 4.4.2.2 Junctures The intersection boundary uniquely defines the geometry of the intersection. Roads abut to the edges of the intersection making the transition from road to intersection smooth and seamless. Similarly, internal corridors seamlessly align with the intersection boundary. Lanes on connecting roads map to specific corridors in the intersection. In general, there are many attributes and elements that need to be specified; intersections are difficult to model. Luckily, in real urban environments multiple intersections often have the same relative structure and characteristics, with only minor differences between intersections being 61 the widths of the roads that connect to the intersections. Downtown city block intersections are an excellent case in point. In the synthetic environment, reuse of structure is even more widely used and therefore apparent in the models. Intersections are difficult to specify and model. In the EDF, we leverage modeling effort by affording intersection reuse. It makes sense to reuse the structure, logic, and relational characteristics defined in the intersection, especially when the only differences between intersections may be the incoming lane widths. The EDF facilitates intersection reuse by defining connection sites called junctures. Junctures are placed along the edges of the intersection boundary and mark locations to which roads, lane centerlines, and corridors can connect. By pre-defining these connection points, intersections can be constructed independent of their connections to roads. As a consequence, corridors can be specified without regard to which roads or lanes they connect. In this sense, intersection definitions becomes templates that can be instantiated as needed in the environment description. Upon instantiation in the EDF, the bindings between the connecting roads, lanes, and corridors become effective and explicit, cementing the intersection at that location. Two types of junctures are used in the EDF: fixed and floating. Fixed junctures mark an exact location on the intersection boundary edge. They are useful for connecting roads and well-defined corridors to the intersection. Floating junctures describe a connection point on the intersection boundary that is not yet bound to a specific location on the edge. Floating junctures are used when we know a connection will be made, but we don’t know 62 Figure 4.9: Illustration of fixed and floating junctures placed along the intersection boundary. Red squares denote fixed junctures while green dots represent floating junctures. The corridors between the junctures are shown as well. at what point on the boundary edge the connection will exist. Floating junctures are the primary tools for making intersections reusable. Internally, corridors can be connected to both fixed and floating junctures. Multiple junctures can be placed on an edge to accommodate multiple connections. When more than one juncture is placed along an edge, the ordering of junctures is always preserved by the EDF. Figure 4.9 illustrates a simple intersection with both types of junctures. The red squares represent fixed junctures, while the green dots represent floating junctures. Juncture ordering along an edge is invariant. When initially defined, fixed junctures remain fixed along the edge while floating junctures are spread out evenly between neighboring absolute junctures and edge vertices. 63 lane 4 lane 3 lane 2 lane 1 lane 4 lane 3 lane 2 lane 1 Figure 4.10: The road axis connects to the fixed juncture (shown as a red square) and the floating junctures (shown as green circles) adapt to fit the offsets of the matching lanes. This three-part illustration shows the process of connecting a road to an intersection. When we connect a road to an intersection, we attach the road’s reference axis to a fixed juncture on the intersection boundary. Only the terminating coordinates of roads may connect to a juncture. However, more than one road may connect to an edge as long as it connects to a separate fixed juncture. Once a road is connected to an intersection, the EDF attempts to align the neighboring floating junctures with the road’s lanes’ centerlines. Floating junctures connect to the ending, or beginning, coordinates of lane centerlines. Floating junctures translate along the boundary edge to accommodate for the fixed spacing of the lanes on the roads. A floating juncture pushes, or pulls, neighboring floating junctures as it translates along the edge to accommodate the width of the connecting lane. Juncture ordering is always preserved along an edge. Figure 4.10 illustrates the process of connecting a road to an intersection and how the junctures adapt to fit the road’s lanes. To assist modeling effort, the EDF can generate cubic spline segments to model the corridor geometry between junctures automatically. These automatically defined segments € € € € € € lane 4 lane 3 lane 2 lane 1 64 are beneficial when using floating junctures. They allow the EDF to adapt the corridor definition to the locations of the floating junctures once roads and intersections connect. On the other hand, corridor segments can be defined explicitly, but in these situations, the EDF cannot adapt the corridor segment to fit updated locations of floating juncture endpoints. It is suggested that in these circumstances, fixed junctures be used to ensure rigid corridor placement. In terms of database modeling, junctures allow intersections to be modular and reusable. A database modeler can reuse an intersection description by delaying the binding, or connection, between road and juncture until a later time. When designing and developing a database model, a single intersection can be instantiated as needed, connecting roads and lanes to the appropriate junctures on the intersection edge. In essence, this produces intersection descriptions that are parameterized by the set of roads that connect to the intersection. 4.4.3 Dependencies - Internal Intersection Constraints As corridors pass through intersections they interweave in complex ways: crossing, overlapping, merging, or splaying. Many corridors feed into the same outgoing lane and single incoming lanes diverge into many corridors connected to different outgoing lanes. In real intersections, traffic control devices and right of way conventions govern interactions of vehicles on different corridors. Without these, chaos would ensue. As drivers enter and negotiate travel through intersections, they need to attend to many differing concerns and constraints. For instance, turning left is preceded by locating 65 Dependencies ADJACENT TO CROSSES CROSSWALK DIVERGES WITH MERGES WITH RIGHT ON RED RIGHT OF WAY Table 4.5: Types of dependencies that can be used in the EDF to describe the inter-object relationships on corridors. a safe gap in oncoming traffic. Even driving straight through an intersection requires that the driver pay some attention to the traffic on crossing corridors. The constraints that are implicitly defined between the objects traveling through intersections are corridor dependent. The EDF encapsulates inter-object constraints into corridor dependency information. Dependencies define potentially hazardous and important interrelations among traffic on different corridors. The EDF encodes dependencies into the intersection on a per-corridor basis. Simply put, dependencies supply behaviors with context dependent information related to the corridor on which they are navigating. A dependency maps a dependency label to one or more corridors, as in (¢  !‰" ¡ ¡ ¢ ¡ ¡ © ¡ ¡ y h h % C ‡ )TC ˜ rrr& C ‡ T)C ˜ & C ‡ ))C ˜ & †¥ƒ … !˜ A ‡ A p A ‡ ‚ C ‚ C ‚ C … A E & q q q Multiple dependencies can be specified for each corridor in an intersection and are encoded into the environment by the database modeler depending upon the capabilities of the autonomous behaviors. The EDF does not specify a preset list of dependencies for use by behaviors. To provide a feel for different types of dependencies, Table 4.5 lists a set of 66 C2 C1 C0 Figure 4.11: An illustration of the use of dependencies to negotiate a left turn. Vehicles traveling on corridor and may pose threats to objects traveling on corridor . © ¨¦ ¤ ¨¦ ¢ §¦ common dependencies. Dependencies compartmentalize behavioral activities. Each dependency in the intersection signifies a relationship between the objects on two corridors. Autonomous behaviors can be designed and organized around these relationships, so that as dependencies are encountered, separate, behavioral-dependency units can be deployed to react to the dependency on an as-needed basis. For example, assume an autonomous vehicle needs to make a left turn at an intersection, as depicted in Figure 4.11. The darker vehicle will enter the intersection on the © ¨¦ corridor labeled . In this circumstance, the vehicle’s turning behavior must accept a ¤ ¨¦ ¢ §¦ safe gap in the oncoming traffic. The oncoming traffic corridors denoted by and are 67 © ¨¦ & specified in the dependency list of corridor % as (¢ P§¦ & merges with ( ¤ r¨¦ crosses % The darker vehicle’s behavior can use the dependency list to dynamically create two turning dependent sub-behaviors: one for dealing with the merging traffic, and another for accepting a safe gap in the oncoming traffic. After the turn is completed, these two newly instanced behaviors can be retired until later use. 4.4.4 Traffic Control Behavior in EDF intersections is regulated by traffic control devices (e.g. stop signs, traffic lights, pedestrian walk signals) that control traffic flow on a per-corridor basis. Bidirectional corridors, such as sidewalks, are controlled unilaterally by a single traffic control state. Autonomous vehicles, for instance, can interrogate the intersection for the traffic control state associated with the corridor the vehicle plans to use to travel through the intersection. Traffic control state may be static or may be determined dynamically by an autonomous behavior. For example, in the EDF, a single traffic light behavior can control the state of a particular intersection’s traffic light. The state of the traffic lights at other intersections are usually controlled by separate behaviors. However, at times, all the traffic lights at a series of intersections may be synchronized with each other to maximize traffic flow through the road networks. Traffic control state can be determined by multiple behaviors. In uncontrolled inter- 68 Traffic Control States R ED Y ELLOW G REEN F LASHING R ED F LASHING Y ELLOW S TOP S IGN T HROUGH U NCONTROLLED Table 4.6: Traffic control state. sections, no traffic control device is required. Table 4.6 lists the traffic control states used in the EDF to regulate behavior at intersections. 4.4.5 Zero Length Intersections Polygonal intersections specify a fixed geometry with corridors mapping routes between incoming and outgoing lanes. However, at times, it makes sense to simply connect the ends of two roads together without the behavioral, and modeling, overhead associated with a complete intersection. In essence, the intersection structure is collapsed to a single edge–a degenerate polygon. In the EDF, these intersections are called zero-length intersections since they occupy no space yet act like regular, polygonal-based intersections. Zero-length intersections use zero-length corridors to describe lane-to-lane connections and specify dependency information between corridors. Zero-length corridors have no length, and are point-based connections that provide a mapping between lanes connect- 69 Lane Addition Lane End Figure 4.12: Example of how a zero-length intersection connects two roads with differing lane definitions. ing to the junctures on the zero-length intersection’s edge. Traffic control at zero-length intersections works identically to traffic control in regular EDF-based intersections. However, zero-length intersections are used to describe seamless changes in road structure where, for the most part, no traffic control state is required and traffic can drive through without change of behavior. There are two instances where zero-length intersection are used: (1) forming road loops, and (2) lane termination or lane addition. Figure 4.12 depicts an instance of using a zero-length intersection to connect two roads with differing lane definitions. When using zero-length intersections, lane centerlines must align on the connecting roads since there are no corridors to merge between lanes of different offsets. 4.4.6 Hierarchical Intersections - Internal Routing The EDF intersection model uses hierarchical intersection descriptions to provide the necessary structure for describing complex internal intersection routing. To understand 70 Pedestrian Lanes Car Lanes Bicycle Lanes Figure 4.13: Example of a hierarchical intersection definition. Note the internal intersections forming the pedestrian sidewalk intersections. internal routing, consider the manner in which sidewalks intersect. Sidewalk lanes in the EDF connect to sidewalk corridors. These corridors criss-cross forming their own intersections. Often times, the location of these pedestrian intersections is within the bounds of the intersection geometry. Within internal intersections, corridors connect to corridors at junctures on the edges of the child intersection. Figure 4.13 illustrates a three-way intersection where sidewalks come together within the boundary of the larger intersection. At these crossings, we define internal intersections. 4.4.7 Intersection and Corridor Behavior Queries Table 4.7 lists queries provided by the intersection and corridor structures in the EDF. Intersection queries provide access to corridors connecting roads, lanes, and corridors 71 Intersection Queries list Corridor queryCorridors( Road, Lane ) Returns a list of Corridors that connect to the road and lane provided as parameters. list Corridor queryCorridors( Corridor, [ start end ] ) Returns a list of Corridors that connect to the specified end of the corridor provided as parameter if the intersection is an internal intersection. TCState queryTCState( Corridor ) Returns the traffic control state value regulating movement along the specified corridor. boolean isZeroLength() Returns a boolean value indicating if the intersection is a zero length intersection. Corridor Specific Queries list Dependency queryDependencies() Returns a list of dependencies on the corridor. boolean connectsToRoad( [ start end ] ) Returns a boolean value indicating whether the specified end of the corridor connects to a road, or not. If false, the corridor connects to an internal intersection. Intersection startIntersection() Returns a reference to the intersection at the beginning of the corridor. Intersection endIntersection() Returns a reference to the intersection at the end of the corridor. Intersection nextIntersection( flow direction ) Given a relative direction of flow on the corridor based on the corridor’s curvilinear axis, return the forward facing (as determined by flow direction) intersection. float distanceToIntersection( , [ start end ]) Given a curvilinear coordinate distance parameter and an intersection label, return the distance to the intersection. float distanceToNextIntersection( , flow direction ) Given a curvilinear coordinate distance parameter and a flow direction, return the distance to the forward facing intersection. Table 4.7: Intersection queries relate connecting roads and corridors and provide traffic control state for corridors. Corridor queries provide access to dependency information and which intersections (in the case of internal intersections) connect to the corridor termination points. } }   }   6 b4 6 74 6 74 z 6 b4 6 b4 6 b4 z z 6 b4 6 b4 6 b4 6 74 6 74 y y y 72 together and also give access to the traffic control device state at the intersection. Because corridors are curvilinear structures, they support all the queries that a curvilinear coordinate system must provide, as described in Section 4.2. They also support the feature and range attribute queries described in the road section (Section 4.3). In addition, EDF corridors provide access to dependency and internal intersection information. 4.5 Occupancy In addition to the structure and layout of the static environment, it is essential that dynamic objects be aware of other dynamic objects. For example, a vehicle on a road must avoid crashing into the vehicle immediately ahead of it in its lane. Pedestrians dodge and weave around other pedestrians on the sidewalk. To be able to avoid collisions and to interact in other ways, autonomous characters need access to the dynamic objects in the environment (e.g. vehicles, pedestrians), but more importantly, they need access to the spatial locations of the objects near them in the environment. The EDF maintains relative and spatial locations of objects. Relative locations refer to the relationships between objects and the surfaces in the environment on which they exist. In other words, relative locations describe occupancy on a surface. For example, relative location information can provide answers to queries, such as what vehicles are on a particular lane, what pedestrians are in a crosswalk, or what objects are in an intersection. Spatial location simply describes the location, or position, of the object in the environment. This can be expressed in two ways. For one, spatial location can be the object’s Cartesian coordinates. On the other hand, if the object is located on a curvilinear surface, 73 ¡ spatial location can be a curvilinear coordinate, described by Occupancy is derived from the spatial positions of objects and how they relate to the structures of the simulated environment. But first, we need an understanding of what it means to occupy something. 4.5.1 Occupying Space in the EDF In the EDF, the term occupancy means how simulation objects occupy space on roads, intersections, and corridors. For our purposes, occupancy is a by-product of an orthographic projection of the objects’ geometry onto the ground surface, and hence onto the EDF structures covering the ground. It is quite important that both animate and inanimate objects be included in occupancy computation. For instance, we expect pedestrians walking on the sidewalk to dodge and weave around each other to avoid collisions. Likewise, we expect similar behavior if there are lamp posts or parking meters placed along the sidewalk. The answer to the question Does a particular object occupy a particular structure? depends upon who asked the question and why the question was asked. Some behaviors simply may not require as much detail about occupancy as others. For instance, whether or not a vehicle’s bumper is encroaching into another vehicle’s lane depends upon the attitude of the driver. A conservative behavior may find the movement of the neighboring vehicle’s bumper offensive and dangerous, while an inattentive driver may not even notice the bumper. Nonetheless, it is important to establish a baseline for occupancy. At the most primitive level, each object in the simulation is represented by a single point denoting the & ¥  . 74 position of it’s center of geometry (CoG). This coordinate can be used for assigning pointbased occupancy. 4.5.2 A Witness to Occupancy We formulate descriptions of occupancy in terms of witnesses. This is in contrast to describing occupancy in terms of the simulation objects themselves. As an example, we can describe occupancy on a road by listing all the objects on that road. The witness description is an occupancy abstraction that can tell us what objects occupy a certain structure, but can also provide additional information that describes how an object occupies a space. For instance, a point-based witness provides the most primitive form of occupancy in the EDF by mapping an object’s CoG onto one or more EDF components2. This witness would provide a reference to the occupying object as well as the point of occupancy. EDF does not restrict occupancy to being a point-based, CoG computation. The witness metaphor is an intuitive abstraction for expressing occupancy and is advantageous in that it allows for many different types of occupancy representations. A witness in the EDF attests to the time and placement of a specific object on a particular road, intersection, or list of corridors. A witness specifies a description (geometric or otherwise) regarding how the object occupied space on the EDF component. This includes a reference to the occupied object. For instance, our point-based witness provides an autonomous behavior with the following information: (1) a reference to the occupying simulation object, (2) a time-stamp dating the occupancy, (3) the Cartesian coordinate, and one of the following: roads and intersections do not allow for overlap, it is only in the presence of overlapping corridors that a CoG may be on more than one component at a time. 2 Since 75 Road Queries list Witness queryObjects() Returns all point-based witnesses to road’s occupants list Witness queryObjectsInLane( lane ref ) Returns all point-based witnesses to occupants on a particular lane Witness queryLeader( distance, ori ) Returns a witness representing the next object in the direction specified by ori after distance on a road Witness queryLeaderInLane( lane ref, distance, ori ) Returns a witness to the leader occupant within a lane list Witness queryAllLeadersInLane( lane ref, distance, ori ) Returns a list of witness to all the objects ahead of a particular distance Intersection Queries list Witness queryObjects() Returns all point-based witnesses to intersection occupants list Witness queryObjectsInCorridor( Corridor, width ) On demand query that calculates the occupants within a specified width of the corridor axis Witness queryLeaderInCorridor( Corridor, width, distance, ori ) On demand query that calculates the leader ahead of a given distance on a corridor General EDF Occupancy Queries list Object queryAllObjects() Returns all the objects in the simulation Table 4.8: Primitive point-based witness queries provided by the EDF. These queries use point-based witnesses for occupancy computations. 6 74 6 74 6 74 6 74 6 74 6 74 z z z z z z 6 74 6 74 6 74 y y y y y y (a) a reference to a road plus a curvilinear coordinate, (b) an intersection reference, or (c) a list of corridor references plus curvilinear coordinates. During every simulation step, we compute point-based witnesses for each dynamic simulation object, but for roads and intersections only. Corridor occupancy is computed on demand, as needed by behaviors. The reason for this is that most of the time, objects entering intersections are only concerned with the occupants on a small number of corri- 76 dors. The intersection dependency structures support this style of corridor occupancy by modularizing the inter-object relationships on the corridors of the intersection; behaviors need only focus on the objects on specific corridors. Thus, given a particular corridor and intersection, the EDF can compare the CoG positions of the objects in the corridor with the single corridor. Contrast this computation with the calculations required to compute point-based occupancy on all corridors in the intersection. In this situation, each corridor is compared with each object. These computations are more often than not, unnecessary and wasteful. Therefore, we leave it to behavior programs to request corridor occupancy as needed. Table 4.8 lists the primitive point-based queries provided by the EDF. As an example, we can query all the objects on a road by using the function queryObjects with a road reference. This function would return back a list of point-based witnesses whose CoG map onto a projection of the road surface. 4.5.3 Sophisticated Witnesses The EDF leverages the definition of occupancy on to the witness abstraction and computation. Point-based occupancy works fine for some behaviors, such as lane based following, but it may not be sufficient for more complex parallel parking behaviors, or lane encroachment detection behaviors. Moreover, the point-based witness is imprecise; it doesn’t tell the behavior much about how the object is situated at its position. More intelligent, informed, and sophisticated witnesses can be built to supply detailed information about how objects occupy structures in the environment. Consider a line-based witness that represents geometry by one or more line segments, describing how 77 each line segment relates to structures in the EDF. A line-based witness can be used to represent the extent of an object in one direction. The witness may output a list of line segments accompanied by the structures those line segments overlay. For instance, A … &  C ƒ ‡ t„£ƒ †y  ƒ )¡ ‡ ( Š h y ‡ …† $¤ —† w‰¤ % ¤ … ƒ A … & C ‡ C& 2 E& 2 & E& … y T¡ r¤ — …† P¢ B† w„¢ % x ‡ ¤ ‡ ¢ … ƒ  ¡ C& E& … & ™C †Grƒ i A ˜ A ‚ & E& which lists two line segments ( and ) and the structures on which they are positioned. The line segments represent portions of the object referred to by object ref such as a line representing the longitudinal extent of the object and how it may be positioned on the underlying surface. In this example, the object may be changing lanes as part of it is located on lane while the other line segment is located on lane . Similarly, an area-based witness may partition the convex hull of a simulation object into polygonal subsets, describing how each sub-region occupies different space on adjacent lanes, roads, or intersections. Š  ¤ ¢ 4.5.4 Level of Detail Occupancy Queries Often times, the distances between objects is a good indicator for how much occupancy information may be important to a behavior. Nearby objects raise more immediate concerns than do far away objects. For instance, a behavior that avoids collisions with other objects in a lane is more concerned with the objects immediately in front of it than it is with the objects two miles down the road. To facilitate efficient access to occupancy information for behaviors, the EDF performs on-demand occupancy witness calculations with level of detail (LOD) thresholds to manage computation expense. LOD occupancy calculations allow us to concentrate (  ˆwƒ h ry line witness = , … … 78 LOD 0 - Nothing LOD 1 - Point Witness LOD 2 - Line Witness LOD 3 - Area Witness Figure 4.14: Level of Detail (LOD) occupancy queries provide general solution to different occupancy needs. Witnesses at different LODs provide different detail. 79 computation where behaviors need it. Figure 4.14 depicts an autonomous behavior and a visualization of the results from querying LOD occupancy. Four levels of detail are shown. In the example, witness calculations closest to the vehicle provide area-based occupancy information (highest level of detail) and witness calculations farther from the vehicle return point-based occupancy status (lowest level of detail). The objects farthest from the vehicle (outside the outer circle) are not returned as a result of this query. Witness calculations for the higher level LODs are computed on demand and are uniquely described and implemented for each simulation object. For instance, consider the difference between pedestrian and vehicle witnesses. Pedestrian occupancy at the highest level of detail may utilize a bounding circle computation whereas the highest level of detail in a vehicle may perform calculations based on how the convex hull of the vehicle occupies EDF structures. Level of detail calculations are described for each type of object to take advantage of the particular object’s geometric definition. Behavioral objects specify which level of detail occupancy witnesses they require. This is accomplished by providing the occupancy queries of the EDF with a LOD profile stating which LOD witnesses to compute at different distance thresholds. A LOD profile is a list of distance and LOD pairs. For example, U W Œ ¢ ¤ % ( U ’ @ Œ & ‡ & iW Œ —#‚ ‡ A © ‡ ¢ % ( d © % U Œ Ž r& ‡ & iW – ‚ r& ‡ i ‹ © ‡ 6 which specifies that the highest LOD witness calculations should be returned for simulation objects that are less than or equal distance away from the behavior requesting the and ‡ information. The medium LOD should be used for objects between units away, t D( j S¡ LODProfile 80 Road Queries list Witness LOD queryObjects( distance, LODProfile ) Returns road occupancy witnesses based on the LODProfile and the distance parameters list Witness LOD queryObjectsInLane( lane ref, distance, LODProfile ) Returns lane occupancy witnesses based on the LODProfile and distance parameters Witness LOD queryLeader( distance, ori, LODProfile ) Returns a witness based on the LODProfile that represents the next object in the direction specified by ori after a particular distance along the road Witness LOD queryLeaderInLane( lane ref, distance, ori, LODProfile ) Returns a witness based on the LODProfile representing the leader occupant within a lane Intersection Queries list Witness LOD queryObjects( , LODProfile ) Returns witnesses based on the LODProfile and the distance between the intersection occupants and the Cartesian vector provided as a parameter list Witness LOD queryObjectsInCorridor( Corridor, distance, width, LODProfile ) On demand query that calculates the occupants within a specified width of the corridor axis using the LODProfile and distance along the corridor Witness queryLeaderInCorridor( Corridor, width, distance, ori, LODProfile ) On demand query that calculates the leader witness ahead of a given distance on a corridor using the LODProfile Table 4.9: Level of detail (LOD) witness queries for EDF roads, intersections, and corridors. LOD occupancy witness queries are computed on an as-needed basis. 20 31 6 74 6 74 6 74 6 74 z z z z 6 74 6 74 6 74 y y y y 81 ¤ ¢ and the lowest LOD calculation should be performed for objects between ¤ and away. Simulation objects beyond units away are not returned as witnesses. Table 4.9 lists the different level of detail (LOD) witness occupancy queries provided by EDF for roads, intersections, and corridors. 4.6 Paths Throughout this chapter, I have used the word path as a generic description for a navigable route used by simulation objects. In the EDF, the word path has more specific meaning denoting an abstraction designed to support the continuous and smooth navigation of objects over the road networks. For the remainder of this chapter the word path will be used as it pertains to this restricted meaning. A path provides a means for tracking and following objects on a lane-based route over sections of the road networks. Paths are intended to be used for local navigation, providing immediate information about the route and the occupants along that route. Paths are composed of an ordered sequence of (road, lane) pairs and (intersection, corridor) pairs. Paths manage their aggregate parts to present a single lane-based curvilinear coordinate system. Paths do for lanes and corridors what roads do for lists of segments (see Section 4.3). The path abstraction is a design decision influenced purely by behavior modeling issues dealing with intersections. With regards to vehicle behaviors, intersections are difficult to navigate. This is even more so for pedestrian behaviors using hierarchical intersections. A single path eliminates the managerial tasks required by a behavior to negotiate complex ‡ ‡ units ‡ 82 Intersection Boundary Geometry Figure 4.15: Illustration of path use by vehicles. Three paths are shown for three different vehicles. The paths cross into and through the intersection boundary. intersections, and allows autonomous behaviors to utilize fewer tracking mechanisms to drive over road networks. In other words, the path structure encapsulates the important behavioral properties necessary for navigating a route. Figure 4.15 illustrates three vehicles approaching an intersection. Each vehicle utilizes a path to seamlessly integrate the connection between the lane they are on and the corridor through the intersection. 4.6.1 Path Augmentation and Queries Paths simplify bookkeeping needed to manage frequent changes in coordinate systems. As intersections are approached, paths grow and shrink by adding on the upcoming corridors and lanes while removing the lanes and corridors at the beginning of the path. 83 Tracking Queries queryXY( ) Converts from Cartesian to curvilinear space queryDO( ) Converts from curvilinear to Cartesian space. Feature and Range Attribute Queries list RangeAttr queryRangeAttributes( ) Returns a list of range attributes along the path list Feature queryFeatures( ) Return a list of features along the path including references to stopline and curb corridor attributes Witness Queries list Witness LOD queryObjects( , LODProfile ) Returns witnesses occupancy information for the path based on the LODProfile and the given distance parameter ( ) Witness LOD queryLeader( , ori, LODProfile ) Returns a witness based on the LODProfile that represents the next object in the direction specified by ori after a particular distance, , along the path Table 4.10: Queries provided by paths in the EDF.   v ( ¡ % )¡ % ( & { &{   & & '  ( Q¡ & ¥  %     ( )¡ 201 & '  6 74 % 6 74 6 74 z   z z 6 b4 6 74 6 4 20 7a31 y y y ( )¡ & '  % Paths can be built incrementally, on an as needed basis by behaviors, using the augmentation routines for appending and removing from the path. Thus, a path is not merely a static description of a route. Although paths can be used to describe specific routes and plans for autonomous behaviors, the intended use of the path is as a local, relative description of the surface of the route. Paths are lane based and not road based. It is up to behaviors to augment and create paths as needed. For instance, during a lane change, the path may have to be reconstructed by the behavior. Table 4.10 lists the queries provided by the path structures in the EDF. Because the path represents a curvilinear surface description, the queries provided by curvilinear surfaces are also supplied by the path, as discussed in Section 4.2. 84 CHAPTER 5 SCENARIO MODELING WITH A SCENARIO SCRIPTING LANGUAGE This chapter describes the Scenario Description Language (SDL) for interactive modeling and prototyping of scenarios in real-time virtual environments. SDL is an interpreted scripting language that interactively evaluates expressions entered by an experimenter, or programmer, and translates them into HCSM that are then inserted into the simulation execution framework. The design and feel of SDL is influenced by our experience in building scenario programs with the HCSM framework. The HCSM paradigm is well-suited to expressing the types of computations required by scenario directors. However, the nature of our lowlevel state machine language can be cumbersome and unnecessarily complicated for director programming. The resulting programs obscure the temporal relationships and event sequences amongst actions in the scenarios. Moreover, these programs require compilation which lead to larger development times involving extended edit, compile, and run cycles. The result is that we are unable to quickly prototype and experiment with new control strategies for scenario programs within our simulation environment. SDL acts as the foundation for building and experimenting with scenario director agents by placing emphasis on simplified process generation while retaining the execution semantics of HCSM. SDL language constructs support highly distributed control strategies 85 that merge event-based and continuous, interval-based computations. Directors independently monitor the progress of the simulation and manipulate the environment by sending instructions to other agents. In scenario development, strong constraints between location and timing of events are required. Expressions in the SDL can establish tight connections between objects, sending streams of messages that continuously adapt behavior to the scenario. SDL follows common programming practice and supports basic language constructs found in most iterative programming languages including variables, assignment, iteration, comparison, boolean logic, conditionals, sets, and functions. These traditional constructs are augmented in SDL with highly specialized language tailored for scenario programming. For example, scenarios frequently create, destroy, and send messages to objects in the simulation. Often times, scenario directors will monitor simulation events and wait for opportune times to send messages. SDL provides language components for the creation and direction of simulation objects while facilitating access to the runtime EDF database and queries. Moreover, SDL utilizes process-based constructs to describe the event-based, as well as continuous computation necessary for scenario direction. Through these mechanisms, SDL supports design and rapid prototyping of scenarios, as well as interaction with simulated environments. 5.1 SDL Basics - Iterative, Interpreted Programming The design of SDL borrows heavily from existing scripting languages’ syntax and semantics to reduce the learning curve of SDL and to increase the flexibility of SDL as 86 a scenario programming language. For instance, scripting languages such as Perl, Tcl, or Python are interpreted, weakly typed, and designed for rapid development and prototyping. Typical scripting languages glue together different components into a single framework. Scripting languages differ from their counterpart system programming languages, such as C or C++, which are compiled, strongly typed, and more specialized to fit the machine hardware (and hence, require more development effort) [48]. Different languages have different purposes and different styles. Combinations of languages can be used to attack computational problems at different levels of interaction providing insight and simplicity into solving problems. We envision a full suite of scenario programming languages that work together to fill the gamut between simple, scripted scenario languages and more powerful, compiled languages [35]. SDL is a high level, scripting language for scenario modeling. In this sense, it is intended to be easier to program than a semantically equivalent set of C++ code. SDL is meant for play and experimentation with scenarios acting as a glue between the running simulation, EDF components(roads, intersections, queries, etc), and HCSM behavior programming framework, allowing scenarios to be created and modeled interactively. This reduces development time and promotes investigation into new and different scenario control strategies. 5.1.1 Statement Execution Semantics In SDL, we distinguish between two types of statements: activities and monitors. These commands separate the what from the when. In both cases, SDL statements are 87 interpreted in a serial manner, executing commands in the order they are entered. Activity statements represent basic SDL commands (i.e. what we want to do). These commands are evaluated immediately and executed directly in the simulation. Activity statement execution is conceptually instantaneous. Monitor commands, on the other hand, represent the statements that determine when we want to do something in SDL. Monitors encapsulate both activity statements and monitor statements. When we execute SDL statements is often contingent on conditions or events detected within the simulation environment. At times, we use monitor statements to establish persistent influence on objects in the simulation by continuously evolving and adapting message streams to fit the evolving simulation state. At other times, monitor statements take on more impulsive and instantaneous forms waiting for an event and then issuing commands to react to the event. Our notion of persistent influence and incremental adaptation to simulation state differs from animation-based scripting languages like Improv [52] and Alice [16]. Improv and Alice are action-based scripting languages. In these systems, actions are independent, preprogrammed routines that control some or all of an object’s degrees of freedom for a set amount of time. When one action is executing other actions may be prevented from executing. Language commands can be issued to objects telling them which action to perform. Both languages’ goals are to make 3D animation more accessible and easier to script. As an example, in Alice, we could tell a robot to rotate it’s right arm upward 180 degrees. This action has a very distinct start, middle, and end. In fact, in both these 88 Assignment operator = Numeric operators +, -, /, *, ++, --, % Comparison and Boolean operators <, >, <=, >=, ==, !=, ||, &&, !, true, false Set operators (union, intersection, difference, complement, member access, empty set) +, -, /, , [], null ˆ Table 5.1: Operators defined in SDL. languages there is a very concrete and explicit representation for the starting and ending of actions. In SDL, no such distinction is made. Monitors must observe the environment to determine if the influence intended by the programmer has been achieved. 5.2 Activities - Instantaneous Commands The SDL interpreter includes common language features, such as scoped variables, assignment statements, conditional branching, and iteration. SDL is type-less; the contents of a variable can take on any type. The SDL interpreter associates internal type information with variables depending upon how the variable is used. While variables in SDL are typeless, they exist in one of two forms: a singleton reference or a set. A reference is a singleitem variable useful for holding integers, floating point values, strings, or references to simulation objects. Sets are possibly empty containers of variable references. The empty set is defined by the null keyword. In the SDL, it is common for sets of objects to be returned as a result of querying the EDF runtime database components (e.g. the set of all simulation objects, or the set of all roads in the EDF). Additionally, sets can be ordered 89 depending upon the circumstance. In these instances, it is useful to treat sets like arrays, indexing the individual elements. For instance, querying a simulation object’s Cartesian position will return an ordered set containing the component parts (e.g. [X, Y, Z]). Because variables in SDL are weakly typed, many operators are overloaded to extend functionality and readability of code. A list of SDL operators is provided in Table 5.1. Context is determined from use and internal type information. Variables in SDL are not dynamically bound and therefore, do not refresh their values automatically, based on changes to the simulation environment. SDL provides a means to group tightly connected activities together into blocks that provide the semantic infrastructure to execute groups of statements immediately, as if the combined statements were a single atomic statement. For example, we might want to direct one car to change to the left lane at the same time a second car is directed to change to the right lane. Both commands to issue the change lane directives would be placed within enclosing curly braces. Within such code blocks, the order of statement execution is preserved to ensure consistency of code. Enclosing SDL code in curly braces determines variable scoping. Variables that are not encapsulated by curly braces are considered globally accessible with respect to top-down scoping rules. Variables within a set of curly braces are only accessible to the code within that block and sub-blocks. In addition to basic calculations, SDL provides bindings to mathematical and simulation system functionality through the math object. These functions are listed in Table 5.2. A special variable, labeled math, provides access to these functions. Consider determining 90 sqrt ln max asin random floor cos atan2 seed ceil sin simtime exp abs tan simframe log min acos systime Table 5.2: SDL defined mathematical functions accessible through the math object. Additional functions can be bound to the math object and used in the SDL environment. the distance between two 3D Cartesian coordinates, stored in the set variables: p1 and p2, x = p2[ ] - p1[ ]; y = p2[ ] - p1[ ]; z = p2[ ] - p1[ ]; distance = math.sqrt( x*x + y*y + z*z ); The set variables p1 and p2 hold vectors representing Cartesian coordinates. Their individual components can be accessed using the index ([]) operator. The math library is accessed providing the square root function to complete the distance calculation.  ~ Š  ~ Š 5.2.1 Simulation State and EDF Query Access The SDL interpreter interfaces directly with the simulation and provides read-only access to simulation state variables through the EDF. The interpreter can influence the simulation environment only by creating, destroying, and directing simulation objects. All access to simulation state is achieved through a dynamic binding between the SDL and the components and queries of the EDF. A global variable, labeled edf , provides a means to directly query the EDF. An object based syntax permeates the SDL’s use of variables referring to simulation objects, such as vehicles, pedestrians, roads, or the EDF. Any query provided by a compo- 91 nent of the EDF is accessible through the SDL’s bindings to the EDF, as illustrated by the syntax object reference.query name( parameter list ); Consider querying the dynamic objects in the simulation (i.e. all vehicles, pedestrians, bicyclists, animals, etc). We would utilize the edf object and the EDF occupancy query queryAllObjects (as described in Section 4.5 of Chapter 4). This query returns a set of references to all the dynamic objects in the simulation environment. The SDL syntax would be all objects = edf.queryAllObjects(); which would result in binding the variable all objects to a set containing the objects currently residing in the simulation environment. 5.2.2 Dynamic Object Creation and Direction Scenario directors influence the environment by (1) creating autonomous agents, (2) destroying agents, or (3) sending messages to agents to manipulate and influence their behavior. The targets of these activities can be material agents or other directors. The SDL defines three language elements for performing these activities: create, destroy, and send. 5.2.2.1 Placement in the Environment Creating objects requires that we be able to place them at appropriate places in the environment. SDL ties in to the EDF’s notions of location by defining a special function la- 92 sdl.locator( xyz, X, Y, Z, R, P, H ); Creates a location structure based on Cartesian coordinates and orientation. sdl.locator( rdo, road ref, distance, offset, orientation ); Creates a location structure based on a curvilinear coordinate on a road. Required parameters are a reference to a road, distance, offset, and orientation on the road. sdl.locator( rdl, road ref, distance, lane id, orientation ); Creates a location structure based on a curvilinear coordinate on a road. Required parameters include a reference to a road, distance, lane id, and orientation on the road. sdl.locator( isect, intersection ref, corridor ref, distance, offset, orientation ); Creates a location structure based on a curvilinear coordinate on a corridor in an intersection. Required parameters are a reference to the intersection and corridor, as well as a distance, offset, and orientation on the corridor. Table 5.3: Illustrative SDL commands for specifying locations in the simulation environment. beled locator that packages up explicit descriptions of places in the EDF for use in creating objects. The locator function is a member of the sdl object. Valid location specifications are listed in Table 5.3. Parameters to the locator function specify the type of place, as well as the position and orientation at that place. Objects can be created at specific Cartesian coordinates, on roads, or on corridors in intersections. 5.2.2.2 Create The create statement issues a command to the simulator that instantiates the object specified by the type and parameter list. Create commands use the following syntax: reference create type( = y © ¢  ‰" y ¢  ‰" y y , © , = A @ B‘… f ’ C q q p rrq A @ ‘… f ’ y C y p 6 74 ) 93 When simulation objects (e.g. vehicles or pedestrians) are created, it is necessary to pass to their constructors any initial state information required for instantiation of the object. The named, parameter-value pairs provide a means to specify initial object state without enforcing strict parameter orderings. Moreover, the named parameters provide language binding flexibility by forcing the object to parse the parameters and use only the ones needed and understood. It our intention and belief that the object types in the EDF environment will change rapidly and frequently over time. Thus, minimizing any hard and rigid links between SDL and EDL is a useful and necessary design consideration. Upon simulator instantiation, the create statement returns an object reference that can be used at later points in the scenario to refer back to the object. For example, consider the code to create an initially upset pedestrian midway along the sidewalk lane (lane 3) of Main Street. road = edf.findRoad(“Main Street”); loc = sdl.locator( rdl, road, road.length()/2.0, 3, 180.0 ); p = create pedestrian( mood=upset, place=loc ); Initially, we locate the road “Main Street” in the EDF. Then we form a specific location that can be passed to the pedestrian’s place parameter. When an object is instantiated using the create command, the parameters used to initialize the object are passed using call by value semantics. This design decision ensures that the connection between the running simulation and the SDL interpreter minimizes side effects and communication costs. 94 5.2.2.3 Send To orchestrate activities in a scenario, director agents monitor the progress of the simulation and manipulate the environment by sending instructions to other agents. In the SDL, the send command is expressed with the following syntax: send object ref message type( , , ‡ rrq q q ‡ ‡ The send command interfaces with the directives programmed into behavior objects. Directives are only received; they are never acknowledged. It is important to emphasize the one-way nature of communication supplied by the send command. In our style of scenario direction, directors can only send messages to objects in the simulation and no response is sent back. Thus, there can be no dialog between scenario directors and objects in the simulation. In fact, behavioral objects, such as vehicles or pedestrians, are unaware of the scenario directors’ presence in the environment. Since directors cannot expect an acknowledgement, the only way they know that objects are responding to messages is by observing their behavior. Moreover, objects choose how to respond to directives, so there is no guarantee that a directive will be applied immediately. In actuality, it may be the case that a directive cannot be performed at the instant it is received. Additionally, messages sent from SDL do not directly set object state variables. This style of direction rules out negotiation. Directors cannot send out bids for actions, selecting the optimal bidder only when all bids have been received. Languages for driving simulation and scenario programming, such as SSL [66] and ISL [22] make tighter commitments than SDL to scheduling scenarios contingent on the ¢  " y  ‰Q'ry , ) ¢ y  ˆ'ry © y  Q'ry 95 starting and stopping of actions. Both SSL and ISL provide explicit notions for the beginning and end of an action. With this information in these languages, scenario programs can be scheduled using simple action sequences. SDL has no explicit notion for scheduling since we don’t know when activities begin or end. There are good reasons for SDL to be organized in this manner. For one, timing constraints are much simplified. Directors can issue directives and be done; they don’t have to wait until the command completes before attempting other tasks. If directors do require that an action be performed, they can continuously monitor the situation and direct the action to occur until it happens, or until such time passes that the action becomes useless. Our thought is that by removing scheduling and explicit notions of beginning and end from activities, SDL becomes more simple and free for play and experimentation with distributed control. There are disadvantages to SDL’s one-way communication style. Scenario director programs must take responsibility for monitoring the environment state to determine when the intended activities have completed. 5.2.2.4 Destroy In SDL, objects can also be destroyed through the interpreter. This is expressed by destroy object ref in which an object reference variable is used to specify which object in the simulation should be removed. 96 5.2.3 Conditionals SDL provides standard if-then/else constructs for conditional branching. The ifthen/else statements are conditioned by Boolean expressions and comparative operators. The branching test is executed immediately, using the current state of all variables referenced in the condition. For instance, the code z if (car1.speed() i t i t i t = 20.0) send car1 stop(); else send car1 increase speed( 20.0 ); performs an immediate test of the current speed of car1. If, at the moment the code is executed, the vehicle’s speed is greater than or equal to ~ ‰Š ~ q , a stop directive will be sent ~ ‰Š ~ q to the vehicle. Otherwise, the vehicle will be instructed to increase it’s speed to . 5.2.4 Iteration In SDL, two constructs are provided for iteration: forall and for. The forall statement works with set variables to iterate SDL commands over each member of a set. The forall variable is unified with each member of the set and can be used in the forall code block to refer generically to set members. For instance, consider instructing all members in a set of vehicles to change lanes to the left, forall v in vehicle set send v change lane( left ); 97 Code like this can be useful for creating empty space in front of another vehicle ensuring a free driving zone. The for statement works similarly iterating a finite number of times. One use of the for statement could be to populate a road with vehicles, as in offset = 2.5; for (distance=0; distance 1000; distance=distance+100) l = locator(rdo, road ref, distance, offset, 180.0); create vehicle( place=l ); t y i i t Both types of loops in SDL are executed in a manner analogous to a macro expansion, or loop unrolling. In other words, loops are unrolled and then each inner statement is executed in the order it is interpreted. Loops provide a convenient and powerful syntax to specify iterative statement execution. 5.2.5 Function Definitions Often times, computations are reused throughout a scenario program. SDL facilitates reuse and abstraction through basic parameterized function definitions. Functions in SDL are parameterized with call by value semantics. SDL functions return a value using the return command. SDL functions take simple activities and group them into commands that can later be invoked through the sdl object. For instance, consider defining a function to return a random value in a defined range. Upon interpretation of the following code defun randomValGenerator( lo, hi ) randval = math.random(); return lo + (hi-lo)*randval; 98 the function randomValGenerator would be registered with the SDL interpreter and accessible through the sdl object. For instance, rval = sdl.randomValGenerator( 12, 92 ); would return a random value between 12 and 92 inclusive and assign that value to the variable rval. Functions are executed as if they were typed in place of the function call itself. In the previous example, the assignment statement and function call would happen instantaneously together, as one activity statement. Using curly braces to denote an atomic set of activities, the previous example could have been written i lo = 12; hi = 92; randval = math.random(); rval = lo + (hi-lo)*randval; t If we need to execute this code many times each with different limits for the high and low values, a parameterized function is much more useful. Hence, the purpose of including functions in SDL is to facilitate development effort by reusing and parameterizing commonly called code blocks. 5.3 Monitors - Computational Language Constructs Scenario programs require careful synchronization of activities and events. Activities, or sets of activities, performed in a scenario are often contingent upon other activities or happenings in the environment. Some activities may need to be grouped together and 99 executed simultaneously, while others may need to be executed sequentially over a period of time, or in order at an instant. SDL introduces monitor statements that dictate when activities occur in scenarios. Monitors are treated as language constituents that determine the time and frequency with which sets of statements are executed. The timing of scenario activities frequently depends on situations and events that occur at unpredictable times during the simulation. Thus, scenario directors must observe the simulation to detect when these conditions are satisfied. Sometimes, the prerequisite conditions can be characterized as predicates of state variables, expressed by Boolean operators on state variables obtained through queries to the simulay tion database (e.g. vehicle.speed() 55.0 ). At other times, the required conditions are best Œ expressed in terms of events (e.g. a vehicle crosses a line ). SDL differs from conventional scripting languages by it’s focus on language constructs that instigate potentially long-term processes in the simulation system. With SDL, highly distributed control strategies can be designed with many scenario directors executing different, yet potentially related portions of a scenario. Simulation scenario processes are specified in SDL with the following monitor constructs: aslongas, whenever, every, and when. These four commands encapsulate the computations required in a scenario providing contingent execution of SDL activities based on situations in the environment. Figure 5.1 illustrates the nature of the four monitor commands. The figure divides the monitor statements into two dimensions describing how the monitors execute (i.e. Impulsive, 100 Impulsive, Event-Based when every Once aslongas whenever Periodic Continuous, Interval-Based Figure 5.1: Illustration of the two dimensions used to define SDL monitor commands. Monitor statements are predicated by either events or state-based queries and can execute once or periodically. Event-Based or Continuous, Interval-Based) and the frequency with which they execute (i.e. Once or Periodic). These dimensions are a direct result from our experiences programming scenarios. On one hand, scenario directors must continuously monitor the environment setting up compliant relationships between the director and the autonomous characters being directed. This style of direction prepares the scene for future situations and requires that scenario directors maintain these compliant relationships over continuous intervals. Moreover, the intervals of execution are based on the values of state variables in the simulation, resulting in continuous querying and monitoring by the scenario director. On the other hand, scenario directors can be impulsive, reacting to events occurring in the simulation. In these 101 situations, the scenario director needs to be notified that a particular event has occurred so that it can react appropriately sending a burst of directives to the autonomous behaviors. Additionally, the figure shows subgraphs detailing what predicates the execution of each command. Each subgraph represents an abstract predicate that can take on a true or false value. For the impulsive, event-based monitors, the green arrows represent the reaction to events generated by the simulation. Events represent the change of a state variable over a threshold. Hence, the arrows are poised at the leading edge of the transition when the state variable crossing a threshold becomes true. The arrow further denotes that the impulsive monitor is active for an instant after receiving notification of the event. The when monitor reacts only to the first event whereas the every monitor reacts to successive events. For continuous, state-based monitors, the subgraphs show the continuous nature of execution of the aslongas and whenever monitors. This is represented by the green shaded areas in the subgraph where the predicate is true. The aslongas monitor is activated initially if the predicate is true and remains active while the predicate stays true. The whenever statement reactivates itself during periods when the predicate is true. When invoked, the four monitor constructs result in the creation of separate processes in the simulation that are distinct from the SDL interpreter process. Monitor commands execute regularly within the simulation framework and at a rate consistent with the autonomous behavior execution cycles. 102 aslongas( Cond ) { SDL Block } aslongas Start State Cond ... !Cond End State !Cond Figure 5.2: State machine diagram that illustrates the necessary transitions and activation states for the aslongas monitor. 5.3.1 5.3.1.1 Continuous, Interval-based Actions Continuation: aslongas A scenario typically consists of a series of episodes during which autonomous agent behaviors are coordinated, interspersed among periods of free running activity during which agents are left to their own devices. During periods of orchestration, directors are responsible for guiding agent behavior to satisfy scenario constraints. This involves tasks such as monitoring circumstances, coordinating events, finely adapting agent behaviors, and restricting behaviors from actions that would interfere with planned activities. Much of scenario directors’ work is often spent in setting the stage for a critical event. For example, we may need to create a gap in traffic ahead of the subject’s vehicle to make room for a particular situation. This could be accomplished by directing vehicles in front of the driver to change lanes, or increase speed. Preparations often require that conditions be sustained over a period of time. An aslongas statement executes its block of statements continuously, routinely ver- 103 ifying its guarding condition. The aslongas command terminates upon the false evaluation of its predicate condition. The SDL interpreter transforms an aslongas statement into an equivalent HCSM and places it into the simulation environment. The code contained in the statement block is executed during the activity phase of HCSM execution. Figure 5.2 illustrates the state machine nature of the HCSM used for representing aslongas statements in the simulation environment. Commands in the statement block of an aslongas monitor are repeatedly evaluated and executed. Through this stream of commands we continuously influence the environment and guide autonomous agent behaviors to meet the changing simulation state. Consider implementing a scenario to continuously send passing messages to a vehicle as long as the speed of the lead vehicle is less than a predetermined threshold // Determine the vehicle’s current curvilinear coordinate information, // which returns the set: [distance, offset, orientation]. doc = v.queryCurvilinearCoordinate(); y aslongas( v.queryLeader( doc ).speed() i t threshold ) send v pass( lead ); In this example, we use adaptive direction to influence the selected vehicle v to pass the vehicle in front of it when it might not otherwise do so. 5.3.1.2 Intermittent Continuation: whenever The whenever monitor addresses the need for intermittent direction contingent on environmental conditions. The statement block of a whenever monitor is executed during 104 whenever( Cond ) { SDL Block } whenever Start State Cond ... !Cond Figure 5.3: State machine diagram for the whenever monitor. all periods that the guarding conditions are satisfied. Scenarios frequently call for attention to circumstances that arise repeatedly at unpredictable times. For example, // Obtain access to the human driven vehicle subject = edf.ownship(); whenever( subject.distanceToLeaderInLane() send subject.leader() change lane(); t y i threshold ) This example begins by obtaining a reference to the ownship, or human driven vehicle in the simulation. The SDL code presented in this example accesses simulation object information to return the data on the leader object in the lane the subject is in. Whenever the distance between the subject and its leader becomes less than a specified threshold, we stream change lane statements to the lead vehicle to clear the path for the subject. Like the aslongas statement, the code block of the whenever statement is transformed into an activity function of an HCSM. The HCSM representing the whenever statement in the simulation is illustrated in Figure 5.3. 105 5.3.2 Impulsive, Event-based Actions Frequently, the prerequisite conditions for a scenario depend on changes in state such as a vehicle crossing a line on the road, or entering an intersection. For instance, a scenario may be contingent upon the subject vehicle’s speed and will only be performed once the subject’s speed has crossed a certain threshold. The transition moment, defined as the crossing of the state variable’s value over the threshold marker, signifies an event. Event-based state changes are different from the queried state variable conditions used by the aslongas and whenever statements. Event detection requires knowledge about the history of the state variable to determine that the transition occurred, whereas conditions test against immediate state and may potentially be true over periods of time. Events are instantaneous. 5.3.3 Sensors and the Event Algebra SDL introduces a sensor as an event monitor. Sensors are defined by a state change to be detected. The sensor automatically maintains the state memory needed to test for change and creates a process to monitor the specified change condition. When the sensor detects the change condition, a globally accessible event is registered at that instant. Scenario activities can be conditioned by these sensor events. By making sensors language constructs, we eliminate tedious bookkeeping and management of state over time. Events are used to detect state variable movement across boundaries. These state changes require knowledge of history and hence require monitoring of the state variable over time. We express events in an event algebra. The event algebra is a notation that 106 St-1 α    St St α St-1 a threshold , two different boundary crossings are defined  y ¢  ’„  z  9 ‘  : “ and and As illustrated in Figure 5.4, the operator defines the transition of the state variable as it  •„  ” succeeds the threshold value. At time  , is less than the threshold value, but at time “ , the value of  has exceeded the threshold. The and defines the event to be when the state variable value becomes less than the threshold value. As an example, the event marked by the transition of an object’s speed from less than to greater than ~ ‰– ~ q ~ ‰– ~ q can be expressed as speed ‘ 30.0 The event algebra expresses entry into regions by forming disjunctions of multiple algebraic statements   or  For instance, consider determining whether a vehicle, v, enters a region defined by curvi-  z ¢  ’„ “  y  9 : ‘ operator works in the opposite manner  defines what it means for a state variable to cross a threshold. Given a state variable ¢   Figure 5.4: The event algebra is based on how the values of successive state variables ( ) relate to the threshold . , and ‘  107 δ1 offset ο2 δ2 distance Figure 5.5: A region being described by the event algebra. illustrated in Figure 5.5). The event algebra expression is (( ¤ ¡ Pr¥Rœ % (¢ ¡ P£›š % ( D¢ v.distance “ and v.offset and v.offset % % and v.offset and v.offset % % In our simulations, crossing lines, entering regions, exiting regions, and waiting for timers to go off are common events. SDL facilitates sensor use by providing abstractions to encapsulate the semantics defined by the event algebra. SDL sensors are parameterized by event type, object reference, region, and event name. The syntax of an SDL event is event( event type, object reference, region or threshold ) triggers event name; Events described in terms of a single variable, can be more simply expressed by using the t (( „P†¤ œ % ( D¢ š % ( ¤ †'¡ v.offset and v.distance and v.distance (( P†¤   œ ( D¢   š (¢ D£¡ v.offset “ and v.distance and v.distance (( ¤ ¡ Pr¥Rœ   (¢ ¡ P£Rš   ( †¤ v.distance ‘ ‡ †¤ linear coordinates bounded by the distance range and the offset range or or or ‡ ¤ ¢ ¡ r'¡ £™… & —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ ¢ …   r&   —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜ —˜     —˜ —˜ —˜ —˜ ‘ —˜ —˜ —˜ —˜ — — — — % D% % D% % D% % D% i ο1 (as 108 ž   double greater than ( and “ ‘ ) and double less than ( ) operators and associating them with the operators from the event algebra. For instance, to create an event that will detect when a vehicle’s speed surpasses 30.0 mph, we can simply express it in SDL, as in  vehicle.speed() ž 30.0 The operators and will be bound to event descriptions, defining events in which state variable values cross thresholds (e.g. leading edge or trailing edge transitions). Thus, these operators can only be used in the when and every monitor statements. Events for timed, periodic events can be specified with the timer event. Timer events issue a single event and are not periodic themselves. A specialized timer event, labeled continuous timer can be used to set up a continual timed event based on the duration of the timer. To create a timer event to go off after 30 seconds, we could write event( timer, null, 30.0 ) triggers Timer30Sec; In this example, the event type is a timer event that works in terms of seconds. Timer events are not triggered by simulation objects, but rather by the internal simulation clock, so a null argument is provided as the object reference. After 30 seconds has elapsed in the simulation, the timer event Timer30Sec becomes true. Events for crossing lines, and entering or exiting regions are commonly occurring events within scenarios and can be specified using the crosses, enters, or exits event labels, respectively. Regions and lines in the environment are described using SDL set notation. A region takes up a 2D area in the environment and may be defined in terms of Cartesian space, or curvilinear space. Lines are similarly described. Table 5.4 outlines the SDL set 109 Event Line Set Notation [cartesian line, [ , ], [ , ]] Describes a line in 2D Cartesian space by specifying two coordinates. [curvilinear line, Curvilinear Object Reference, distance] Describes a line in curvilinear space. Requires a reference to the curvilinear structure and a distance. [curvilinear line, Curvilinear Object Reference, distance, offset , offset ] Describes a line in curvilinear space. Requires a reference to the curvilinear structure, a distance, and two offsets. Event Region Set Notation [cartesian region, [ , ], [ , ], , [ , ]] Describes a convex polygon in 2D Cartesian space by specifying at least three coordinates. [curvilinear region, Curvilinear Object Reference, [distance , offset ], [distance , offset ]] Describes a curvilinear region bounded by lower left and upper right curvilinear coordinates. Table 5.4: SDL set notation for describing event lines and regions. ¤ ¢   ‰" ¢  x ¥w p Ÿ¢  r‰" x ¥w p q q $rq  ¢ p u vu Ÿ r¢ ¤ p E ¤ —†   S© u vu p ¢ Ÿ ¥© E p ¢ B† notation that is used to describe event regions and lines. Using this information, consider the SDL code to detect when a vehicle, denoted by v, crosses the line specified by [x1, y1], [x2, y2] event( crosses, v, [cartesian line, [x1,y1], [x2,y2]] ) triggers crossesLineEvent; To simplify entrance onto (or exit from) roads, or intersections, SDL provides a function binding that references the geometry of simulation objects such as roads or intersections in a manner consistent with SDL event region notation. This function is accessed through the respective object and is labeled geometry. As an example, to determine when a vehicle v enters the intersection isect, we can create the event event( enters, v, isect.geometry() ) triggers IsectEntranceEvent; 110 Event Record type instigator threshold timestamp Table 5.5: The components returned in an event record describe the type of event, who instigated it, the event threshold or region value, and the time the event occurred. SDL events can be specified using sets of objects in place of the object reference. In these situations, the event will be fired when any object in the set causes the event condition to become true. Often times, it is common to fire events through the course of a simulation for any object that triggers the event. For instance, consider creating a line crossing event for maintaining a count of vehicles that cross a particular mile marker on a road. This can be expressed in a SDL event by setting the object reference to null, signifying that any object that crosses the line should trigger the event, as in event( crosses, null, [curvilinear line, road, distance] ) triggers Counter Event; When an event is triggered, an event record can be accessed by monitor statements contingent on the event to obtain information about the instigator of the event. An event instigator may be system triggered (as in the case of the timer), or it may be a reference to the object that crossed the line or entered the region. A time stamp is also associated with the event to provide sufficient detail about the moment the event was detected. The event record is accessed through the label given to the event. For instance, if the event label is EntranceEvent, then EntranceEvent.instigator provides a reference 111 to the object that triggered the event. EntranceEvent.type provides the event type. EntranceEvent.threshold returns a set reference that describes the region or event threshold value with which the event fired. EntranceEvent.timestamp provides the simulation time for the moment the event was detected. The components of the event record are summarized in Table 5.5. The SDL cannot guarantee that an event will always be detected. Events can always be formulated that will be difficult or impossible to detect. However, when an event is caught, it is guaranteed that all event-based monitor statements awaiting that event will have a chance to execute. Sensors are a powerful means to express events commonly used by scenario programs in the simulation environment. Events are used strictly as guards to impulsive actions. Impulsive actions react to changes in simulation state and serve as instantaneous modes of direction. There are two statements in SDL to handle event-based actions: every and when. 5.3.3.1 Intermittent Actions: every In addition to creating scenario episodes, directors take responsibility for regulating the ambient characteristics of the environment such as adjusting traffic and pedestrian densities. Background activity is important for realism. Such maintenance tasks often require periodic attention. The every statement executes its code block once per event. The every statement is transformed into a HCSM that executes its code periodically as a function of the every’s guarding event. Figure 5.6 details the HCSM used to provide every semantics 112 every( Event ) { SDL Block } every Start State Event ... true Figure 5.6: State machine diagram for the every monitor. in the simulation. An every monitor statement is persistent and remains active indefinitely, executing its code block each time its guarding event is raised. For example, the following SDL code will create a new vehicle at the same location every four and a half seconds: loc = sdl.locator( rdl, road, 10, 3, 180 ); event( continuous timer, null, 4.5 ) triggers CREATE EVENT; every( CREATE EVENT ) ¡ ¢ create vehicle( place=loc ); 5.3.3.2 Deferred Actions: when Many of the situations we wish to create require specific conditions to hold before action can be taken. The condition could be that a car enters a region, a state variable surpasses a threshold, or that two simulation objects stand in some particular relationship to each other. Using a when statement, we can defer execution of a set of activities until a 113 when( Event ) { SDL Block } when Start State Event ... true End State Figure 5.7: State machine diagram for the when monitor. specified event arises. In the following code snippet, the when statement is used to instruct the first object to cross a line on a road to stop once the line is crossed: event( crosses, null, [curvilinear line, road, distance] ) triggers Stop Event; when( Stop Event ) ¡ ¢ send Stop Event.instigator stop(); In this example, the when statement executes once an object crosses the line on a road. The code inside the when statement tells the object to stop. The semantics of the when statement ensure that only the first event will be caught. Consecutive events (i.e. when other objects cross the line) will not be handled by the when statement in the example. A when statement waits for an event and executes its body once the event is raised. In a when statement, the body will execute only once. When statements are especially useful when we cannot anticipate the exact timing of an event. 5.3.4 Nesting - Mixing Interval and Event Monitors Complex foundations for direction can be created by nesting monitor statements within one another. For instance, consider a whenever statement nested in the body of an 114 aslongas statement. The intended semantics links the fate of the nested whenever statement to the continuation of the aslongas statement. aslongas( vehicle.road type() == FOUR LANE ROAD ) £ ¡ ¢ ¡ ¢ ¡ ¡ whenever( vehicle.speed() threshold ) send vehicle increase speed( 0.10 ) The power of nesting is that it allows us to define intricate relationships among the conditions that govern which activities to take. It is meaningful to couple the destiny of nested statements to the continuation of their parent statements only when the nesting statement is active for a continuous interval of time. Therefore, statements nested within aslongas or whenever statements are terminated when the nesting aslongas or whenever becomes inactive. In this regard, aslongas and whenever statements enforce a parent/child, hierarchical relationship on the execution of their nested commands; child statements are only executed when the parent is active. Treating nesting of continuous statements within impulsive, event-based statements (every or when) in the same manner would lead to momentary activation of conditions that would appear to have more long-term consequences. For instance, consider the following example: when( X ) £ ¢ ¢ aslongas( vehicle.offset() threshold ) send vehicle change lane( left ); 115 In this example, if we tie the execution of the nested statements to the execution framework of the parent statement, the aslongas statement will only execute for the instant that the when command is active. The intended affect of the aslongas statement on the simulation state is to force the vehicle to eventually change lanes by streaming lane change messages to the vehicle until the desired offset has been reached. For impulsive statements (every or when), it is more useful to treat nested conditionals as independent directors that are initiated when the monitor containing them is executed. In a manner similar to macro expansion, the body of a when statement replaces the when statement the moment the event guard fires. Thus, an aslongas statement nested in the action block of a when statement is deferred until the when event becomes true. At that point, the aslongas statement commences as if it were entered in place of the when statement. Another way of describing impulsive nesting semantics is that the outer level of nesting is removed and the inner statements become liberated and free to execute independently. This macro-like interpretation for impulsive nesting permits sequences of activities to be simply expressed in a series of nested when statements. For example, when ( X ) v = create vehicle; when ( Y ) send v message(); ¡ ¢ ¢ ¡ When X occurs, the inner block will be executed command by command. There is no relationship with the parent so once X is detected, the inner body of the when is expanded and executed. 116 As a second example, consider using an every statement to periodically create new vehicles. At creation time, we often want to attach directors to autonomous agents to monitor the new object’s actions and guide it to behave in some way. In the example below, we create a monitor for each newly created vehicle to instruct the vehicle to decelerate whenever it’s speed exceeds a maximum value. loc = sdl.locator( rdl, road, 10, 3, 180 ); event ( continuous timer, null, 30.0 ) triggers timer 30sec; every ( timer 30sec ) v = create vehicle( place=loc ); whenever ( v.speed() max ) send v reduce speed(rate); ¡ ¤ ¡ ¢ ¢ Twice a minute, the every statement is activated for an instant. During each momentary activation a new whenever statement is activated. As with when statement nesting, the nested whenever statement continues to be active beyond the moment of its creation, continuously monitoring the speed of the vehicle for which it was created, sending reduce speed messages when necessary. 5.3.4.1 Difficulties with Nesting Continuous Monitors Our design decisions for determining nesting semantics are based on the nature of the different monitor statements. For impulsive monitor statements such as when and every it makes sense to let the body of the monitor stand alone once the guarding event is detected. An event serves as an initiator that determines when a set of code should be executed. There is no long term influence that needs to be maintained as the detection of the 117 event is sufficient to trigger the execution of the body. Thus, statements nested in impulsive monitors are detached and run independently of their nesting scope predicated only by the guarding event. With continuous, interval-based monitors such as aslongas and whenever, the influence on the environment is intended to be more long term. This is illustrated by the manner in which we stream directives to simulation objects (e.g. continually sending lane change messages until a specific curvilinear offset has been attained). In this regard, our decision to link the execution of internal statements to the activation of the nesting monitor make sense. However, unlike our impulsive statement nesting, there is potential for confusion and unintended proliferation of monitors within the nesting semantics of our continuous monitors. Consider the following example in which the intention is to monitor a vehicle, labeled v1, and whenever it is in the center lane of a road, we want to direct the vehicles in front of it to change lanes to the right whenever( v1.lane id() == center lane id ) ¡ // Determine the set of vehicles in front of v1 leaders = edf.queryAllLeadersInLane( v1.lane id(), v1.distance(), v1.ori() ); forall x in leaders ¡ ¢ ¢ send x change lane( right ); This example works fine and continuously evaluates the vehicles in front of v1 and instructs them to change lanes. In essence, this example is no different than the following code segment 118 whenever( v1.lane id() == center lane id ) ¡ ¢ ¡ ¢ // Assumption: we know all the vehicles in front of v1 send l1 change lane( right ); send l2 change lane( right ); ... send ln change lane( right ); which assumes that we know all the vehicles in front of v1. My point in illustrating this is to show that the forall statement, when placed by itself or with other activity statements, inside the whenever does not introduce any confusing circumstances. The forall simply affords a compact means to send a stream of messages to a set of objects. However, once we nest another monitor statement into this example, the situation becomes tricky to understand. For instance, whenever( v1.lane id() == center lane id ) // Determine the set of vehicles in front of v1 leaders = edf.queryAllLeadersInLane( v1.lane id(), v1.distance(), v1.ori() ); forall x in leaders £ ¡ ¢ ¡ ¢ whenever ( math.abs( v1.distance() - x.distance() ) send x change lane( right ); threshold In this modification to the previous example, we continuously create a whenever monitor to wait for the particular lead vehicle to be within a set distance to v1. The trouble is that we may create many more monitors than originally intended because the forall statement when mixed with the code to refresh the set of leaders, instances a new child whenever statement 119 for each leader. The end result is that a potentially large number of whenever statements may be created to tell the same lead vehicle to change lanes, which is computationally inefficient and unnecessary. These issues force the programmer to be more conscious of programming design decisions when creating SDL code. A potentially more serious problem occurs when multiple monitor statements are interleaved with activity statements inside a continuous monitor. For instance, consider the following example, aslongas ( condition ) ¡ v = create vehicle( place=loc ); whenever( f (v) ) ¡ ¢ ¡ ¢ ¢ ... send v message(); aslongas( g(v) ) ... Because of our parent/child nesting semantics for continuous monitor statements, it is unclear what will happen with interleaved activity statements. Moreover, it is unclear how or when commands will be executed. Thus, order of execution is not preserved, or determined by order of evaluation. For interest, contrast the above example with the following code where the outer aslongas has been replaced by a when statement 120 when ( event ) ¡ v = create vehicle( place=loc ); whenever( f (v) ) ¡ ¢ ¡ ¢ ¢ ... send v message(); aslongas( g(v) ) ... When modified in this manner, the example becomes simpler to understand. When the event occurs, create a vehicle, produce an independent whenever monitor to watch for some condition of the vehicle, send a message to the vehicle, and finally create an aslongas monitor to prepare the vehicle for some future situation. Nesting SDL statements within continuous monitor constructs is tricky, but is easier to understand if only one level of nesting is allowed. When deeper nesting is required, our stance is to enforce that activity statements can only be used in the leaf monitor statements, where order of execution can be maintained. This decision is a potential limitation, but only if deep nesting of continuous statements proves beneficial and useful. On the other hand, this interpretation provides a middle ground for allowing a form of deep nesting in continuous monitor statements. 5.3.5 Parameterized Scenario Modules At times, it is important to be able to encapsulate well-defined scenarios into parameterized modules that can be reused and combined with other modules. SDL supplies 121 functionality to abstract SDL statements into parameterized scenario modules using the defscenario construct. ¨ « ª © © ¨ ¦ ‰§¥ rr© —¥ §¥ defscenario director name ( , , , ) sdl statements Once a scenario director is defined in SDL, it can be created with the create activity statement. However, whereas simulation objects require named parameter-value pairs, SDL scenario directors use unnamed parameters only. The reason for this is explained in Section 5.2.2.2, but an additional reason is that scenario directors are defined using SDL code and parameter ordering is more easily maintained. SDL scenario directors are purely behavioral objects, and have no material realization. Once they are created, they run as an independent behavior in the simulation. Like other simulation objects, they can be destroyed using the destroy statement. When a SDL scenario director is invoked, its execution semantics are analogous to running another SDL interpreter. To illustrate the use of scenario modularization, consider a scenario director named LightSequencer that controls the timing of a sequence of directable traffic lights. 5.3.5.1 The LightSequencer Scenario The LightSequencer director will direct traffic lights at the intersections crossed by a scenario vehicle to turn red just before the vehicle enters the intersection. To synchronize the light cycle with the motion of a vehicle, our traffic light model responds to a directive to be at a specific point in its cycle at a future target time. The traffic light cycle is represented on a normalized clock. That is, state transition times are expressed as fractions of the total ¢ ¡ 122 defscenario LightSequencer( vehicle ) ¡ // Obtain a reference to the vehicle’s current road. road = vehicle.queryRoad(); // Locate the intersection towards which the vehicle // is approaching on it’s current road. isect = vehicle.queryNextIntersection(); // Determine the traffic light controlling the next // intersection towards which the vehicle is heading. traffic light = isect.queryTrafficControlDevice(); sync point = TRANSITION TO RED; aslongas (vehicle.queryDistanceToNextIntersection() ¤ ¡ ¢ ¡ ¢ 0.0) eta = vehicle.queryDistanceToNextIntersection() / vehicle.speed(); send traffic light sync( sync point, eta ); event( exit, vehicle, isect.geometry() ) triggers EXIT EVENT; when( EXIT EVENT ) create LightSequencer( EXIT EVENT.instigator ); Table 5.6: The LightSequencer scenario creates a succession of LightSequencer scenarios. Every LightSequencer directs the behavior of a single traffic light to synchronize its cycle to the approach of the driver. ¢ 123 Directable Traffic Light Model Initial Cycle tcurrent desired cycle phase tevent Adjusted Cycle tcurrent 10 minutes tevent desired cycle phase Figure 5.8: The basic behavior of the traffic light is to cycle between green, yellow, and red states. The behavior responds to a directive to synchronize the cyclic pattern so that at a time in the future ( ) the light will be at a specific point in the cycle (labeled desired cycle phase). In the example shown the cycle duration is lengthened to insure that the light . will be at the transition from yellow to red at time ¯ ª­®­ $##¬ ¯ ­ ­ ˆª !® ¬ green-yellow-red cycle duration. A light synchronization directive consists of a normalized cycle time and a target event time. For example, the light could be directed to be halfway through its green phase at the time a vehicle is expected to arrive at the intersection, extrapolating from its current position and speed. The traffic light will adjust its cycle duration, uniformly compressing or expanding intervals between the current time and the target time, so that the light will be at the appropriate point in its cycle at the target time. This process maintains the normal progression of states and makes the smallest possible change to the normal cycle. Figure 5.8 illustrates how a traffic light would respond to a directive to be at the transition from yellow to red at a time 10 minutes from the current time [35]. A new instance of the LightSequencer is recursively created for each successive intersection encountered by the vehicle. As the vehicle approaches the intersection, the 124 Light Sequencer sync(sync_point, estimated_arrival_time) (A) sync(sync_point, estimated_arrival_time) Light Sequencer (B) Figure 5.9: When the driver enters the intersection, the current message stream is discontinued, and as the driver leaves the intersection a new agent is created. 125 LightSequencer sends directives that synchronize the traffic light at the upcoming intersection with the arrival of the vehicle. When the vehicle arrives at the intersection, the synchronization stream is terminated. As the vehicle leaves the intersection, a new LightSequencer is created. Figure 5.9 illustrates the motion of the vehicle as it approaches the first intersection (sub-figure A) and just after it has left the first intersection (sub-figure B). It is important to note that the driving decisions of the vehicle are unconstrained. The recursively created light sequencer adapts to the turn decisions of the vehicle. The SDL code for the LightSequencer scenario is listed in Table 5.6. 126 CHAPTER 6 CONCLUSION In this thesis, I have introduced the Environment Description Framework (EDF) and the Scenario Description Language (SDL) as parts of an integrated system to address intertwined problems of modeling behaviors and scenarios in structured, urban environments. To summarize, the EDF defines a road and intersection based representation that supports programming of autonomous agents in real-time virtual environments. As its core function, EDF gives structure and meaning to physically-based components while overlaying logical and spatial relationship information on the geometric structure of the environment. Our environment model is distinct in that it embeds behavioral constraints and inter-object relationships into the environment structure. SDL is used for interactive modeling and prototyping of scenario directors in realtime virtual environments. SDL is an interpreted scripting language that translates scenario directives into state machines which are then inserted into the simulation execution framework. SDL advances standard scripting languages by clearly separating statements that describe what happens from statements that determine when things happen. SDL is tightly bound to simulation activity and forms a glue between the EDF, autonomous behaviors, and the scenario directors that influence the environment. This chapter describes the intended use of EDF and SDL, and what limitations may 127 stem from using these frameworks. I also discuss where this research should go next. 6.1 Environment Models and Representations The EDF has been implemented for use in our Hank simulation software. A file format that describes the structures and constructs of the environment framework has also been developed to allow off-line creation of environment descriptions. The syntax for this file format, presented in EBNF1 is provided in Appendix A. A sample environment file describing one of our databases is also included in Appendix B to supplement the EBNF. The file format attempts to facilitate environment modeling efforts by adapting object oriented abstractions where useful. Our use of junctures in the EDF highlights our interest in facilitating environment modeling. Geo-typical virtual environments tend to be very symmetric and regular; intersections and roads throughout the database are reused to save time and effort. The file format created for use with the EDF provides templating constructs and single object inheritance semantics. For instance, a templated road, or intersection, can be defined and later instantiated at different locations and orientations in the environment. Additionally, commonly used roads, or intersections, can serve as foundation for new road or intersection definitions. In our inheritance model, newly specified componentry completely overrides the inherited componentry. For instance, consider creating a new road based on a road model that specifies both a lane definition profile and a segment structure. If we specify only a lane definition in our new road, it will completely override the inher1 Extended Backus-Naur Form 128 ited road’s lane definition, but keep the segment structure. Likewise, if we specify only a segment structure in our new road, it will keep the lane definition but override the inherited road’s segment structure. The main goal of the EDF representation is to model road-based environments in a manner that facilitates behavior and scenario programming of urban entities and situations. However, EDF does not provide representation for off-road terrain. In other words, if autonomous characters drive off the road, there is no information that they can query to determine elevation, surface normal, or direction to nearest road or intersection. Future research in environment representations for virtual environments will either need to provide a global representation for all surfaces, or merge two distinct terrain representations. One of EDF’s potential limitations is it’s focus on road-based, vehicular environments, where a vehicle is defined to be anything that moves along a path. The usefulness of representing roads and intersections in explicit detail is of benefit for environments where these structures can be applied to behavior programs. EDF does not provide a general framework for representing everything. Little emphasis has been placed on providing queries for interaction between objects that exist on structures other than roads. For instance, consider the physical actions required to have an autonomous pedestrian pick up a glass located on a table. First, the pedestrian must find the table. Once it locates the table, it needs to find out where the glass is positioned on the table. These actions alone are enough to illustrate the limitations of the EDF. The EDF provides road-based interactivity, but does not provide a coherent and concise structure for general inter-object 129 relationships, such as the relationship between the glass and the table. Thalmann et al. provide such affordances for behaviors by instilling desired behaviors, or means of interaction, into the objects themselves [24]. For instance, in the case of the glass on the table, the glass could have contained rules, or behavioral actions that a pedestrian wanting to interact with the glass could use to make interaction with external objects more simple. The notion of environmental affordance in programming behaviors for virtual environments is intriguing because it illustrates the balance that exists between behavior and environment. At one extreme, all behavior can be encoded into environmental structures (e.g. benches contain code that instruct pedestrians to sit down, door handles contain behavior code that tell pedestrians where to grab the handle). In this framework, the pedestrian behavior program, for instance, becomes an object behavior processor executing the various behavioral codes attached to objects in the environment. In essence, behavior programs become sets of code to manipulate the resources of objects that will use the behavior. At the other extreme, the environment provides almost no behavioral content other than geometry and Cartesian coordinates of the objects in the environment. In this context, behavior programs are difficult to code and rely on programmer defined interpretations of the world. I believe research into environment representations that merge the two extremes will be the most beneficial as those representations will allow programmers to have flexibility and simplicity in programming behavioral components. Additionally, from our current experience, it is not clear if our environment repre- 130 sentation scales well for extremely large, city size environments. For instance, the number of corridors in moderately complex intersections can be around 40 corridors. If there are hundreds, or even thousands, of intersections in an environment, the accumulation of data structures starts to add up. At present, the EDF provides no means for subdividing the environment into subgroups that may facilitate more efficient query structures for large environments. 6.2 Scenario Scripting with an Interpreted Language The goal of SDL is to tie together our notions of environment, behavior, and scenario affording us the ability to develop scenarios interactively. SDL achieves these goals by simplifying the process of creating HCSM scenario directors to monitor events and conditions in the environment. Language constructs separate the commands that cause changes to the environment from those that respond to changes in the simulation environment. SDL has been implemented for use with Hank and the EDF components. The language syntax, also expressed in terms of EBNF, is provided in Appendix C. By far, the most problematic aspect with SDL is how it deals with nesting. It is unclear what the optimal nesting semantics for continuous, interval-based monitor statements should be. Without significant experience programming scenarios with SDL, it is difficult to grasp the effect that different nesting structures may have on scenario creation, let alone how much monitor statement nesting may actually be utilized in practice. Our solution stands as one way in which nested monitor statements can be handled. It uses the natural meaning of the impulsive and interval based language constructs to intuitively give 131 meaning to nesting. Future investigation into different nesting strategies and relationships is important. 6.3 Summary This thesis presents an integrated approach for support of behavior and scenario modeling in real-time, simulated, urban environments. My research addresses two essential components required to create engaging environments: (1) Representations of the physical environment adapted to the needs of autonomous behavior programming in urban settings, and (2) An interpreted scripting language for online programming of scenario control processes. My research illustrates the tight coupling between environment representation, behavior modeling, and scenario programming as integrated components of a virtual environment simulation. These components work together to provide a complete solution for creating and orchestrating the objects in a virtual environment. 132 APPENDIX A EDF FILE FORMAT A.1 edf-start edf-stmt ::= ::= EDF EBNF Specification ° (group-def )+ units-decl (edf-stmt)+ ° ¢ group-def road-def intersection-def ° ° ° ° connection-def segment-def instance-stmt instance-stmt template-decl group-def group-stmts units-decl transform-decl translation rotation object-decl object-assgn object-expr object-param-list object-param ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= instance (identifier =)? identifier transform-decl (template)? template-decl group identifier (edf-stmt)+ units identifier (translation)? (rotation)? trans [ floating-pt-literal floating-pt-literal floating-pt-literal ] rot [ floating-pt-literal floating-pt-literal floating-pt-literal ] object-decl (object-assgn)+ identifier = object-expr ; create identifier ( object-param-list ) object-param , object-param-list object-param identifier = param ° ¢ ¡ ¡ units-decl group-stmts 133 param intersection-def ::= ::= system defined parameters such as float, int, string template-decl intersection identifier ( : identifier )? isect-components ¢ ¡ isect-components ::= ipoly ( internal-isect )? (corridor)* (traffic-control-info)? ipoly ipoly-type igeometry virtualedge edge-info ::= ::= ::= ::= :: geom ipoly-type igeometry virtualedge (edge-info)+ ¢ ° ¡ null junctor-info floating-pt-literal floating-pt-literal floating-pt-literal junctor-info junctor-info junctor-definition ::= ::= ( (junctor-definition)* ) ° identifier [ floating-pt-literal , identifier ] internal-isect corridor ::= ::= internal intersections (instance-stmt)+ corridor identifier = junctor-identifier to junctor-identifier ° ¢ ¡ (corridor-data)* (segment-set)? junctor-identifier junctor-ident-full corridor-data ::= ::= ::= identifier junctor-ident-full ( identifier , identifier ) ° corridor-type corridor-flow dependency-info ° ° ¢ ° general-param-list ::= param , general-param-list param ¡ 134 corridor-stopline corridor-curb corridor-type dependency-info dependency corridor-flow corridor-stopline corridor-curb ::= ::= ::= ::= ::= ::= type identifier (bidirectional)? dependencies (dependency)+ identifier ( general-param-list ) flowpercentage ( junctor-identifier , floating-pt-literal ) stopline ( junctor-identifier , floating-pt-literal ) curb ( junctor-identifier , floating-pt-literal ) traffic-control (identifier)? object-decl (tc-entry)+ ¢ ¡ ¢ ¡ ° traffic-control-info ::= tc-entry tc-corridor-set tc-state-map road-def ::= ::= ::= ::= tc-corridor-set : tc-state-map identifier , tc-corridor-set tc-corridor-set ( identifier , identifier ) template-decl road identifier (: identifier)? (road-components)* (segment-set)? ° ° ¢ ¡ ° road-components segment-set lane-def lane ::= ::= ::= ::= lane-def attribute-set feature-set (segment)+ lanedef ° (lane)+ axis lane integer-literal identifier floating-pt-literal floating-pt-literal identifier attribute-set attribute ::= ::= range-attributes (attribute)+ ¢ ¡ identifier (range-data)? (attribute-data)? (inline-object-decl)? ; ¢ ¡ 135 range-data distance-range offset-range ::= ::= ::= [ distance-range , offset-range ] floating-pt-literal : floating-pt-literal ° floating-pt-literal : floating-pt-literal identifier : identifier attribute-data feature-set feature ::= ::= ::= ( general-param-list ) features (feature)+ ¢ ¢ ° ° ¡ identifier doc-tuple (feature-data)? (inline-object-decl)? ; doc-tuple feature-data ::= ::= [ floating-pt-literal , floating-pt-literal , identifier ] ( general-param-list ) behavior = object-expr segment segment identifier segment-type-data ¢ ¡ inline-object-decl ::= segment-def segment ::= ::= (orientation identifier)? segment-type-data ::= connection-def connection connection-data ::= ::= ::= system defined, such as straight, curve, or spiral connection connections ¡ (connection-data)* road ( identifier , connection-end ) to intersection ( identifier , identifier ) ° connection-end identifier :: ::= begin end (’a’..’z’ ’ ’) (’a’..’z’ ’ ’ ’0’..’9’)* ° 136 APPENDIX B EDF EXAMPLE - BIKETOWN B.1 Biketown Example E1 P1 m1 m9 P2 R1 P3 P4 m8 P16 PInt1 m23 PInt2 m10 P5 m2 m5 m3 m4 m6 m7 m11 P15 m12 P6 E4 R4 m24 m16 R2 E2 P14 m18 m13 P7 m14 m19 P13 PInt4 m22 m20 m21 PInt3 m15 m17 P8 P12 P11 R3 E3 28m P10 P9 Figure B.1: Illustration of the main Biketown intersection showing junctures, corridors, and internal pedestrian intersections. 137 The Biketown environment represents a simple rural, Iowa town with a long main street composed of the roads (South 000 Block through South 1500 Block). Crossing roads occur regularly over the course of the main north-south streets. Comments in the EDF are single line comments that begin with the characters . Figure B.1 shows a detail of the Biketown’s primary intersection BikeTown Intersection. This intersection is defined once and instanced as needed to create each individual intersection at different locations throughout the database. // BikeTown Database // * Terrain measures 2550 meters by 600 meters. // * Roads include one main 2-lane road extending the entire length of the // * terrain. Starting 300 meters from the south end, intersections occur // * every 150 meters. All roads are straight 2-lane roads with sidewalks. // Database is expressed in terms of meters. units meters template intersection ”Pedestrian Intersection” ¡ // In geom section, [float, identifier] marks fixed junctures // with a parameterized distance between v i and v i+1 of the // polygon with a symbolic token. geom ¡ 138 -0.75 0.75 0.0 ([0.5, P1]) 0.75 0.75 0.0 ([0.5, P2]) 0.75 -0.75 0.0 ([0.5, P3]) -0.75 -0.75 0.0 ([0.5, P4]) ¢ corridor m1 = P1 to P2 type sidewalk bidirectional flowpercentage (P1, 0.3333) flowpercentage (P2, 0.3333) dependencies ¡ merges at ( P1, m2, m3 ) merges at ( P2, m4, m5 ) ¢ corridor m2 = P1 to P3 type sidewalk bidirectional flowpercentage (P1, 0.3333) flowpercentage (P3, 0.3333) dependencies ¡ merges at ( P1, m1, m3 ) merges at ( P3, m5, m6 ) ¡ ¡ ¢ 139 crosses ( m4 ) ¢ corridor m3 = P1 to P4 type sidewalk bidirectional flowpercentage (P1, 0.3333) flowpercentage (P4, 0.3333) dependencies ¡ merges at ( P1, m1, m2 ) merges at ( P4, m4, m6 ) ¢ corridor m4 = P2 to P4 type sidewalk bidirectional flowpercentage (P2, 0.3333) flowpercentage (P4, 0.3333) dependencies ¡ merges at ( P2, m1, m5 ) merges at ( P4, m3, m6 ) crosses ( m2 ) ¢ ¡ ¡ ¢ ¢ 140 ¢ corridor m5 = P2 to P3 type sidewalk bidirectional flowpercentage (P2, 0.3333) flowpercentage (P3, 0.3333) dependencies ¡ merges at ( P2, m1, m4 ) merges at ( P3, m2, m6 ) ¢ corridor m6 = P3 to P4 type sidewalk bidirectional flowpercentage (P3, 0.3333) flowpercentage (P4, 0.3333) dependencies ¡ merges at ( P3, m2, m5 ) merges at ( P4, m3, m4 ) ¢ // the pedestrian intersection is an ”uncontrolled” intersection so // it does not have a traffic control section ¡ ¡ ¢ ¢ 141 ¢ template intersection ”BikeTown Intersection” ¡ // geometry is centered about the origin to make translation and // rotation easy – these intersections are 28m x 28m areas geom ¡ -14.0 14.0 0.0 (P1 P2 [0.5, R1] P3 P4) 14.0 14.0 0.0 (P5 P6 [0.5, R2] P7 P8) 14.0 -14.0 0.0 (P9 P10 [0.5, R3] P11 P12) -14.0 -14.0 0.0 (P13 P14 [0.5, R4] P15 P16) ¢ internal intersections instance PInt1 = ”Pedestrian Intersection” trans [-10.75 10.75 0.0] instance PInt2 = ”Pedestrian Intersection” trans [10.75 10.75 0.0] instance PInt3 = ”Pedestrian Intersection” trans [10.75 -10.75 0.0] instance PInt4 = ”Pedestrian Intersection” trans [-10.75 -10.75 0.0] ¢ corridor m1 = P1 to (PInt1, P1) type sidewalk bidirectional ¢ corridor m2 = P2 to P15 ¡ ¡ ¡ 142 type vehicle // vehicle lane with flow in one direction stopline (P2, 2.5) // stopline occurs at 2.5 meters into the corridor from P2 flowpercentage (P2, 0.50) // 50% of vehicles should turn here dependencies ¡ ¡ ¡ ¡ merges at ( P15, m11, m18 ) crosswalk ( m9, m24 ) ¢ corridor m3 = P2 to P11 type vehicle corridor m4 = P2 to P7 type vehicle stopline (P2, 2.5) // stopline occurs at 2.5 meters into the corridor ¢ ¢ stopline (P2, 2.5) // stopline occurs at 2.5 meters into the corridor flowpercentage (P2, 0.25) // 25% of vehicles should take this route dependencies crosswalk ( m9, m21 ) crosses ( m11, m18, m5, m13 ) merges at ( P11, m19, m12 ) ¢ 143 flowpercentage (P2, 0.25) // 25% of vehicles should take this route dependencies ¡ ¡ ¡ ¡ ¡ crosswalk ( m9, m16 ) crosses ( m11, m5, m12, m6 ) merges at ( P7, m13, m14 ) ¢ corridor m5 = P14 to P3 type vehicle stopline (P14, 2.5) dependencies corridor m6 = P10 to P3 type vehicle stopline (P10, 2.5) dependencies ¢ ¢ crosswalk ( m24, m9 ) crosses ( m3, m18, m4, m11 ) merges at ( P3, m6, m7 ) ¢ crosswalk ( m21, m9 ) 144 crosses ( m13, m4, m12, m11 ) merges at ( P3, m5, m7 ) ¢ corridor m7 = P6 to P3 type vehicle stopline (P6, 2.5) dependencies ¡ crosswalk ( m16, m9 ) merges at ( P3, m5, m6 ) ¢ corridor m8 = P4 to (PInt2,P1) type sidewalk bidirectional ¢ corridor m9 = (PInt1,P2) to (PInt2,P4) type sidewalk bidirectional curb ((PInt1, P2), 2.5) curb ((PInt2, P4), 2.5) dependencies ¡ crosses ( m2, m3, m4, m5, m6, m7 ) ¡ ¡ ¡ ¢ ¢ 145 ¢ corridor m10 = (PInt2,P2) to P5 type sidewalk bidirectional ¢ corridor m11 = P6 to P15 type vehicle stopline (P6, 2.5) dependencies ¡ crosswalk ( m16, m24 ) merges at ( P15, m2, m18 ) crosses ( m6, m5, m4, m3 ) ¢ corridor m12 = P6 to P11 type vehicle stopline (P6, 2.5) dependencies ¡ crosswalk ( m16, m21 ) merges at ( P11, m3, m19 ) crosses ( m6, m4, m18, m13 ) ¡ ¡ ¡ ¢ ¢ 146 ¢ corridor m13 = P14 to P7 type vehicle stopline (P14, 2.5) dependencies ¡ crosswalk ( m24, m16 ) merges at ( P7, m4, m14 ) crosses ( m3, m12, m18, m6 ) ¢ corridor m14 = P10 to P7 type vehicle stopline (P10, 2.5) dependencies ¡ crosswalk ( m21, m16 ) merges at ( P7, m13, m4 ) ¢ corridor m15 = P8 to (PInt3,P2) type sidewalk bidirectional ¡ ¡ ¡ ¢ ¢ ¢ 147 ¢ corridor m16 = (PInt2,P3) to (PInt3,P1) type sidewalk bidirectional curb ((PInt2, P3), 2.5) curb ((PInt3, P1), 2.5) dependencies ¡ crosses ( m7, m11, m12, m4, m13, m14 ) ¢ corridor m17 = P9 to (PInt3,P3) type sidewalk bidirectional ¢ corridor m18 = P10 to P15 type vehicle stopline (P10, 2.5) dependencies ¡ crosswalk ( m21, m24 ) merges at ( P15, m11, m2 ) crosses ( m13, m12, m5, m3 ) ¢ ¡ ¡ ¡ ¢ ¢ 148 ¡ ¡ ¡ ¡ corridor m19 = P14 to P11 type vehicle stopline (P14, 2.5) dependencies ¡ ¡ crosswalk ( m24, m21 ) merges at ( P11, m3, m12 ) ¢ corridor m20 = P12 to (PInt4,P3) type sidewalk bidirectional ¢ corridor m21 = (PInt3,P4) to (PInt4,P2) type sidewalk bidirectional curb ((PInt3, P4), 2.5) curb ((PInt4, P2), 2.5) dependencies corridor m22 = P13 to (PInt4,P4) type sidewalk bidirectional ¢ ¢ crosses ( m6, m14, m18, m3, m12, m19 ) ¢ 149 ¢ corridor m23 = P16 to (PInt1,P4) type sidewalk bidirectional ¢ corridor m24 = (PInt4,P1) to (PInt1,P3) type sidewalk bidirectional curb ((PInt4, P1), 2.5) curb ((PInt1, P3), 2.5) dependencies ¡ crosses ( m19, m13, m5, m18, m11, m2 ) ¢ traffic control object decl light = create traffic light( p1=23.2, p2=291.0, p3=32, p4=Ident4 ); ¢ // For the sake of completeness, m1, m8, m10, m15, m17, m20, m22, // and m23 are all uncontrolled corridors and do not need to be // explicitly matched to a particular traffic control state // variable in this section. // these corridors are the main vehicular corridors ¡ ¡ ¡ ¡ ¢ 150 m2, m3, m4, m6, m14, m18: (light, ns state) m7, m11, m12, m5, m13, m19: (light, ew state) // these are all controlled by the walk/don’t walk signals of // of the traffic control devices. note that groups (m9, m21) and // (m16, m24) are each regulated by the same traffic light state m9, m21: (light, ew walk state) m16, m24: (light, ns walk state) ¢ // // INTERSECTIONS // instance ”South and 1st” = ”BikeTown Intersection” trans [-1050.0 0.0 -0.3] instance ”South and 2nd” = ”BikeTown Intersection” trans [-900.0 0.0 -0.3] instance ”South and 3rd” = ”BikeTown Intersection” trans [-750.0 0.0 -0.3] instance ”South and 4th” = ”BikeTown Intersection” trans [-600.0 0.0 -0.3] instance ”South and 5th” = ”BikeTown Intersection” trans [-450.0 0.0 -0.3] instance ”South and 6th” = ”BikeTown Intersection” trans [-300.0 0.0 -0.3] instance ”South and 7th” = ”BikeTown Intersection” trans [-150.0 0.0 -0.3] instance ”South and 8th” = ”BikeTown Intersection” trans [0.0 0.0 -0.3] instance ”South and 9th” = ”BikeTown Intersection” trans [150.0 0.0 -0.3] ¢ 151 instance ”South and 10th” = ”BikeTown Intersection” trans [300.0 0.0 -0.3] instance ”South and 11th” = ”BikeTown Intersection” trans [450.0 0.0 -0.3] instance ”South and 12th” = ”BikeTown Intersection” trans [600.0 0.0 -0.3] instance ”South and 13th” = ”BikeTown Intersection” trans [750.0 0.0 -0.3] instance ”South and 14th” = ”BikeTown Intersection” trans [900.0 0.0 -0.3] instance ”South and 15th” = ”BikeTown Intersection” trans [1050.0 0.0 -0.3] // All but one North/South roads follows this pattern, so make a template // and instantiate as needed. This road is 122m long and is centered at // the origin. template road ”BikeTown NS Road” ¡ // ”lane” TYPE WIDTH REL HEIGHT RECOMMENDED DIR OF FLOW // the ”axis” keyword denotes the location of the road axis in // relation to the lanes lanedef ¡ lane 1 grass 2.5 0.333 both lane 2 sidewalk 1.5 0.333 both lane 3 parkway 4.0 0.333 both lane 4 vehicle 6.0 0.0 neg axis lane 5 vehicle 6.0 0.0 pos 152 lane 6 parkway 4.0 0.333 both lane 7 sidewalk 1.5 0.333 both lane 8 grass 2.5 0.333 both ¢ // this section specifies range, or zonal, attributes range attributes ¡ speedlimit (35, mph); ¢ // If we needed any ”features” on this road, we could specify them // in the feature section. // We can imagine creating and placing behavioral objects such as: // * intelligent parking meters // * automatic door openers // * pedestrians serving as crossing guards, construction zone workers, or // flagpersons // * moving walkways, escalators, elevators // * gates // * toll booths // * etc... features ¡ // given an exact location relative to the road coordinates and features may, 153 // or may not contain a behavior, or data list feature1 [13.4, 10.2, pos]; feature3 [120.4, 10.0, none] behavior=create flagperson( p1=23.2, p2=32 ); feature2 [10.2, -2.34, both] (data1, data2, data3); feature4 [1.0, 1.23, neg] (data1, data2, data3) behavior=create flagperson( height=2.2, place=302 ); ¢ segment straight start 0.00 0.00 -0.30 end 122.00 0.00 -0.30 ¢ // The South 000 Block is almost identical to all the other roads, // except it is slightly longer, so we can inherit the properties // from the base North/South roads and override the segment // list. road South 000 Block ¡ lanedef lane 1 grass 2.5 0.333 both lane 2 sidewalk 1.5 0.333 both ¡ ¡ ¢ 154 lane 3 parkway 4.0 0.333 both lane 4 vehicle 6.0 0.0 neg axis lane 5 vehicle 6.0 0.0 pos lane 6 parkway 4.0 0.333 both lane 7 sidewalk 1.5 0.333 both lane 8 grass 2.5 0.333 both ¢ // this section specifies range, or zonal, attributes range attributes ¡ speedlimit (35, mph); ¢ features // given an exact location relative to the road coordinates and features may, // or may not contain a behavior, or data list feature1 [13.4, 10.2, pos]; feature3 [192.4, 10.0, none] behavior=create flagperson( p1=23.2, p2=32 ); feature2 [10.2, -2.34, both] (data1, data2, data3); feature4 [1.0, 1.23, neg] (data1, data2, data3) behavior=create flagperson( height=2.2, place=302 ); ¢ ¡ 155 ¡ segment straight start -1350.00 0.00 -0.30 end -1064.00 0.00 -0.30 ¢ instance South 100 Block = ”BikeTown NS Road” trans [ -1036.0 0.0 0.0 ] instance South 200 Block = ”BikeTown NS Road” trans [ -886.0 0.0 0.0 ] instance South 300 Block = ”BikeTown NS Road” trans [ -736.0 0.0 0.0 ] instance South 400 Block = ”BikeTown NS Road” trans [ -586.0 0.0 0.0 ] instance South 500 Block = ”BikeTown NS Road” trans [ -436.0 0.0 0.0 ] instance South 600 Block = ”BikeTown NS Road” trans [ -286.0 0.0 0.0 ] instance South 700 Block = ”BikeTown NS Road” trans [ -136.0 0.0 0.0 ] instance South 800 Block = ”BikeTown NS Road” trans [ 14.0 0.0 0.0 ] instance South 900 Block = ”BikeTown NS Road” trans [ 164.0 0.0 0.0 ] instance South 1000 Block = ”BikeTown NS Road” trans [ 314.0 0.0 0.0 ] instance South 1100 Block = ”BikeTown NS Road” trans [ 464.0 0.0 0.0 ] instance South 1200 Block = ”BikeTown NS Road” trans [ 614.0 0.0 0.0 ] instance South 1300 Block = ”BikeTown NS Road” trans [ 764.0 0.0 0.0 ] instance South 1400 Block = ”BikeTown NS Road” trans [ 914.0 0.0 0.0 ] instance South 1500 Block = ”BikeTown NS Road” trans [ 1064.0 0.0 0.0 ] // —————- Begin East Roads —————– ¢ 156 // All of the East/West roads follows this pattern, so make a template // and instantiate as needed. This road is 286m long and is centered at // the origin. template road ”BikeTown EW Road” ¡ // ”lane” TYPE WIDTH REL HEIGHT RECOMMENDED DIR OF FLOW // the ”axis” keyword denotes the location of the road axis in // relation to the lanes lanedef ¡ ¡ lane 1 grass 2.5 0.333 both lane 2 sidewalk 1.5 0.333 both lane 3 parkway 4.0 0.333 both lane 4 vehicle 6.0 0.0 neg axis lane 5 vehicle 6.0 0.0 pos lane 6 parkway 4.0 0.333 both lane 7 sidewalk 1.5 0.333 both lane 8 grass 2.5 0.333 both ¢ range attributes speedlimit (35, mph); 157 electronic sign [20.0:35.0, -2.3:3.0] (data, list, if, any) behavior=create trafficsign( sync=23.2, p2=291.0, p3=32 ); ¢ segment straight start 0.00 0.00 -0.30 end 0.00 286.00 -0.30 ¢ instance East 1st = ”BikeTown EW Road” trans [ -1050.0 -300.0 0.0 ] instance East 2nd = ”BikeTown EW Road” trans [ -900.0 -300.0 0.0 ] instance East 3rd = ”BikeTown EW Road” trans [ -750.0 -300.0 0.0 ] instance East 4th = ”BikeTown EW Road” trans [ -600.0 -300.0 0.0 ] instance East 5th = ”BikeTown EW Road” trans [ -450.0 -300.0 0.0 ] instance East 6th = ”BikeTown EW Road” trans [ -300.0 -300.0 0.0 ] instance East 7th = ”BikeTown EW Road” trans [ -150.0 -300.0 0.0 ] instance East 8th = ”BikeTown EW Road” trans [ 0.0 -300.0 0.0 ] instance East 9th = ”BikeTown EW Road” trans [ 150.0 -300.0 0.0 ] instance East 10th = ”BikeTown EW Road” trans [ 300.0 -300.0 0.0 ] instance East 11th = ”BikeTown EW Road” trans [ 450.0 -300.0 0.0 ] instance East 12th = ”BikeTown EW Road” trans [ 600.0 -300.0 0.0 ] instance East 13th = ”BikeTown EW Road” trans [ 750.0 -300.0 0.0 ] ¡ ¢ 158 instance East 14th = ”BikeTown EW Road” trans [ 900.0 -300.0 0.0 ] instance East 15th = ”BikeTown EW Road” trans [ 1050.0 -300.0 0.0 ] // ————- Begin West Roads ————— instance West 1st = ”BikeTown EW Road” trans [ -1050.0 14.0 0.0 ] instance West 2nd = ”BikeTown EW Road” trans [ -900.0 14.0 0.0 ] instance West 3rd = ”BikeTown EW Road” trans [ -750.0 14.0 0.0 ] instance West 4th = ”BikeTown EW Road” trans [ -600.0 14.0 0.0 ] instance West 5th = ”BikeTown EW Road” trans [ -450.0 14.0 0.0 ] instance West 6th = ”BikeTown EW Road” trans [ -300.0 14.0 0.0 ] instance West 7th = ”BikeTown EW Road” trans [ -150.0 14.0 0.0 ] instance West 8th = ”BikeTown EW Road” trans [ 0.0 14.0 0.0 ] instance West 9th = ”BikeTown EW Road” trans [ 150.0 14.0 0.0 ] instance West 10th = ”BikeTown EW Road” trans [ 300.0 14.0 0.0 ] instance West 11th = ”BikeTown EW Road” trans [ 450.0 14.0 0.0 ] instance West 12th = ”BikeTown EW Road” trans [ 600.0 14.0 0.0 ] instance West 13th = ”BikeTown EW Road” trans [ 750.0 14.0 0.0 ] instance West 14th = ”BikeTown EW Road” trans [ 900.0 14.0 0.0 ] instance West 15th = ”BikeTown EW Road” trans [ 1050.0 14.0 0.0 ] // // CONNECTION SECTION // 159 ¡ connections road (South 000 Block, end) to intersection (”South and 1st”, R4) road (South 100 Block, begin) to intersection (”South and 1st”, R2) road (East 1st, end) to intersection (”South and 1st”, R3) road (West 1st, begin) to intersection (”South and 1st”, R1) road (South 100 Block, end) to intersection (”South and 2nd”, R4) road (South 200 Block, begin) to intersection (”South and 2nd”, R2) road (East 2nd, end) to intersection (”South and 2nd”, R3) road (West 2nd, begin) to intersection (”South and 2nd”, R1) road (South 200 Block, end) to intersection (”South and 3rd”, R4) road (South 300 Block, begin) to intersection (”South and 3rd”, R2) road (East 3rd, end) to intersection (”South and 3rd”, R3) road (West 3rd, begin) to intersection (”South and 3rd”, R1) road (South 300 Block, end) to intersection (”South and 4th”, R4) road (South 400 Block, begin) to intersection (”South and 4th”, R2) road (East 4th, end) to intersection (”South and 4th”, R3) road (West 4th, begin) to intersection (”South and 4th”, R1) 160 road (South 400 Block, end) to intersection (”South and 5th”, R4) road (South 500 Block, begin) to intersection (”South and 5th”, R2) road (East 5th, end) to intersection (”South and 5th”, R3) road (West 5th, begin) to intersection (”South and 5th”, R1) road (South 500 Block, end) to intersection (”South and 6th”, R4) road (South 600 Block, begin) to intersection (”South and 6th”, R2) road (East 6th, end) to intersection (”South and 6th”, R3) road (West 6th, begin) to intersection (”South and 6th”, R1) road (South 600 Block, end) to intersection (”South and 7th”, R4) road (South 700 Block, begin) to intersection (”South and 7th”, R2) road (East 7th, end) to intersection (”South and 7th”, R3) road (West 7th, begin) to intersection (”South and 7th”, R1) road (South 700 Block, end) to intersection (”South and 8th”, R4) road (South 800 Block, begin) to intersection (”South and 8th”, R2) road (East 8th, end) to intersection (”South and 8th”, R3) road (West 8th, begin) to intersection (”South and 8th”, R1) road (South 800 Block, end) to intersection (”South and 9th”, R4) 161 road (South 900 Block, begin) to intersection (”South and 9th”, R2) road (East 9th, end) to intersection (”South and 9th”, R3) road (West 9th, begin) to intersection (”South and 9th”, R1) road (South 900 Block, end) to intersection (”South and 10th”, R4) road (South 1000 Block, begin) to intersection (”South and 10th”, R2) road (East 10th, end) to intersection (”South and 10th”, R3) road (West 10th, begin) to intersection (”South and 10th”, R1) road (South 1000 Block, end) to intersection (”South and 11th”, R4) road (South 1100 Block, begin) to intersection (”South and 11th”, R2) road (East 11th, end) to intersection (”South and 11th”, R3) road (West 11th, begin) to intersection (”South and 11th”, R1) road (South 1100 Block, end) to intersection (”South and 12th”, R4) road (South 1200 Block, begin) to intersection (”South and 12th”, R2) road (East 12th, end) to intersection (”South and 12th”, R3) road (West 12th, begin) to intersection (”South and 12th”, R1) road (South 1200 Block, end) to intersection (”South and 13th”, R4) road (South 1300 Block, begin) to intersection (”South and 13th”, R2) 162 road (East 13th, end) to intersection (”South and 13th”, R3) road (West 13th, begin) to intersection (”South and 13th”, R1) road (South 1300 Block, end) to intersection (”South and 14th”, R4) road (South 1400 Block, begin) to intersection (”South and 14th”, R2) road (East 14th, end) to intersection (”South and 14th”, R3) road (West 14th, begin) to intersection (”South and 14th”, R1) road (South 1400 Block, end) to intersection (”South and 15th”, R4) road (South 1500 Block, begin) to intersection (”South and 15th”, R2) road (East 15th, end) to intersection (”South and 15th”, R3) road (West 15th, begin) to intersection (”South and 15th”, R1) ¢ 163 APPENDIX C SCENARIO DESCRIPTION LANGUAGE C.1 SDL EBNF Specification ° ° sdl-start function-def scenario-def ::= ::= ::= (sdl-statement function-def scenario-def )* defun identifier ( arg-list ) ( sdl-statement )+ ¡ ¢ ¢ ° ¡ defscenario identifier ( arg-list ) ( sdl-statement )* ¢ ¡ ( sdl-statement )+ compound-statement:= : sdl-statement ::= if ( expression ) sdl-statement ( else sdl-statement )? for ( expression-list ; expression ; expression-list ) ° sdl-statement ° expression ; ° activity-exit ; ° activity-send ; ° activity-destroy ; ° activity-create ; ° sensor ; ° monitor-when monitor-every ° ° monitor-aslongas monitor-whenever ° ° compound-statement 164 return ( expression )? ; monitor-aslongas ::= monitor-whenever ::= monitor-when monitor-every sensor activity-create activity-destroy activity-send activity-exit expression expression-list assign-expr assign-op logic-or-expr logic-and-expr equal-expr equal-op compare-expr compare-op ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= aslongas ( expression ) ( sdl-statement )+ ¡ ¢ ¡ whenever ( expression ) when ( expression ) every ( expression ) ¡ ( sdl-statement )+ ¢ ( sdl-statement )+ ¡ ( sdl-statement )+ event ( arg-list ) triggers identifier create identifier ( ( object-param-list )? ) destroy identifier send expression identifier ( arg-list ) exit assign-expr expression ( , expression )* logic-or-expr ( assign-op assign-expr )? = += -= *= /= ° ±° ° ° ° ° logic-and-expr ( logic-and-expr )* equal-expr ( && equal-expr )* compare-expr ( equal-op compare-expr )* != == addition-expr ( compare-op addition-expr )* £ £ ° ¤ ¤ ³Q²³)° ¤ Q° ° = = ¢ ° forall identifier in identifier sdl-statement ¢ £ ° ¤ ° )²Q²£ 165 addition-expr addition-op mult-expr mult-op unary-expr postfix-expr misc-expr expr-elem ::= ::= ::= ::= ::= ::= ::= ::= mult-expr ( addition-op mult-expr )* + unary-expr ( mult-op unary-expr )* * / % ++ unary-expr – unary-expr - unary-expr + unary-expr postfix-expr expr-elem ( misc-expr )* . identifier [ expression ] ( arg-list ) ° ° ° ° ° ° ° ° ° ° identifier constant activity-create true false null [ arg-list ] ( assign-expr ) ° ° ° ° ° ° ° object-param-list object-param arg-list identifier constant empty-rule ::= ::= ::= ::= ::= ::= object-param , object-param-list object-param identifier = expression expression-list empty-rule (’a’..’z’ ’A’..’Z’ ’ ’) (’a’..’z’ ’A’..’Z’ ’ ’ ’0’..’9’ )* integer character string float ° ° ° ° ° ° ° ° ° 166 REFERENCES [1] O. Ahmad. Scenario control at ids. Unpublished slides and notes from Scenario Control Workshop. Orlando, FL, 1996. [2] O. Alloyer, E. Bonakdarian, J. Cremer, J. Kearney, and P. Willemsen. Embedding scenarios in ambient traffic. In Driving Simulation Conference, September 1997. [3] B.E. Artz. An analytical road segment terrain database for driving simulation. In Driving Simulation Conference, DSC’95, pages 274–284, Sophia Antipolis, France, September 1995. [4] A.C. Bailey, A.H. Jamson, Parkes A.M., and Wright S. Recent and future development of the leeds driving simulator. In Proceedings of DSC’99 (Driving Simulation Conference), July 1999. [5] J. Bates. Dramatic presence. Presence, 2(1):1–15, 1993. [6] S. Bayarri, M. Fernandez, and M. Perez. Virtual reality for driving simulation. Communications of the ACM, 39(5):72–76, May 1996. [7] S. Bayarri, M. Fernandez, M. Perez, and R. Rodriguez. A database model for urban and motorway driving simulation. In Proceedings of DSC’95 (Driving Simulation Conference), September 1995. [8] B. Blumberg. Old Tricks, New Dogs: Ethology and Interactive Creatures. PhD thesis, MIT Media Lab, 1996. [9] B. Blumberg. Go with the flow: synthetic vision for autonomous animated creatures. In Proceedings of the first international conference on Autonomous agents, February 1997. [10] B. Blumberg and T. Galyean. Multi-level direction of autonomous creatures for realtime virtual environments. In SIGGRAPH 95, 1995. [11] E. Bonakdarian, J. Cremer, and J. Kearney. Generating and managing ambient traf- 167 fic for real-time driving simulation. In Proceedings of DSC’99 (Driving Simulation Conference), July 1999. [12] E. Bonakdarian, J. Cremer, J.K. Kearney, and P. Willemsen. Generation of ambient traffic for real-time driving simulation. In Image Conference, pages 123–133, August 1998. [13] M. Booth, J. Cremer, and J. Kearney. Scenario control for real-time driving simulation. In Proceedings of Fourth Eurographics Animation and Simulation Workshop, pages 103–120, 1993. [14] R.A. Brooks. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, RA-2(1):14–23, March 1986. [15] O. Carles and S. Espi´ . Database generation system for road applications. In Proe ceedings of DSC’99 (Driving Simulation Conference), July 1999. [16] M.J. Conway. Alice: Easy-to-Learn 3D Scripting for Novices. PhD thesis, School of Engineering and Applied Science, University of Virginia, December 1997. [17] J. Cremer, J. Kearney, and Y. Papelis. Hcsm: A framework for behavior and scenario control in virtual environments. ACM Transactions of Modeling and Computer Simulation, 5(3):242–267, July 1995. [18] J. Cremer, J.K. Kearney, and P. Willemsen. Directable behavior models for virtual driving scenarios. Transactions of the Society for Computer Simulation, 14(2), 1997. special issue on Multiagent Systems. [19] T. Dingus, J. Cremer, J. Kearney, and R. Romano. Development of prototype simulation scenarios for the Iowa Driving Simulator. Technical report, National Highway Traffic Safety Administration Contract No. DTNH22-93-YU-07237, 1995. Final Report. [20] S. Donikian. Vuems: A virtual urban environment modeling system. In Computer Graphics International ’97, June 1997. [21] S. Donikian and A. Chauffaut. A general animation and simulation platform. In D. Terzopoulos and D. Thalmann, editors, Computer Animation and Simulation, pages 197–209, 1995. [22] S. Donikian, F. Devillers, and G. Moreau. The kernel of a scenario language for animation and simulation. In Eurographics Workshop on Animation and Simulation, September 1999. 168 [23] D.F. Evans. Ground vehicle database modeling. In Real Time Systems ’94, Paris, France, 1994. [24] N. Farenc, S. Raupp Musse, E. Schweiss, M. Kallmann, O. Aune, R. Boulic, and D. Thalmann. One step towards virtual human management for urban environment simulation. In Proceedings of the ECAI Workshop on Intelligent User Interfaces, Brighton, UK, 1998. [25] Reymond G., Munier O., and Kemeny A. A road description using reference points. In Proceedings of the first seminar on traffic generation, pages 15–21, 1994. [26] T. Galyean. Narrative guidance. In AAAI Spring Symposium on Interactive Story Systems: Plot and Character, pages 52–55, March 1995. [27] T. Galyean. Narrative Guidance of Interactivity. PhD thesis, Media Lab, Massachusetts Institute of Technology, 1995. [28] J.J. Gibson. The Senses Considered As Perceptual Systems. Houghton Mifflin Company, 1966. [29] A. G¨ ll¨ , A. Deshpande, P. Hingorani, and P. Varaiya. Smartdb: An object oriented o o simulation framework for intelligent vehicles and highway systems. In Fifth Annual Conference on AI, Simulation, and Planning in High Autonomy Systems, pages 244– 250. IEEE Computer Society Press, December 1994. [30] J.M Hankey, D.V. McGehee, T.A. Dingus, E.N. Mazzae, and E.N. Garrott. Initial driver avoidance behavior and reaction time to an unalerted intersection incursion. In 40th Annual Meeting of the Human Factors and Ergonomics Society, 1996. [31] S.A. Hansen. Conceptual Control Programming for Physical System Simulation. PhD thesis, Department of Computer Science, University of Iowa, May 1993. [32] D. Harel. Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8:231–274, 1987. [33] David Harel, Hagi Lachover, Amnon Naamad, Amir Pnueli, Michal Politi, Rivi Sherman, Aharon Shtull-Trauring, and Mark Trakhtenbrot. Statemate: A working environment for the development of complex reactive systems. IEEE Transactions on Software Engineering, 16(4), April 1990. [34] B. Hayes-Roth and R. van Gent. Story-making with improvisational puppets. In Proceedings of the First International Conference on Autonomous Agents, pages 1–7, February 1997. 169 [35] J.K. Kearney, P. Willemsen, S. Donikian, and F. Devillers. Scenario languages for driving simulation. In Proceedings of DSC’99 (Driving Simulation Conference), pages 123–133, July 1999. [36] J. Lee, D.V. McGehee, T.A. Dingus, and T. Wilson. Collision avoidance behavior of unalerted drivers using a front-to-rear-end collision warning display on the Iowa Driving Simulator. Technical report, Transportation Research Record, National Research Council, 1997. [37] W. Leutzbach. Introduction to the Theory of Traffic Flow. Springer-Verlag, Berlin, Germany, 1988. [38] A. Brian Loyall and J. Bates. Hap: A reactive, adaptive architecture for agents. Technical report, School of Computer Science, Carnegie Mellon University, June 1991. [39] A. Brian Loyall and J. Bates. Real-time control of animated broad agents. In Fifteenth Annual Conference of the Cognitive Science Society, June 1993. [40] M. Lutz. Programming Python, Object-Oriented Scripting. O’Reilly & Associates, Inc., October 1996. [41] P. Maes. The agent network architecture (ana). SIGART Bulletin, 2(4), 1991. [42] P. Maes. Modeling adaptive autonomous agents. Artificial Life Journal, 1:135–162, 1994. [43] A. Mathur. Task scheduling and temporal consistency in a real-time virtual driving environment. Master’s thesis, Department of Computer Science, University of Iowa, 1998. [44] A.D. May. Traffic Flow Fundamentals. Prentice Hall, Englewood Cliffs, New Jersey, 1990. [45] D.V. McGehee, J.M. Hankey, T.A. Dingus, E.N. Mazzae, W.R. Garrott, A. Grant, and S.R. Reinach. Examination of drivers’ collision avoidance behavior using conventional non-anti-lock brakes. Technical report, National Highway Traffic Safety Administration, 1996. Contract No. NRD-20-95-08086. [46] G. Moreau and S. Donikian. From psychological and real-time interaction requirements to behavioural simulation. In Eurographics Workshop on Computer Animation and Simulation, Lisbonne, Portugal, September 1998. 170 [47] J.K. Ousterhout. Tcl and the Tk Toolkit. Addison-Wesley Professional Computing, May 1994. [48] J.K. Ousterhout. Scripting: Higher level programming for the 21st century. IEEE Computer, March 1998. [49] Y.E. Papelis, , and S. Bahauddin. Logical modeling of roadway environment to support real-time simulation of autonomous traffic. In SIVE95: The First Workshop on Simulation and Interaction in Virtual Environments, pages 62–71, Iowa City, Iowa, USA, July 1995. Department of Computer Science, University of Iowa. [50] Paramics. Paramics: Microscopic traffic simulation. World Wide Web. URL: http://www.paramics-online.com. [51] R. Pausch, J. Snoddy, R. Taylor, S. Watson, and E. Haseltine. Disney’s aladdin: First steps toward storytelling in virtual reality. In SIGGRAPH 96, pages 193–203, August 1996. [52] K. Perlin and A. Goldberg. Improv: A system for scripting interactive actors in virtual worlds. In Computer Graphics, pages 205–216, 1996. [53] C.S. Pinhanez, K. Mase, and A. Bobick. Interval scripts: a design paradigm for storybased interactive systems. Technical Report 391, M.I.T. Media Laboratory Perceptual Computing Section, October 1996. [54] A.K. Rathi and A. Santiago. Urban network traffic simulation: Trafnetsim program. Journal of Transportation Engineering, 116:734–743, November 1990. [55] D. Reece. Selective Perception for Robot Driving. PhD thesis, Carnegie Mellon University, May 1992. [56] E.S. Reed. James J. Gibson and the Psychology of Perception. Yale University Press, 1988. [57] W. Scott Neal Reilly. Believable Social and Emotional Agents. PhD thesis, School of Computer Science, Carnegie Mellon University, May 1996. [58] C.W. Reynolds. Flocks, herds, and schools: A distributed behavioral model. In Computer Graphics, volume 21, pages 25–34, July 1987. [59] SGI. IRIS Performer Programmers Manual. 171 [60] R. Sukthankar. Situation Awareness for Tactical Driving. PhD thesis, The Robotics Institue, Carnegie Mellon University, January 1997. [61] Sun Microsystems Computer Company. Java 3D Overview, July 1997. [62] T. Sweeney. Unrealscript language reference. URL:http://www.unrealed.net/timnotes.phtml. World Wide Web. [63] G. Thomas. Environnements virtuels urbains: mod´ lisation des informations e n´ cessaires a la simulation de pi´ tons. PhD thesis, Universit´ de Rennes I, equipe e e e ´ ` SIAMES, 1999. [64] G. Thomas and S. Donikian. Modelling virtual cities dedicated to behavioural animation. In Eurographics 2000, volume 19, 2000. [65] B. Thongphasuk. Human Locomotion for Directable, Interactive, Real-Time Virtual Environments. PhD thesis, University of Iowa, December 1998. [66] W. van Winsum. SSL/NSL Specification Release 1.2. TRC, November 1994. [67] W. Van Winsum. From Adaptive Behaviour to Adaptive Control. PhD thesis, University of Groningen, Netherlands, 1996. [68] P. van Wolffelaar and W. van Winsum. Traffic modeling and driving simulation - an integrated approach. In Driving Simulation Conference (DSC 95), pages 235–244, 1995. [69] VRML Technical Specificatation. [70] L. Wall, T. Christiansen, and R.L. Schwartz. Programming Perl. O’Reilly & Associates, Inc., 2nd edition, 1996. [71] P. Weir. The truman show. Movie, Paramount Pictures, 1998. [72] J. Wernecke. The Inventor Mentor: programming Object-oriented 3D graphics with Open Inventor, release 2. Addison-Wesley, 1994. [73] P. Weyhrauch. Guiding Interactive Drama. PhD thesis, School of Computer Science, Carnegie Mellon University, January 1997. [74] P. Willemsen. A directable vehicle behavior model for driving simulation. Master’s thesis, Department of Computer Science, University of Iowa, May 1996. 172 [75] J. Winsor and B. Freeman. Jumping JavaScript. Sun Microsystems Press, A Prentice Hall Title, 1997.