Architecture by keralaguest


									COMP 145 Project: Sami Says
Architecture Design
                                             Tian Wang, Jim Irving, Hao Wang, Dustin Tsang

                                                                                      Tian Wang
                                                                 May 1, 2005          Version 1.1

1 Overview
    Sami Says is an audio game for visually impaired children. User can record a sound,
add sound effects to it, load sound clips from a library, and add sounds together to make a
story. All the sounds that the user has created or modified as well as their stories can be
saved in the library. The program is based on a menu which can be controlled using
UP/DOWN and SPACE/ESC keys. Choices in the menus lead either to other static menus,
dynamically generated menus which are created by a Menu Iterator or Dialog Processes
that perform manipulations on the sounds that the user is working on. The menu is
structured like this:

Menu                        Record New                                Sound Library
Structure                  Play Current                                 Folder 1
                          Load Sound Clip                                                Clip1
                                                                        Folder 2
                             Add Effect                                                  Clip2
                           Save to library
                            Add to Story

  Recording mode                                                         Sound Effects

    Story Mode                                                              Effect1

                              Play story
                             Save story

                             Add Effect
       Menu iterator
                             Erase Story
       Menu item
       Dialog process

Figure: Menu Structure of the System

      Some of the behavior of the menu interface is configurable and can be changed and
extended by setting the proper values in the system.xml file in the installation directory.
Typically, when the user enters a menu for the first time, an instructional message will be
played which describes what can be done in that menu. This message cannot be skipped,
but these messages can be turned off by setting the “ReadIntro” value in the “menudef”
tag in the system.xml file. After the instructional message has been played, or
immediately if the instructions have been turned off, the sound file associated with the
current menu item will be played. The sound files associated with a particular menu item
are specified in the <audiofile> section of the system.xml file and syntax for specifying
audio files can be found in the Detailed Design Document. User uses UP and DOWN to
move through the menu, and presses SPACE to select an item from the menu. If the
selected item leads to another static menu, the user will be taken to that menu. If the item
led to a dynamic menu, the menu will be initialized and the user will be taken there. If
the item led to a Dialog Process, that Dialog Process will take control. Dialog Processes
are what actually perform actions on the sounds that the user is working on. ESC can be
used to go back to the previous menu from a static or dynamic menu but not from a
Dialog Process.
      The dynamic menus created by Menu Iterators and Dialog Processes are complex
enough to merit further explanation:
      1)       Menu Iterator: The subitems of some menus cannot be decided at the
               design time, they have to be generated at the run time. The menu controller
               (called AudioMenuWalker in the system) will delegate menu operations
               (such as getting the next item in the menu or activate the Dialog Process for
               the current item) to the Menu Iterator to get current menu item information.
               The boxes outlined in orange in the diagram above shows their uses in the
               menu structure. For example, the submenu which is accessed when you
               choose “Load sound clip” from the Sound Mode menu is composed of the
               contents of the sound library. The items in that menu are loaded when you
               choose “Load sound clip” by parsing the contents of the cliplib folder in the
               installation directory.
      2)       Dialog Processes: Each menu item at the leaf node of the menu tree has a
               handler which provides the corresponding function of the menu item. Here
               we these functions Dialog Processes. In each Dialog Process, you can hear
               some instructions that tell you how to operate the function and you need to
               press keys in order to interact with the system.
                    For example, in the Recording Dialog Process, you will first hear the
               instruction “Press space bar to start recording and press space again to
               stop”. Then you press SPACE and hear a beep indicating that the recording
               has started. You then speak into the microphone and press SPACE again.
               You then hear another beep and a message saying “Your sound has been
               recorded”. Then another sound indicating we are going back to the menu
               would be played and you would be returned to an appropriate place in the
    To make this program extensible and easy to modify, we use an XML file system.xml
to describe some dynamic parts of the system. It has three parts:
   1)     Defines sound clips used in the system, provides textual names for each clip
          that can be used throughout the program.
   2)     Defines the menu structure (may specify Menu Iterators for parts of the menu)
          and specify handler classes for menu items.
   3)     Defines sound effects which can be applied to sounds through the “Add a
          sound effect” item in the Sound Mode menu. Please refer to Chapter 4 of
          Detail Design document for more details.
2 Layered View
     The system architecture of Sami Says can be viewed as a set of modules organized in
several layers. The system configuration file interacts with the first layer—XML parser.
The configuration file, system.xml, is parsed and used to initialize the menu structure,
system sounds, and available sound effects. All this information is fed into two lower
layers: Audio Based UI Engine and Manipulators. The user only interacts with a simple
window provided by the Audio Based UI Engine, which presents underlying menu (and
its embedded Menu Iterators) and Dialog Processes in an audio form. When user selects a
certain menu item, Dialog Process objects in the Manipulators layer will be used to
provide an interactive Dialog Process corresponding to the menu item, such as recording,
saving clips, erasing current story, etc. The Data Representation layer provides
underlying functions to organize clip compositions and save sound clips or compositions.

    Considering the ways different layers affect the system behavior, we can summarize
them like this:
    XML Parser: Parses the description of the user interface and definitions of system
data and creates internal data structures for them.
    Audio Based UI Engine: Interacts with user and provides high level system actions
(menu movement and launching Dialog Processes)
    Manipulators: Defines exact behavior of each Dialog Process and implements Menu
    Data Representation: Provides underlying support for data manipulation, file saving
and other utilities.

   Developers can change or extend the function of Sami Says in the following ways:
       Update XML description for more complicated menu structure or to use
          different sound files for menu items (in case you find the default voice to be
          putting the children to sleep).
       Add more Dialog Processes to the Manipulators layer for more functionality.
       Add more Menu Iterators (like IEffects and ILibrary) for dynamically
          generated contents in the menu.
       Add more sound effects using command line utilities via the <effects>
          section of the system.xml file or through a class based approach (no class
          based sound effects are currently used).

    In the diagram on the next page we represent the sections of the program in the
layered view and their corresponding Python modules with matching colors in order to
further clarify their relationship.
             System Architecture and Python Modules

                                                                                     xmlparser                 main
                                                   Holds as
                  XML PARSER                    KeyboardHandler
                                                                                     Delegates menu
                                                                                      operations to
                      System                window
                    Information                                             starts                               Menu Iterators

                                           Dialog                      dp                             menuit
                 AUDIO BASED UI          Processes
keyboard            ENGINE

                                          AddEffect     AddToComp       Replay                        IEffects
                                         Recording    SaveComp         PlayComp                       ILibrary

                 REPRESENTATION          ClearComp       SelectClip     SaveClip

                                                           utils            audio          cliplib


           Figure: System Architecture (left) and corresponding python modules (right)
3 File Structure
   Here is the file structure of the Sami Says:

          │ ├─my clips
          │ ├─my stories
          │ └─(other folders)
          │ ├─effects
          │ ├─menu
          │ └─sys

    All folders shown above are necessary! Deleting any of them may cause system

   Python code files are in the following folders:
        samisays\                    all fundamental modules (system.xml is also here)
        samisays\effects\            all class based effects makers classes
        samisays\iterators\          all iterator classes
        samisays\processes\          all Dialog Process classes

   cliplib stores the sound clips available to the system. There are two special folders
   called My Clips and My Stories:
         cliplib\my clips: Saves user’s current working clip in the sound mode, this
             can be user’s recorded sound clips or other modified clips loaded from
         cliplib\my stories: Saves user’s composed stories. The WAV file with name
             starting with “#” is the recorded title for each story, and the file with the
             same name but without “#” is the audio of story itself. While browsing this
             folder using Menu Iterator ILibrary, only files with “#” will be played as the
             sound for the menu item representing the story, and corresponding file
             without “#” will be loaded as current working clip when that item is selected.

   The /sound folder stores necessary system sounds, including sound files for menu
   items, instructions for Dialog Processes, event indicator sounds, names for each
   sound effect, etc. All sounds here will be defined in the system.xml and be assigned a
   textual name to facilitate using it in other parts of system.xml and in the program code.
   See detail design document Chapter 2.1 for more details.
The /temp folder is used to store temporary files created during the execution of the

The /tools folder is used to store external tools used in the system, currently the only
tool here is sox.exe and its companion program soxmix.exe, which is used to apply all
of the currently supported sound effects and concatenate sounds when adding clips
into the story.
4 Extension Programming Guide
    Sami Says is designed to be extensible. Current architecture allows future developers
to add to or modify the current system. There are two main ways to extend the
functionality of this system.

4.1     Adding new menu structure
This can involve several different areas of development. To modify the menu structure so
as to provide a new function, the programmer should do following things:
1. Decide the new menu structure. It should be described using XML according to the
    format of system.xml. It can be embedded anywhere inside the original menu structure.
    Please refer to Detail Design Document Chapter 3 for details of the XML style menu
2. Record some sound files to play for the new menu items. Add its definition in the
    <audiofile> section of system.xml. Typically the line is formatted like this:
       <file id="YOUR_AUDIO_ID"       location="Your_File_Location" />
   Your file location should be relative to samisays\sound folder. For example, if you
   have your sound file at samisays\sound\menu\myitem.wav, you should write
   “menu\myitem.wav” for the location.
3. Write related Menu Iterator classes (if any). Programmer should create a module in
   the iterators folder. Inside this module there should be a class inheriting
   MenuIterator and which has the same name as the containing module. (typically the
   name of this class should start with I to indicate that it is an iterator). In the current
   design, the programmer should also add corresponding import statements in for the iterator class. See Module iterators.menuit in Chapter 2.3 of the
   Detail Design document for more details.
4. Write the related Dialog Processes. The programmer should create modules in the
   processes folder. Inside each module there should be a class with the same name as
   the containing module, which inherits DialogProcess and implement all necessary
   functions. Newly added Dialog Process classes should also be imported in
   See Module processes.dp in Chapter 2.3 of the Detail Design document for more

Some Common Questions:
    How do I end a Dialog Process and return to the menu?
    When this Dialog Process finishes, it can use“END_PROCESS”)
or“END_PROCESS2”) to notify the window object that is has
finished. The programmer can also develop their own window events for specific exit
handling by modifying SamiSaysWindow.notify().
    How can I access information in other Dialog Processes?
    You should use the global context to communicate between Dialog Processes. You
can set key-value pairs in the context and retrieve them anywhere else in the program, so
long as you have stated from samisays.utils import *. Please refer to Chapter 2.1 of the
Detail Design Document for more details about global context.
4.2     Adding new sound effects
    There are two ways to add new sound effects to the system: command line based
effects and class based effects.

4.2.1 Command line based effects: You need to find an external tool that can produce
your desired sound effect, and it must accept input and output filename as command line
parameters. The steps to integrate this into Sami Says are as follows:

   1   Record a sound file to play for the item in the “Add a sound effect” menu. Add its
       definition in the <audiofile> section of system.xml. typically the line looks like
       <file id="MY_ID"      location="Your_File_Location" />
       You file location should be relative to samisays\sound folder. For example, if you
       have your sound file at samisays\sound\menu\myitem.wav, you should write
       “menu\myitem.wav” for the location.
   2   Describe the command pattern in system.xml <effects> section. Typically the
       description looks like this:
       <effect name="NAME" audio="MY_ID" cmd="sometool %s %s other param" />
       Here name is the textual name of this effect, which will be displayed while going
       through the “Add sound effect” menu. audio is the audio ID of the sound file that
       will be played this effect in the “Add sound effect” menu. This audio ID should
       be defined in the <audiofile> section. cmd is the command line pattern used to
       call the external tool. The two “%s” in this line indicate the filenames of the input
       and output files.
   3   Copy the tool (sometool.exe in this example) and related files to “tools” folder.

       NOTE: It is simple to implement a new sound effect, but there are some
       restrictions on the tool parameter order (input filename must be in front of output
       filename). This can be improved by using more complicated command patterns.
       To implement this, you will need to change

4.2.2 Class based effects
    Class based effects are implemented by writing a class which manipulates the sound
data directly. Here are the steps to add one:
    1 Record your effect name and define it in <audiofile> section in system.xml as
    2 You need to write a class which inherits EffectMaker and put it in a module with
         the same name in the effects folder. Implement the sound effects by doing sound
         manipulation in the EffectMaker.applyEffect(clip) function.
    3 Add a corresponding import statement in to make sure its instance
         can be created in the run time. (See for examples)
    4 Add a line in system.xml <effects> section indicating the adding of the new effect.
         A typical line looks like the one below. Here name and audio have the same
         meaning as in Section 4.2.1. handler here indicates the class used for this sound
       <effect name="NAME"        audio="MY_ID"        handler="ClassName" />
   Now your sound effect should appear in the “Add sound effect” menu.

Please refer to Detail Designs document for more details about python modules and other
programming utilities.

4.3 Making an installer
        In order to make an installer, you will need to first be able to run the Sami Says
python scripts as python scripts instead of through the .exe’s that are used for distribution.
The necessary modules have to be installed in a particular order in order to operate
correctly. This is how we suggest going about the installation of all the necessary

   1.      Copy mfc71.dll to your /system32 or /systemNT directory if it isn’t there
           already. This file is necessary and often missing from systems.
   2.      Install Python 2.4.
   3.      Install the Microsoft SAPI 5.1 redistributable.
   4.      Install the extra Microsoft voices.
   5.      Install pySonic 0.8 for Python 2.4.
   6.      Copy fmod.dll to your /system32 or /systemNT directory (however just
           putting it in your /Python24 directory seems to work just fine).
   7.      Install Mark Hammond’s Python win32all extensions.
   8.      Install pyTTS 3.0.
   9.      Install wxPython 2.5 for Python 2.4, Unicode version.

Now you can copy the Sami Says source files to a folder in your /Python/Lib folder and
run them with python. In order to make an installer, there are a few more tasks which
must be done.

   1.      Install py2exe.
   2.      Install ZODB 3.2 for Python 2.4.
   3.      Install epydoc 2.1.
   4.      Install Inno Setup.
   5.      Run the script using py2exe from inside the Sami Says directory in
           order to create a folder with (most) of the files you will need to install. Try
           running the SamiSays.exe file found in the /dist folder that is generated by
           py2exe. Due to software conflicts, py2exe sometimes doesn’t quite retrieve
           all of the necessary libraries correctly and so there are a few hacks that you
           may need to employ.
           a. Make absolutely sure that your Python path (as viewed both from your
                system environment variables and PythonWin) points into /Lib/site-
           b. In, which is found in the Pythong24/Lib/site-packages/pyTTS
                directory, take the import statements (import sapi, from sapi import …) at
                the beginning of the file out of their try block. The files are there, but the
                method by which py2exe imports them will make the try block fail.
     c. In, which is found in the Python24/Lib/site-packages/pyTTS
        directory, add the line “from win32com import gen_py”.

6.   The file in the /dist directory created by py2exe will not contain
     some necessary files due to its method of copying. To fix this, you will need
     to do the following.
     a. Overwrite the win32com folder inside with a copy of the
         /win32com folder located in your /Python24/Lib/site-packages folder.
     b. Copy all the files from /Python24/Lib/site-packages/pyTTS to the /pyTTS
         folder inside without overwriting! Only the files that are not
         currently present in should be added. Overwriting
         any of the files that are currently in could result in
         your executable not being functional!
7.   Now open the inno.iss file inside the /dist directory created by py2exe. This is
     the setup script used by Inno Setup to create a single file installer for
     distribution. From the Build menu, select Compile to create the installer. The
     installer will be called samisetup.exe (or whatever is specified in the
     OutputBaseFilename attribute in inno.iss) and will be created in the /Output
     directory inside the /dist directory.

To top