VIEWS: 14 PAGES: 11 POSTED ON: 6/12/2011
COMP 145 Project: Sami Says Architecture Design Tian Wang, Jim Irving, Hao Wang, Dustin Tsang Tian Wang May 1, 2005 Version 1.1 1 Overview Sami Says is an audio game for visually impaired children. User can record a sound, add sound effects to it, load sound clips from a library, and add sounds together to make a story. All the sounds that the user has created or modified as well as their stories can be saved in the library. The program is based on a menu which can be controlled using UP/DOWN and SPACE/ESC keys. Choices in the menus lead either to other static menus, dynamically generated menus which are created by a Menu Iterator or Dialog Processes that perform manipulations on the sounds that the user is working on. The menu is structured like this: Menu Record New Sound Library Structure Play Current Folder 1 Load Sound Clip Clip1 Folder 2 Add Effect Clip2 … Save to library Add to Story Recording mode Sound Effects Story Mode Effect1 Effect2 Play story … Save story Add Effect Menu iterator Erase Story Menu item Dialog process Figure: Menu Structure of the System Some of the behavior of the menu interface is configurable and can be changed and extended by setting the proper values in the system.xml file in the installation directory. Typically, when the user enters a menu for the first time, an instructional message will be played which describes what can be done in that menu. This message cannot be skipped, but these messages can be turned off by setting the “ReadIntro” value in the “menudef” tag in the system.xml file. After the instructional message has been played, or immediately if the instructions have been turned off, the sound file associated with the current menu item will be played. The sound files associated with a particular menu item are specified in the <audiofile> section of the system.xml file and syntax for specifying audio files can be found in the Detailed Design Document. User uses UP and DOWN to move through the menu, and presses SPACE to select an item from the menu. If the selected item leads to another static menu, the user will be taken to that menu. If the item led to a dynamic menu, the menu will be initialized and the user will be taken there. If the item led to a Dialog Process, that Dialog Process will take control. Dialog Processes are what actually perform actions on the sounds that the user is working on. ESC can be used to go back to the previous menu from a static or dynamic menu but not from a Dialog Process. The dynamic menus created by Menu Iterators and Dialog Processes are complex enough to merit further explanation: 1) Menu Iterator: The subitems of some menus cannot be decided at the design time, they have to be generated at the run time. The menu controller (called AudioMenuWalker in the system) will delegate menu operations (such as getting the next item in the menu or activate the Dialog Process for the current item) to the Menu Iterator to get current menu item information. The boxes outlined in orange in the diagram above shows their uses in the menu structure. For example, the submenu which is accessed when you choose “Load sound clip” from the Sound Mode menu is composed of the contents of the sound library. The items in that menu are loaded when you choose “Load sound clip” by parsing the contents of the cliplib folder in the installation directory. 2) Dialog Processes: Each menu item at the leaf node of the menu tree has a handler which provides the corresponding function of the menu item. Here we these functions Dialog Processes. In each Dialog Process, you can hear some instructions that tell you how to operate the function and you need to press keys in order to interact with the system. For example, in the Recording Dialog Process, you will first hear the instruction “Press space bar to start recording and press space again to stop”. Then you press SPACE and hear a beep indicating that the recording has started. You then speak into the microphone and press SPACE again. You then hear another beep and a message saying “Your sound has been recorded”. Then another sound indicating we are going back to the menu would be played and you would be returned to an appropriate place in the menu. To make this program extensible and easy to modify, we use an XML file system.xml to describe some dynamic parts of the system. It has three parts: 1) Defines sound clips used in the system, provides textual names for each clip that can be used throughout the program. 2) Defines the menu structure (may specify Menu Iterators for parts of the menu) and specify handler classes for menu items. 3) Defines sound effects which can be applied to sounds through the “Add a sound effect” item in the Sound Mode menu. Please refer to Chapter 4 of Detail Design document for more details. 2 Layered View The system architecture of Sami Says can be viewed as a set of modules organized in several layers. The system configuration file interacts with the first layer—XML parser. The configuration file, system.xml, is parsed and used to initialize the menu structure, system sounds, and available sound effects. All this information is fed into two lower layers: Audio Based UI Engine and Manipulators. The user only interacts with a simple window provided by the Audio Based UI Engine, which presents underlying menu (and its embedded Menu Iterators) and Dialog Processes in an audio form. When user selects a certain menu item, Dialog Process objects in the Manipulators layer will be used to provide an interactive Dialog Process corresponding to the menu item, such as recording, saving clips, erasing current story, etc. The Data Representation layer provides underlying functions to organize clip compositions and save sound clips or compositions. Considering the ways different layers affect the system behavior, we can summarize them like this: XML Parser: Parses the description of the user interface and definitions of system data and creates internal data structures for them. Audio Based UI Engine: Interacts with user and provides high level system actions (menu movement and launching Dialog Processes) Manipulators: Defines exact behavior of each Dialog Process and implements Menu Iterators. Data Representation: Provides underlying support for data manipulation, file saving and other utilities. Developers can change or extend the function of Sami Says in the following ways: Update XML description for more complicated menu structure or to use different sound files for menu items (in case you find the default voice to be putting the children to sleep). Add more Dialog Processes to the Manipulators layer for more functionality. Add more Menu Iterators (like IEffects and ILibrary) for dynamically generated contents in the menu. Add more sound effects using command line utilities via the <effects> section of the system.xml file or through a class based approach (no class based sound effects are currently used). In the diagram on the next page we represent the sections of the program in the layered view and their corresponding Python modules with matching colors in order to further clarify their relationship. System Architecture and Python Modules XML description xmlparser main Holds as XML PARSER KeyboardHandler menu Delegates menu operations to System window Information starts Menu Iterators Dialog dp menuit AUDIO BASED UI Processes keyboard ENGINE AddEffect AddToComp Replay IEffects MANIPULATORS Recording SaveComp PlayComp ILibrary DATA REPRESENTATION ClearComp SelectClip SaveClip disk utils audio cliplib disk Figure: System Architecture (left) and corresponding python modules (right) 3 File Structure Here is the file structure of the Sami Says: samisays ├─effects ├─iterators ├─processes ├─cliplib │ ├─my clips │ ├─my stories │ └─(other folders) ├─sound │ ├─effects │ ├─menu │ └─sys ├─temp └─tools All folders shown above are necessary! Deleting any of them may cause system failure. Python code files are in the following folders: samisays\ all fundamental modules (system.xml is also here) samisays\effects\ all class based effects makers classes samisays\iterators\ all iterator classes samisays\processes\ all Dialog Process classes cliplib stores the sound clips available to the system. There are two special folders called My Clips and My Stories: cliplib\my clips: Saves user’s current working clip in the sound mode, this can be user’s recorded sound clips or other modified clips loaded from library. cliplib\my stories: Saves user’s composed stories. The WAV file with name starting with “#” is the recorded title for each story, and the file with the same name but without “#” is the audio of story itself. While browsing this folder using Menu Iterator ILibrary, only files with “#” will be played as the sound for the menu item representing the story, and corresponding file without “#” will be loaded as current working clip when that item is selected. The /sound folder stores necessary system sounds, including sound files for menu items, instructions for Dialog Processes, event indicator sounds, names for each sound effect, etc. All sounds here will be defined in the system.xml and be assigned a textual name to facilitate using it in other parts of system.xml and in the program code. See detail design document Chapter 2.1 for more details. The /temp folder is used to store temporary files created during the execution of the program. The /tools folder is used to store external tools used in the system, currently the only tool here is sox.exe and its companion program soxmix.exe, which is used to apply all of the currently supported sound effects and concatenate sounds when adding clips into the story. 4 Extension Programming Guide Sami Says is designed to be extensible. Current architecture allows future developers to add to or modify the current system. There are two main ways to extend the functionality of this system. 4.1 Adding new menu structure This can involve several different areas of development. To modify the menu structure so as to provide a new function, the programmer should do following things: 1. Decide the new menu structure. It should be described using XML according to the format of system.xml. It can be embedded anywhere inside the original menu structure. Please refer to Detail Design Document Chapter 3 for details of the XML style menu description. 2. Record some sound files to play for the new menu items. Add its definition in the <audiofile> section of system.xml. Typically the line is formatted like this: <file id="YOUR_AUDIO_ID" location="Your_File_Location" /> Your file location should be relative to samisays\sound folder. For example, if you have your sound file at samisays\sound\menu\myitem.wav, you should write “menu\myitem.wav” for the location. 3. Write related Menu Iterator classes (if any). Programmer should create a module in the iterators folder. Inside this module there should be a class inheriting MenuIterator and which has the same name as the containing module. (typically the name of this class should start with I to indicate that it is an iterator). In the current design, the programmer should also add corresponding import statements in instance.py for the iterator class. See Module iterators.menuit in Chapter 2.3 of the Detail Design document for more details. 4. Write the related Dialog Processes. The programmer should create modules in the processes folder. Inside each module there should be a class with the same name as the containing module, which inherits DialogProcess and implement all necessary functions. Newly added Dialog Process classes should also be imported in instance.py. See Module processes.dp in Chapter 2.3 of the Detail Design document for more details. Some Common Questions: How do I end a Dialog Process and return to the menu? When this Dialog Process finishes, it can use self.observer.notify(“END_PROCESS”) or self.observer.notify(“END_PROCESS2”) to notify the window object that is has finished. The programmer can also develop their own window events for specific exit handling by modifying SamiSaysWindow.notify(). How can I access information in other Dialog Processes? You should use the global context to communicate between Dialog Processes. You can set key-value pairs in the context and retrieve them anywhere else in the program, so long as you have stated from samisays.utils import *. Please refer to Chapter 2.1 of the Detail Design Document for more details about global context. 4.2 Adding new sound effects There are two ways to add new sound effects to the system: command line based effects and class based effects. 4.2.1 Command line based effects: You need to find an external tool that can produce your desired sound effect, and it must accept input and output filename as command line parameters. The steps to integrate this into Sami Says are as follows: 1 Record a sound file to play for the item in the “Add a sound effect” menu. Add its definition in the <audiofile> section of system.xml. typically the line looks like this: <file id="MY_ID" location="Your_File_Location" /> You file location should be relative to samisays\sound folder. For example, if you have your sound file at samisays\sound\menu\myitem.wav, you should write “menu\myitem.wav” for the location. 2 Describe the command pattern in system.xml <effects> section. Typically the description looks like this: <effect name="NAME" audio="MY_ID" cmd="sometool %s %s other param" /> Here name is the textual name of this effect, which will be displayed while going through the “Add sound effect” menu. audio is the audio ID of the sound file that will be played this effect in the “Add sound effect” menu. This audio ID should be defined in the <audiofile> section. cmd is the command line pattern used to call the external tool. The two “%s” in this line indicate the filenames of the input and output files. 3 Copy the tool (sometool.exe in this example) and related files to “tools” folder. NOTE: It is simple to implement a new sound effect, but there are some restrictions on the tool parameter order (input filename must be in front of output filename). This can be improved by using more complicated command patterns. To implement this, you will need to change xmlparser.py. 4.2.2 Class based effects Class based effects are implemented by writing a class which manipulates the sound data directly. Here are the steps to add one: 1 Record your effect name and define it in <audiofile> section in system.xml as above. 2 You need to write a class which inherits EffectMaker and put it in a module with the same name in the effects folder. Implement the sound effects by doing sound manipulation in the EffectMaker.applyEffect(clip) function. 3 Add a corresponding import statement in instance.py to make sure its instance can be created in the run time. (See instance.py for examples) 4 Add a line in system.xml <effects> section indicating the adding of the new effect. A typical line looks like the one below. Here name and audio have the same meaning as in Section 4.2.1. handler here indicates the class used for this sound effect. <effect name="NAME" audio="MY_ID" handler="ClassName" /> Now your sound effect should appear in the “Add sound effect” menu. Please refer to Detail Designs document for more details about python modules and other programming utilities. 4.3 Making an installer In order to make an installer, you will need to first be able to run the Sami Says python scripts as python scripts instead of through the .exe’s that are used for distribution. The necessary modules have to be installed in a particular order in order to operate correctly. This is how we suggest going about the installation of all the necessary modules. 1. Copy mfc71.dll to your /system32 or /systemNT directory if it isn’t there already. This file is necessary and often missing from systems. 2. Install Python 2.4. 3. Install the Microsoft SAPI 5.1 redistributable. 4. Install the extra Microsoft voices. 5. Install pySonic 0.8 for Python 2.4. 6. Copy fmod.dll to your /system32 or /systemNT directory (however just putting it in your /Python24 directory seems to work just fine). 7. Install Mark Hammond’s Python win32all extensions. 8. Install pyTTS 3.0. 9. Install wxPython 2.5 for Python 2.4, Unicode version. Now you can copy the Sami Says source files to a folder in your /Python/Lib folder and run them with python. In order to make an installer, there are a few more tasks which must be done. 1. Install py2exe. 2. Install ZODB 3.2 for Python 2.4. 3. Install epydoc 2.1. 4. Install Inno Setup. 5. Run the script setup.py using py2exe from inside the Sami Says directory in order to create a folder with (most) of the files you will need to install. Try running the SamiSays.exe file found in the /dist folder that is generated by py2exe. Due to software conflicts, py2exe sometimes doesn’t quite retrieve all of the necessary libraries correctly and so there are a few hacks that you may need to employ. a. Make absolutely sure that your Python path (as viewed both from your system environment variables and PythonWin) points into /Lib/site- packages/win32com/gen_py. b. In __init__.py, which is found in the Pythong24/Lib/site-packages/pyTTS directory, take the import statements (import sapi, from sapi import …) at the beginning of the file out of their try block. The files are there, but the method by which py2exe imports them will make the try block fail. c. In sapi.py, which is found in the Python24/Lib/site-packages/pyTTS directory, add the line “from win32com import gen_py”. 6. The library.zip file in the /dist directory created by py2exe will not contain some necessary files due to its method of copying. To fix this, you will need to do the following. a. Overwrite the win32com folder inside library.zip with a copy of the /win32com folder located in your /Python24/Lib/site-packages folder. b. Copy all the files from /Python24/Lib/site-packages/pyTTS to the /pyTTS folder inside library.zip without overwriting! Only the files that are not currently present in library.zip/pyTTS should be added. Overwriting any of the files that are currently in library.zip/pyTTS could result in your executable not being functional! 7. Now open the inno.iss file inside the /dist directory created by py2exe. This is the setup script used by Inno Setup to create a single file installer for distribution. From the Build menu, select Compile to create the installer. The installer will be called samisetup.exe (or whatever is specified in the OutputBaseFilename attribute in inno.iss) and will be created in the /Output directory inside the /dist directory.
Pages to are hidden for
"Architecture"Please download to view full document