COMP 145 Project: Sami Says
Tian Wang, Jim Irving, Hao Wang, Dustin Tsang
May 1, 2005 Version 1.1
Sami Says is an audio game for visually impaired children. User can record a sound,
add sound effects to it, load sound clips from a library, and add sounds together to make a
story. All the sounds that the user has created or modified as well as their stories can be
saved in the library. The program is based on a menu which can be controlled using
UP/DOWN and SPACE/ESC keys. Choices in the menus lead either to other static menus,
dynamically generated menus which are created by a Menu Iterator or Dialog Processes
that perform manipulations on the sounds that the user is working on. The menu is
structured like this:
Menu Record New Sound Library
Structure Play Current Folder 1
Load Sound Clip Clip1
Add Effect Clip2
Save to library
Add to Story
Recording mode Sound Effects
Story Mode Effect1
Figure: Menu Structure of the System
Some of the behavior of the menu interface is configurable and can be changed and
extended by setting the proper values in the system.xml file in the installation directory.
Typically, when the user enters a menu for the first time, an instructional message will be
played which describes what can be done in that menu. This message cannot be skipped,
but these messages can be turned off by setting the “ReadIntro” value in the “menudef”
tag in the system.xml file. After the instructional message has been played, or
immediately if the instructions have been turned off, the sound file associated with the
current menu item will be played. The sound files associated with a particular menu item
are specified in the <audiofile> section of the system.xml file and syntax for specifying
audio files can be found in the Detailed Design Document. User uses UP and DOWN to
move through the menu, and presses SPACE to select an item from the menu. If the
selected item leads to another static menu, the user will be taken to that menu. If the item
led to a dynamic menu, the menu will be initialized and the user will be taken there. If
the item led to a Dialog Process, that Dialog Process will take control. Dialog Processes
are what actually perform actions on the sounds that the user is working on. ESC can be
used to go back to the previous menu from a static or dynamic menu but not from a
The dynamic menus created by Menu Iterators and Dialog Processes are complex
enough to merit further explanation:
1) Menu Iterator: The subitems of some menus cannot be decided at the
design time, they have to be generated at the run time. The menu controller
(called AudioMenuWalker in the system) will delegate menu operations
(such as getting the next item in the menu or activate the Dialog Process for
the current item) to the Menu Iterator to get current menu item information.
The boxes outlined in orange in the diagram above shows their uses in the
menu structure. For example, the submenu which is accessed when you
choose “Load sound clip” from the Sound Mode menu is composed of the
contents of the sound library. The items in that menu are loaded when you
choose “Load sound clip” by parsing the contents of the cliplib folder in the
2) Dialog Processes: Each menu item at the leaf node of the menu tree has a
handler which provides the corresponding function of the menu item. Here
we these functions Dialog Processes. In each Dialog Process, you can hear
some instructions that tell you how to operate the function and you need to
press keys in order to interact with the system.
For example, in the Recording Dialog Process, you will first hear the
instruction “Press space bar to start recording and press space again to
stop”. Then you press SPACE and hear a beep indicating that the recording
has started. You then speak into the microphone and press SPACE again.
You then hear another beep and a message saying “Your sound has been
recorded”. Then another sound indicating we are going back to the menu
would be played and you would be returned to an appropriate place in the
To make this program extensible and easy to modify, we use an XML file system.xml
to describe some dynamic parts of the system. It has three parts:
1) Defines sound clips used in the system, provides textual names for each clip
that can be used throughout the program.
2) Defines the menu structure (may specify Menu Iterators for parts of the menu)
and specify handler classes for menu items.
3) Defines sound effects which can be applied to sounds through the “Add a
sound effect” item in the Sound Mode menu. Please refer to Chapter 4 of
Detail Design document for more details.
2 Layered View
The system architecture of Sami Says can be viewed as a set of modules organized in
several layers. The system configuration file interacts with the first layer—XML parser.
The configuration file, system.xml, is parsed and used to initialize the menu structure,
system sounds, and available sound effects. All this information is fed into two lower
layers: Audio Based UI Engine and Manipulators. The user only interacts with a simple
window provided by the Audio Based UI Engine, which presents underlying menu (and
its embedded Menu Iterators) and Dialog Processes in an audio form. When user selects a
certain menu item, Dialog Process objects in the Manipulators layer will be used to
provide an interactive Dialog Process corresponding to the menu item, such as recording,
saving clips, erasing current story, etc. The Data Representation layer provides
underlying functions to organize clip compositions and save sound clips or compositions.
Considering the ways different layers affect the system behavior, we can summarize
them like this:
XML Parser: Parses the description of the user interface and definitions of system
data and creates internal data structures for them.
Audio Based UI Engine: Interacts with user and provides high level system actions
(menu movement and launching Dialog Processes)
Manipulators: Defines exact behavior of each Dialog Process and implements Menu
Data Representation: Provides underlying support for data manipulation, file saving
and other utilities.
Developers can change or extend the function of Sami Says in the following ways:
Update XML description for more complicated menu structure or to use
different sound files for menu items (in case you find the default voice to be
putting the children to sleep).
Add more Dialog Processes to the Manipulators layer for more functionality.
Add more Menu Iterators (like IEffects and ILibrary) for dynamically
generated contents in the menu.
Add more sound effects using command line utilities via the <effects>
section of the system.xml file or through a class based approach (no class
based sound effects are currently used).
In the diagram on the next page we represent the sections of the program in the
layered view and their corresponding Python modules with matching colors in order to
further clarify their relationship.
System Architecture and Python Modules
XML PARSER KeyboardHandler
Information starts Menu Iterators
Dialog dp menuit
AUDIO BASED UI Processes
AddEffect AddToComp Replay IEffects
Recording SaveComp PlayComp ILibrary
REPRESENTATION ClearComp SelectClip SaveClip
utils audio cliplib
Figure: System Architecture (left) and corresponding python modules (right)
3 File Structure
Here is the file structure of the Sami Says:
│ ├─my clips
│ ├─my stories
│ └─(other folders)
All folders shown above are necessary! Deleting any of them may cause system
Python code files are in the following folders:
samisays\ all fundamental modules (system.xml is also here)
samisays\effects\ all class based effects makers classes
samisays\iterators\ all iterator classes
samisays\processes\ all Dialog Process classes
cliplib stores the sound clips available to the system. There are two special folders
called My Clips and My Stories:
cliplib\my clips: Saves user’s current working clip in the sound mode, this
can be user’s recorded sound clips or other modified clips loaded from
cliplib\my stories: Saves user’s composed stories. The WAV file with name
starting with “#” is the recorded title for each story, and the file with the
same name but without “#” is the audio of story itself. While browsing this
folder using Menu Iterator ILibrary, only files with “#” will be played as the
sound for the menu item representing the story, and corresponding file
without “#” will be loaded as current working clip when that item is selected.
The /sound folder stores necessary system sounds, including sound files for menu
items, instructions for Dialog Processes, event indicator sounds, names for each
sound effect, etc. All sounds here will be defined in the system.xml and be assigned a
textual name to facilitate using it in other parts of system.xml and in the program code.
See detail design document Chapter 2.1 for more details.
The /temp folder is used to store temporary files created during the execution of the
The /tools folder is used to store external tools used in the system, currently the only
tool here is sox.exe and its companion program soxmix.exe, which is used to apply all
of the currently supported sound effects and concatenate sounds when adding clips
into the story.
4 Extension Programming Guide
Sami Says is designed to be extensible. Current architecture allows future developers
to add to or modify the current system. There are two main ways to extend the
functionality of this system.
4.1 Adding new menu structure
This can involve several different areas of development. To modify the menu structure so
as to provide a new function, the programmer should do following things:
1. Decide the new menu structure. It should be described using XML according to the
format of system.xml. It can be embedded anywhere inside the original menu structure.
Please refer to Detail Design Document Chapter 3 for details of the XML style menu
2. Record some sound files to play for the new menu items. Add its definition in the
<audiofile> section of system.xml. Typically the line is formatted like this:
<file id="YOUR_AUDIO_ID" location="Your_File_Location" />
Your file location should be relative to samisays\sound folder. For example, if you
have your sound file at samisays\sound\menu\myitem.wav, you should write
“menu\myitem.wav” for the location.
3. Write related Menu Iterator classes (if any). Programmer should create a module in
the iterators folder. Inside this module there should be a class inheriting
MenuIterator and which has the same name as the containing module. (typically the
name of this class should start with I to indicate that it is an iterator). In the current
design, the programmer should also add corresponding import statements in
instance.py for the iterator class. See Module iterators.menuit in Chapter 2.3 of the
Detail Design document for more details.
4. Write the related Dialog Processes. The programmer should create modules in the
processes folder. Inside each module there should be a class with the same name as
the containing module, which inherits DialogProcess and implement all necessary
functions. Newly added Dialog Process classes should also be imported in instance.py.
See Module processes.dp in Chapter 2.3 of the Detail Design document for more
Some Common Questions:
How do I end a Dialog Process and return to the menu?
When this Dialog Process finishes, it can use self.observer.notify(“END_PROCESS”)
or self.observer.notify(“END_PROCESS2”) to notify the window object that is has
finished. The programmer can also develop their own window events for specific exit
handling by modifying SamiSaysWindow.notify().
How can I access information in other Dialog Processes?
You should use the global context to communicate between Dialog Processes. You
can set key-value pairs in the context and retrieve them anywhere else in the program, so
long as you have stated from samisays.utils import *. Please refer to Chapter 2.1 of the
Detail Design Document for more details about global context.
4.2 Adding new sound effects
There are two ways to add new sound effects to the system: command line based
effects and class based effects.
4.2.1 Command line based effects: You need to find an external tool that can produce
your desired sound effect, and it must accept input and output filename as command line
parameters. The steps to integrate this into Sami Says are as follows:
1 Record a sound file to play for the item in the “Add a sound effect” menu. Add its
definition in the <audiofile> section of system.xml. typically the line looks like
<file id="MY_ID" location="Your_File_Location" />
You file location should be relative to samisays\sound folder. For example, if you
have your sound file at samisays\sound\menu\myitem.wav, you should write
“menu\myitem.wav” for the location.
2 Describe the command pattern in system.xml <effects> section. Typically the
description looks like this:
<effect name="NAME" audio="MY_ID" cmd="sometool %s %s other param" />
Here name is the textual name of this effect, which will be displayed while going
through the “Add sound effect” menu. audio is the audio ID of the sound file that
will be played this effect in the “Add sound effect” menu. This audio ID should
be defined in the <audiofile> section. cmd is the command line pattern used to
call the external tool. The two “%s” in this line indicate the filenames of the input
and output files.
3 Copy the tool (sometool.exe in this example) and related files to “tools” folder.
NOTE: It is simple to implement a new sound effect, but there are some
restrictions on the tool parameter order (input filename must be in front of output
filename). This can be improved by using more complicated command patterns.
To implement this, you will need to change xmlparser.py.
4.2.2 Class based effects
Class based effects are implemented by writing a class which manipulates the sound
data directly. Here are the steps to add one:
1 Record your effect name and define it in <audiofile> section in system.xml as
2 You need to write a class which inherits EffectMaker and put it in a module with
the same name in the effects folder. Implement the sound effects by doing sound
manipulation in the EffectMaker.applyEffect(clip) function.
3 Add a corresponding import statement in instance.py to make sure its instance
can be created in the run time. (See instance.py for examples)
4 Add a line in system.xml <effects> section indicating the adding of the new effect.
A typical line looks like the one below. Here name and audio have the same
meaning as in Section 4.2.1. handler here indicates the class used for this sound
<effect name="NAME" audio="MY_ID" handler="ClassName" />
Now your sound effect should appear in the “Add sound effect” menu.
Please refer to Detail Designs document for more details about python modules and other
4.3 Making an installer
In order to make an installer, you will need to first be able to run the Sami Says
python scripts as python scripts instead of through the .exe’s that are used for distribution.
The necessary modules have to be installed in a particular order in order to operate
correctly. This is how we suggest going about the installation of all the necessary
1. Copy mfc71.dll to your /system32 or /systemNT directory if it isn’t there
already. This file is necessary and often missing from systems.
2. Install Python 2.4.
3. Install the Microsoft SAPI 5.1 redistributable.
4. Install the extra Microsoft voices.
5. Install pySonic 0.8 for Python 2.4.
6. Copy fmod.dll to your /system32 or /systemNT directory (however just
putting it in your /Python24 directory seems to work just fine).
7. Install Mark Hammond’s Python win32all extensions.
8. Install pyTTS 3.0.
9. Install wxPython 2.5 for Python 2.4, Unicode version.
Now you can copy the Sami Says source files to a folder in your /Python/Lib folder and
run them with python. In order to make an installer, there are a few more tasks which
must be done.
1. Install py2exe.
2. Install ZODB 3.2 for Python 2.4.
3. Install epydoc 2.1.
4. Install Inno Setup.
5. Run the script setup.py using py2exe from inside the Sami Says directory in
order to create a folder with (most) of the files you will need to install. Try
running the SamiSays.exe file found in the /dist folder that is generated by
py2exe. Due to software conflicts, py2exe sometimes doesn’t quite retrieve
all of the necessary libraries correctly and so there are a few hacks that you
may need to employ.
a. Make absolutely sure that your Python path (as viewed both from your
system environment variables and PythonWin) points into /Lib/site-
b. In __init__.py, which is found in the Pythong24/Lib/site-packages/pyTTS
directory, take the import statements (import sapi, from sapi import …) at
the beginning of the file out of their try block. The files are there, but the
method by which py2exe imports them will make the try block fail.
c. In sapi.py, which is found in the Python24/Lib/site-packages/pyTTS
directory, add the line “from win32com import gen_py”.
6. The library.zip file in the /dist directory created by py2exe will not contain
some necessary files due to its method of copying. To fix this, you will need
to do the following.
a. Overwrite the win32com folder inside library.zip with a copy of the
/win32com folder located in your /Python24/Lib/site-packages folder.
b. Copy all the files from /Python24/Lib/site-packages/pyTTS to the /pyTTS
folder inside library.zip without overwriting! Only the files that are not
currently present in library.zip/pyTTS should be added. Overwriting
any of the files that are currently in library.zip/pyTTS could result in
your executable not being functional!
7. Now open the inno.iss file inside the /dist directory created by py2exe. This is
the setup script used by Inno Setup to create a single file installer for
distribution. From the Build menu, select Compile to create the installer. The
installer will be called samisetup.exe (or whatever is specified in the
OutputBaseFilename attribute in inno.iss) and will be created in the /Output
directory inside the /dist directory.