descriptions of captioning accessibility technologies by stariya


									                                       The Journal for Undergraduate Multimedia Projects

                Descriptions of Captioning Accessibility Technologies:
              How do they function with varying computational interfaces?
                                 Working Document

Captioning Technologies
Synchronized Accessible Media Interchange (SAMI)

“In 1997, when no standard for closed captions in digital media existed, Microsoft
developed Synchronized Accessible Media Interchange (SAMI) and implemented it in
Microsoft Encarta Reference Suite, Windows Media Player, and Microsoft DirectShow
application programming interface.

Concurrently, members of the World Wide Web Consortium (W3C) created Synchronized
Multimedia Integration Language (SMIL), which can be used for closed captions. Microsoft
Internet Explorer supports the latest version, SMIL 2.0, creating a synchronization
framework based on it called HTML+TIME.

In practice, to attempt to integrate captions into an existing, fully synchronized SMIL 2.0
media file is sometimes costly. In contrast, a SAMI document contains only closed
captions text; it can be edited separately from the media file and requires fewer
programming skills.

SAMI simplifies captioning for developers, educators, and multimedia producers and
designers who will now find it easier to make their work more universally accessible. The
SAMI file format specification is available to the public as an open (no licensing fees)
standard.” i

“SAMI is a Microsoft public specification that allows closed captions to be played in the
Windows Media Player on the Windows OS only. When the Windows Media Player is used
as a stand-alone player, viewers can turn the captions on and off using a menu selection.
However, when the player is embedded in another application, such as a Web page, the
developer must provide the toggling feature through a button in the application's
interface. For additional flexibility, this interface can also provide options to change the
font size or text color.

At the time of this writing, SAMI does not support closed audio descriptions. Instead,
descriptions must be recorded permanently as open descriptions directly into a video's
regular soundtrack. If this approach is used, authors should also provide a separate
version of the video with the original program audio (without audio descriptions).”ii

“For details about the SAMI format and the platforms, see the MSDN Library. Examples
are also available in the MSDN article Understanding SAMI 1.0.”iii

Synchronized Multimedia Integration Language (SMIL)

“Internet Explorer supports SMIL 2.0 as part of HTML+TIME implementation. For details
on the specification and examples of HTML+TIME see the MSDN Library. For more
information on SMIL 2.0, see the W3C recommendation, and W3C Synchronized
Multimedia page.”iv

“SMIL is played by the QuickTime Player, RealPlayer, the Oratrix GRiNS Player and the
Ambulant Player. SMIL multimedia presentations are made up of elements-sound, video,
pictures and text that are stored separately and then synchronized at the time of
playback. SMIL-formatted multimedia can be delivered via the Internet or a local file
system via hard drive, CD or DVD. Visit the W3C's Synchronized Multimedia page for
complete information about SMIL and related activities at the W3C.

When authored correctly, SMIL allows users to turn captions and descriptions on and off
via a player interface. The QuickTime Player, GRiNS Player, Ambulant Player and
RealPlayer each provide a menu selection or dialog box for this feature, but for better
accessibility authors should consider adding accessible buttons to the player interface for
easier toggling of tracks. In fact, this is crucial when embedding a player into a Web

SMIL provides test attributes that can be used to programmatically determine, among
other things, the viewer's player preferences for captions and descriptions. The
RealPlayer and Ambulant Player, for example, support the SMIL test attributes for
captions (systemCaptions) and descriptions (systemAudioDesc), while the Quicktime
Player does not.

You can use MAGpie to author accessible SMIL presentations, but if you wish to write
SMIL code yourself, details are provided on the NCAM website. The RealPlayer and
Ambulant Player provide reliable support for the current SMIL recommendation. Support
for SMIL in the QuickTime Player is selective, so test your presentation thoroughly before
making it available publicly. Also read about QuickTime's support for SMIL at Apple's
Developer Connection site.

Take note that different SMIL players provide varying levels of implementation-- in other
words, some players implement the entire specification, and some use only parts of it.
Also, not all of SMIL's accessibility features are supported by all SMIL players. In these
cases, we offer workaround solutions that will work with existing players.”v

Open Captions
“Open caption is text embedded directly in the video screen. Open captions do not
require a caption decoder in the media player.

In a computer environment, open captions are simply an additional track that is added at
the end of a video production process. They are embedded in the media file itself, and
therefore are readable by any player. For example, any MPEG movie that contains open
captions can be played in any multimedia player with MPEG decoder support.

For computers, you need a browser (for example, Microsoft Internet Explorer) or player
(for example, Windows Media Player) that can decode, or parse the caption text, which is
stored in an additional file.

Use open captions when:

      You are not sure that your media file(s) will be read by a browser or player that
       can read the caption text.
      You are not able to specify which player will be used. Because there are
       competing standards and implementations, it may be too costly to create caption
       text for multiple players.”vi

Captioning Options for Various Streaming Media Players
Stand-alone Players

Display the video in a separate window outside of the Web browser

Popular options: Windows Media Player, Real Player, and QuickTime

Embedded Players

Display the video inside of a Web page

Popular options: Flash-based (e.g., YouTube and Google Video); Java-based (e.g.,
Clipstream) players; several other new competing players and formats (e.g., Microsoft’s
Silverlight technology, though it is currently limited to the Microsoft Windows and Apple
Mac operating systems)vii

Challenges of Captioning for Various Players

Each of the various streaming media players requires specially formatted caption files.
Windows Media Player, Real Player, and QuickTime, for instance, all have their own
special requirements for the makeup of these files, how they are named, and where they
go in conjunction with the video file. Video producers will need to research the various
players and formats before deciding which ones are right for their needs.viii

Adobe Flash

“The Flash-based video format, by Adobe Systems, Inc., is used on thousands of Web
sites around the world and has become the de-facto standard for video and multimedia
content on the Web. Streaming video sites such as YouTube and Google Video have
popularized the format due to the ease of player installation, support for multiple
browsers and operating systems, and a video compression technology that offers VHS-or-
higher quality using a relatively small amount of bandwidth. The installation process for
the Adobe Flash player is accomplished via a small plug-in for your Web browser which
takes only a few mouse clicks to install.

Adobe provides a number of accessibility options for Flash video using the Distribution
Format Exchange Profile (DFXP) of the timed text authoring format to embed subtitles
into the video file. To learn more about creating captions for the Flash format, visit the
National Center for Accessible Media’s (NCAM) CC for Flash Web site. You can download
the free CC for Flash captioning tool, which will allow you to either create a new caption
file or import an existing caption file from any of several different formats for use in a
Flash video. Additionally, you can learn more about efforts underway to create an
Internet Captioning Forum (ICF) at the NCAM Web site. The ICF is being developed to
help coordinate captioning technologies between the various multimedia vendors on the
Internet. This exciting new initiative has the potential to bring a wealth of new captions
and interest in accessibility to the streaming media format. To learn more about
providing captions for Flash media, and to keep up with the latest developments in Flash
player accessibility, visit Adobe’s accessibility blog. Be sure to visit the DCMP accessibility
search gateway to learn more about the tools and practices involved in adding captions
for whichever media format(s) you choose.”ix

Caption It Yourself (CIY): subtitling/audio description software tools
Media Access Generator (MAGpie)
“Web-based video and audio clips can now be made accessible to users who are disabled
and non-English speakers with a complimentary captioning/subtitling and description
software tool--callled MAGpie--developed by the Media Access Group at WGBH. User-
friendly for educators, young users and those new to multimedia, MAGpie allows authors
to add captions, subtitles and audio descriptions to the three most popular digital
multimedia formats:

       Apple's QuickTime Player
       RealNetworks' RealPlayer
       Microsoft's Windows Media Player” x

“MAGpie can be used to create captions and audio descriptions for SMIL presentations,
and captions only for SAMI and Adobe Flash presentations. Both regular and extended
audio descriptions can be digitally recorded and integrated into a SMIL presentation using
NCAM's free utility, MAGpie 2.01. You may also use any sound-editing program, such as
SoundForge or Audacity, to record the audio files, and then use MAGpie to integrate them
into an accessible presentation.

The easiest method for adding captions to multimedia presentations is to use MAGpie.
MAGpie allows authors to write captions once and output them in formats for
RealNetworks' RealPlayer, Oratrix's GRiNS Player, Apple's QuickTime Player and
Microsoft's Windows Media Player. These players each use proprietary text-display
formats, so captions that play in one player will not play in another (except for the GRiNS
Player, which plays RealNetworks' RealText format). Read complete information on using
MAGpie to create and synchronize captions on the MAGpie Web site.

As of this writing, there are serious shortcomings with the display of scientific and
mathematical expressions in captions. No standard method exists for displaying complex
math or science notation within multimedia players. (Existing mathematical markup
languages, such as MathML, are not yet supported in multimedia players.) Therefore,
representing anything beyond simple mathematical expressions in captions can only be
accomplished via text.

Research and development efforts will eventually simplify the creation and display of
scientific and mathematical expressions in captions. The W3C is currently defining a
standard timed-text format that could eventually be adopted by multimedia player
manufacturers” xi

“For additional information about MAGpie's features and capabilities, and to download the
free software, visit”xii

“MAGpie 2.5.1 contains the following updates:

       An export selection for the TTXT format, which can be used to add closed captions
    to movies playable on BlackBerry® smartphones.
        An export selection to automatically combine (or mux) TTXT caption files with
     MP4 or 3GP source files. These closed-captioned videos can be played, and the
     captions decoded, on BlackBerry smartphones.

As before, you can use MAGpie to create caption files for YouTube videos.
MAGpie is a Java2 application which runs on Windows 9x/NT/2000/XP/Vista/Windows 7.
Please note that MAGpie 2.5.1 removes support for the GRiNS player. If you are
currently using GRiNS to caption RealMedia source files, do not replace your current
version of MAGpie with version 2.5.1. (MAGpie 2.5.1 will still export SMIL files for the
RealPlayer, however.) Windows Media clips cannot be used as source files except with
MAGpie 1.0.”xiii

The JW Player for Flash. “It's one of the few players that can display both closed
captions and a closed audio description. Since it is built in Adobe's Flash, 95-99% of your
visitors will be able to watch it.

Other accessibility features of this player are its keyboard controls (using the TAB and
SPACE controls), its screen reader-labeled buttons and the full-screen button. Also,
immediately above the player, hidden controls allow screen reader users to control the
basic functionality of the player (Play / Mute / Stop).

This example uses the SWFObject JavaScript to detect if your visitors have the right Flash
plugin (and JavaScript) installed. If so, the player is shown. If not, the player falls back to
show an image linking to a downloadable video file. You can see a demonstration of how
this works by turning off your browser’s JavaScript and reloading this page. Again, both
the image and link have a text label for screen readers.

Note that the downloadable video file is an MP4. This is a high-quality format, which is
very popular among mobile devices such as iPods / iPhones, the PSP and many smart
phones. The MP4 format, though, isn't very useful if your target audience has older
browsers or devices. It's fairly new and isn't supported by the default Windows
installations. If your goal is to maximize reach, it’s best to encode any download as an
MPG; its quality relative to file size is poor, but it’s able to play on just about everything.
If you want to maximize accessibility even further, you can always hard code the captions
and audio description into the video.

There's a whole range of applications that can transcode your original video files into any
format you’d like (e.g. FLV, MP4, WMV, MPG).

1.       NCAM's magPIE (free, also for MAC)
2.       URUSoft's Subtitle Workshop (free)
3.       Manitu Group's Captionate ($60)
4.       VideoToolShed's SubBits subtitler ($199)

Both magPIE and Subtitle Workshop support the export of captions to W3C's TimedText
(Flash), SMIL/QTtext (Quicktime) and ASX/SAMI (Windows Media). For Subtitle
Workshop you need the TimedText output script (available from Naomi Spirit).
Captionate, (intended for FLV video) can only export to W3C's TimedText, though it can
also save captions as metadata.
Only magPIE has support for recording audio descriptions, but it’s somewhat limited. You
cannot record to MP3 and cannot merge recorded samples or integrate them with a
video. For now, a full-fledged audio/video suite, like Adobe's Premiere / Audition or
Apple's iMovie/GarageBand, is needed for creating closed audio descriptions.”xiv

“CCforFlash, a free Flash component that can be used to display captions of Flash video
and audio content”xv

“CCforFlash is a component that is added to your Flash project to display captions. If you
are not a Flash programmer and need a pre-existing media player that already has
CCforFlash inside it, check out NCAM's Flash-based media player, ccPlayer.
CCforFlash is used to display captions with Flash video and audio content. These captions
are stored in external files formatted in the W3C's DFXP format which can be created with
MAGpie, NCAM's free captioning application. CC for Flash can also display captions saved
in Apple's QTtext format. QTtext files can be created by professional caption authoring
tools, with MAGpie or with CaptionKeeper, NCAM's line-21 caption-conversion tool.

The CC for Flash component is customizable and allows authors to set defaults, such as
the width and height of the caption area, the caption area's background and text colors,
the opacity of the background (for placing captions over the video), and the font family
and font size of the captions themselves.
Other features include:

      access to captions stored internally in the Flash movie or embedded in the FLV
       video file;

      caption search, returning the timecodes where text strings occur;

      roll-up caption display;

      switching between multiple languages of captions that have been stored in the
       same DFXP file;

      ability to provide parsed caption data to other objects in the Flash movie;

      manual operation where a timecode is presented and the corresponding caption is
       displayed (AS3 version only).

For those who are not handy with Flash programming, NCAM has created two flexible
players which ease the process of providing captioned Flash video and MP3 audio files:
ccPlayer, incorporating the CC for Flash component, allows you to embed a FLV player on
your Web page; ccMP3Player, which also incorporates the CC for Flash component, plays
back MP3 files in a Web page with corresponding caption files. Both players are accessible
to screen readers and can be operated solely from the keyboard.”xvi
“ccPlayer and ccMP3Player, free players that incorporate CCforFlash components;
useful for non-Flash authors who want to add captions to Flash video or audio,

“CaptionKeeper, software that converts television closed-caption data into Web
streaming formats”xviii

Web-based Captioning/Subtitling Tools

CaptionTube The latest CIY tool, CaptionTube has a clean (and simple) user interface
and multi-language capability similar to many of the other resources identified below. It
is slightly more integrated with YouTube than the others, and features a convenient
export tool which allows you to e-mail the captions to a video’s owner (if you’re
captioning for someone else) or download a .SUB or .SRT file. A CNET review of
CaptionTube (which includes a couple video tutorials) provides a handy introduction to
the service.

TubeCaption An innovative YouTube “mashup” that actually pays you to caption. Sign
up and begin captioning your favorite YouTube videos using the full-featured
Captionizer captioning tool, which provides many useful features including: handy
keyboard shortcuts, multi-language support, and the ability to import and export SRT
files. As an added bonus for your effort, you get 50% of the ad Google AdSense revenues
generated from your captions.

dotSUB Allows people from around the world to create caption files in multiple
languages for streaming videos. The dotSUB caption file can be exported to a SRT
format for use with Subtitle Workshop or Google Video (see below). dotSUB Repair is
an online script that replaces missing zeroes in SRT files exported from dotSUB and
Overstream. Use this tool to repair your SRT file before uploading it to Google Video or
importing it to Subtitle Workshop.

Overstream Provides a graphical interface for creating captions. The Overstream Editor
allows the user to export a SRT file which can be uploaded to Google Video to provide
closed captions, or converted to Timed Text XML using Subtitle Workshop, Subtitle
Horse, or MAGpie.

Subtitle Horse A tool for transcribing Flash videos online and exporting/converting a
caption file in several different formats including the Timed Text XML format used by
multiple video players including both the Adobe Flash and Microsoft Silverlight players.
(Subtitle Horse was used to convert timing information for Equal Access in the
Classroom's Timed Text captions)

Easy YouTube Caption Creator A simple tool (designed for easy of use in mind) used to
create captions for YouTube videos. Read about Easy YouTube Caption Creator on the
Accessify blog or watch the Easy YouTube Caption Createor Screencast, also from

“Bear in mind that none of these free captioning sites and tools will make it possible to
create captions which conform to the DCMP Captioning Key. Creating captions with the
proper font style, color, and placement characteristics required to meet the Captioning
Key’s rigorous guidelines involves more extensive tools and techniques than those
covered here. As streaming video player caption technologies continue to evolve, the
DCMP will provide more information and tutorials on our site to assist parents,
educators, and others in creating “DCMP Approved” captions to benefit learners of all
ages and abilities.”xix

Universal Subtitles: Works with Vimeo, YouTube,, h.264, HTML5

“It's powered by viewers and other volunteers: it's collaborative editing and translation,
like a wiki. It's the world's easiest subtitle creator - type and tap. The fastest way to add
subtitle functionality to a single video or a whole site. Super easy to integrate with no
software to install. Free, open-source, and non-profit!”xx

Desktop Captioning/Subtitling Software

CapScribe (free) A new, open source application for adding captioning and video
description to web and desktop video and audio files. While the editor requires a Mac
(OS 10.3 and greater), your edited content will play back on both Mac and Windows.
Quicktime 6.5 or greater is required for playback on Macs and Windows. (A Web-based
version of CapScribe is currently under development.) A series of CapScribe tutorials
(video/PDF/etc.) are available from the University of Toronto’s Adaptive Technology
Resource Centre.

Captionate A desktop application for captioning Flash videos. The caption output can be
displayed using either the JW FLV Media Player or the Adobe FLVPlayback skin.

CC For Flash (free) A Flash component that can be used to display captions for Flash
video and audio content, as well as caption files saved in Apple’s Quicktime QText

Jubler (free and open source) (Linux, Windows, and Mac) A Java-based tool for creating
captions and subtitles in a variety of formats (installation can be tricky on Windows
machines—Luckily, there is a step-by-step Jubler installation guide available.)

MacCaption Works with Final Cut Pro or any Non-Linear Editing (NLE) system to
produce captions for multiple formats and players. No closed captioning hardware

MovCaptioner (Mac only [Windows version in development]) Utilizes a GUI to create
and synchronize captions in a number of popular formats. Single- and multi-user licenses
Subtitle Workshop (free) (Windows only) The most complete, efficient and convenient
freeware subtitle editing tool. It supports all the subtitle formats you need and has all the
features you would want from a subtitle editing program.xxi

Caption-Ready Video Hosting Providers

YouTube Google has launched its “auto caption” feature for new YouTube uploads by
which text-to-speech software (of the type used by Google Voice) will attempt to
automatically generate captions from a video’s soundtrack. If you’d like slightly more
control over the content of your captions, Google has also implemented “automatic
caption timing.” To use this feature, simply transcribe all of the words found in your
video, and upload it as you would your SRT (timed) caption file (see below). The same
text-to-speech algorithms used for “auto caption” will synchronize your transcribed text
for you, eliminating the need to worry about timecodes. (Check YouTube help for more
about these automatic features.)

However, the safest (and most accurate) bet would be to upload an SRT file with your
video; once completed, this will enable a “CC” button on the video player interface and
captioning will be turned on by default. YouTube has provided instructions for users
interested in adding captions to their YouTube videos.

Overstream Users can create, import, and export a SRT file with their video. The SRT
“overstreams” are displayed along the bottom of the video during playback.

dotSUB Users can create, import, and export a SRT file with their video. Captions are
displayed on the video during playback. Captions can be turned on or off using up and
down arrow buttons.xxii
 "Captions and Audio Descriptions for PC Multimedia." MSDN . Microsoft Corporation, 22 Mar. 2010.
Web. 27 Jan. 2011. <>.
 "Invent + Build/ Web + Multimedia/Accessible Digital Media Guidelines/Guideline H:
Multimedia/NCAM." The WGBH Carl and Ruth Shapiro Family National Center for Accessible Media.
Media Access Group at WGBH, 2009. Web. 27 Jan. 2011.
< guide/guideline-h-
      "Captions and Audio Descriptions for PC Multimedia."
 "Invent + Build/ Web + Multimedia/Accessible Digital Media Guidelines/Guideline H:
      "Captions and Audio Descriptions for PC Multimedia."
  Kevin Jones, "An Introduction to DVD and Web Captioning." Described and Captioned Media
Program. U.S. Department of Education, Nov. 2008. Web. 27 Jan. 2011.
 "MAG Guide, Vol. 13." WGBH Media Access Group. WGBH Educational Foundation, 2011. Web. 27
Jan. 2011.
 "Invent + Build/ Web + Multimedia/Accessible Digital Media Guidelines/Guideline H:
       "MAG Guide, Vol. 13."
   "Invent + Build/ Web + Multimedia/Tools & Guidelines/MAGpie home/NCAM." Carl and Ruth
Shapiro Family National Center for Accessible Media. Media Access Group at WGBH, 2009. Web. 11
Feb. 2011. <>.
   "Making Video Accessible: LongTail Video: Home of the JW Player." Advertisement. LongTail
Community. LongTail Ad Solutions, 2010. Web. 27 Jan. 2011. <
  "Invent + Build/ Web + Multimedia/ Tools & Guidelines/ NCAM." The WGBH Carl and Ruth Shapiro
Family National Center for Accessible Media. Media Access Group at WGBH, 2009. Web. 27 Jan. 2011.
   "Invent + Build/Web + Multimedia/ Tools & Guidelines/ CCforFlash/NCAM." The WGBH Carl and
Ruth Shapiro Family National Center for Accessible Media. Media Access Group at WGBH, 2009. Web.
12 Feb. 2011. <>.
        "Invent + Build/ Web + Multimedia/ Tools & Guidelines/ NCAM."
   Bill Stark, dir. "Caption It Yourself: Basic Guidelines for Busy Teachers, Families, and Others Who
Shoot Their Own Video." Described and Captioned Media Program. U.S. Department of Education;
National Association of the Deaf, 21 Apr. 2010. Web. 11 Feb. 2011. <>.

  "Universal Subtitles--Make Subtitles, Translations, and Captions for Almost Any Video." Universal
Subtitles. Participatory Culture Foundation, n.d. Web. 12 Feb. 2011. <>.
       See Bill Stark’s guidelines.

To top