DARPA One-way Phrase Translation System (PTS) Ace J Sarich PH: 410-757-5358 Email: email@example.com What is the DARPA One-way? • It is a voice-to-voice oneway phrase translation system • Input phrase is matched with prerecorded translation and played through a speaker • Software runs on notebook or handheld computer. Background • Initially a text-tovoice phrase translator • Voice-to-voice capability added 1996-97 by NOMI • Multilingual Interview System (MIS) deployed to Bosnia 1997 Arabian Gulf Summer 1998 • Used for Maritime boardings • 500+ phrases in Arabic, Farsi, Urdu, Hindi • <2 months to build and deliver system Kosevar-Albanian One-Way Call for assistance Select Phrases Purchase Computers Record Albanian Speaker 72-Hour Notice Deliverable System 4/6/99 4/8/99 4/10/99 4/13/99 • Builds on Prior Bosnian Effort • For Rapid Deployment in Field • Transparent Setup and Use by Untrained Personnel Deployment Package: 10 lightweight notebooks with microphone, speaker, and case Latest version of DARPA One-way developmental software 800+ humanitarian and medical phrases translated into Albanian Portable Inmarsat phone for software and module downloads in the field. What you can do with the DARPA One-way • Provide general information – “We are here to help you.” – “The soup is cold.” – “Read Miranda Rights.” • Give orders or provide direction – “Open your mouth.” – “lie face down on the ground.” • Ask Questions with yes/no or easily conveyed answer – “Are you thirsty ? – “How old are you? Hold up the number with your fingers.” Who could use it? – Military • • • • Peacekeeping operations Intelligence screening Training Ship boardings and inspections – Government and Law Enforcement • Police departments, Fire departments • Customs inspectors and border patrols • Coast Guard inspections and safety – Humanitarian Assistance and Medical – Travel Industry • Disaster relief • Medical diagnostics and treatment • Business travelers, Aviation flight crews, Ticket Agents, check-in Types of users • Casual (tourist) – Wants simple easy to use PTS with limited capability • Professional (law enforcement, travel, etc) – Wants flexible system but doesn‟t want to be bothered with the details • Dedicated (military) – Wants total control over phrase modules and translations Why not Machine Translation (MT) approach? • How MT approach works • Advantages – The Holy Grail – The Star Trek Universal Translator • Disadvantages – – – – works for simple, non-complex phrases limited number of languages supported with MT models high quality speech input needed for speech recognition. powerful processor, large amount of storage and computer memory needed – translation accuracy is not good – cannot create new language translations easily Components Application Software Speech Recognizer Microphone/ Speaker Tourist Notebook computer MIO Phrase Modules Medical Phrase Database „99 PTS Goals • Phrase translation – 95% accuracy – Rapid translation • Reliability/Robustness – No system crashes or lockups – Absolute control of command and phrase modes – Operation in noisy or vocal environment • Improved Mic, Speaker – Amplified speaker – PTT „99 PTS Goals (cont.) • Ease of installation – Automated installation of integrated speech recognition, application, and user modules – Installation <15 min Training time (hrs) 10 8 6 4 2 0 Bosnia Onew ay DOW 98 DOW 99 • Ease of registration and training – Automated registration <15 min – Computer interactive training < 1 hour „99 PTS Goals (cont.) • Module build – Build mission specific modules in less than two weeks – Automated phrase addition in field Module build time (w ks) 30 20 10 0 Bosnia One-w ay DOW 98 DOW 99 24 8 2 • Voice/audio interface – Voice commands – Audio prompts – Closed notebook operation Performance Progress • Reliability/Robustness • Accuracy • Ease of use with minimal training • Speaker independence • Rapid module build • Voice/audio interface • Software installation • Software portability 100 90 80 70 60 50 40 30 20 Bosnia '97 MIO '98 FBE March March '00 '99 Schedule 2000 plans • • • • • • • Field test and evaluate DARPA One-way „99 Build USB McSpeak Add Alternative phrasing capability Add one and a half-way capability Develop two-way PTS in selected domains Commercialize DARPA One-way Develop handheld multilingual voice phrasebook (MVP) BACKUP SLIDES MIS - DARPA One-way Comparison FEATURES Capabilities Display User interface Portability MIS Multi purpose - WIN 95 full screen GUI mouse/pointer, keyboard and voice interface. Not portable to other OSs Win 95 or NT OS requires Dragon Dictate Large (>300Mb) Graphics interface runs on Win OS; needs Dragon Dictate, MIS, Wave files Must speak phrase exactly to be recognized DARPA One-way '99 Single purpose wit GUI hooks - small, JAVA based display or no display - voice/audio interface Portable to most OS's Written in Java. Small (< 30Mb including OS, application and audio files) Software size Alternate Phrasing - - Phrase addition in the field Custom modules - Not practical. Requires special training and Dragon tools Yes. Allows alternate phrasing Generates best guess Can add alternate phrasing on the fly simple audio prompted phrase and voice recordings on the fly Ability to create custom modules quickly and easily MIS - DARPA One-way Comparison (cont.) Database numbered MIS and wave files numbered “8.3” file names relatively difficult to manage limited number of files Record capable, requires screen interaction requires graphics video display, mouse or pointer, voice input, keyboard unreliable Attempts to translate extraneous noise and speech PTT mode has problems adjusting to background noise User dependent 15-30 min registration Dragon Dictate, MIS and Collect programs installed separately User: 4 hours by trainer Power user: 2days Management program part of basic MIS. Yes. files can be managed using DB program such as Access unlimited number of files Search capability Record capability Control Features Yes, on word search Automated record capability tone or audio prompts absolute control of command and translation mode. Reliable operation in noisy environment Optional PTT Operation in noisy of vocal environment - - User registration and independence Software Installation - User independent No or minimal registration Single install program with prompts User Training DARPA One-way '99 management program Manual - - - User: 1 hour using manual Power user: 1 day Runs on WIN OS. MS Access based phrase database management. Simple, user friendly manual for basic and advanced user.