Virtual Worlds Technology Roadmap

Document Sample
Virtual Worlds Technology Roadmap Powered By Docstoc
					Virtual Worlds Technology Roadmap
                 December 2008

  Seung Wook Kim, Francisco Imai, Stefan Marti
   HCI Research Team, Computer Science Lab
         Samsung R&D Center, San Jose
VW Technology Trends Analysis

  1. State of The Art
       1.1 Virtual World Clients for Fixed Platform
       1.2 Virtual World Clients for Mobile Platform
       1.3 Virtual World Server Technologies

  2. Catalyst Elements for VW
       2.1 3D content
       2.2 Natural interaction for 3D
       2.3 Artificial Intelligence in VW

  3. Mixed Reality as a Killer Application

  Backup Slides
        Mixed Reality Workgroup Mission and Goals
        Issues in 3D

SISA CSL “Innovate by doing”               -1-
1.1 State of The Art: VW Clients for Fixed Platforms

 Trends:                                                                          Opportunities:
  1. Variety of VW clients:                                                       •Actively apply multicore/parallel
     - standalone, Web-based (ActiveX or Flash), CE-based (game console), etc     computing technologies (for both
  2. Multimodal in-world communication (e.g. textual + bubble + voice chat)       CPU and GPU) on VW client
  3. Support of user created 3D content via standard format (e.g. COLLADA)        development
                                                                                  •Sensor technologies that will
 Challenges:                                                                      accurately translate physical motions
 • Insufficient client performance: 3D rendering quality, physics, animation      into virtual actions
 • Voice chat is becoming standard, but it still doesn’t benefit from non-        •Design intuitive and intelligent UI
   verbal communication (i.e., non-intuitive avatar gesture/action control)       for easier creation and/or
 • Complicated 3D content creation pipeline: CAD/CAM skills are required          manipulation of virtual objects

Flash-based VW                        VW on CE Devices                            Current status:
                                                                                  • The Electric Sheep Company:
                                                                                    Providing Flash-based & private-
                                                                                    labeled VW service, WebFlock
                                                                                  • SMALLWORLDS: Flash-based VW
                                                                                  • VIVATY: Combining Web plug-in
WebFlock          SmallWorlds                                                       Virtual Community (VIVATY Scene)
                                                        Sony’s Playstation Home
                                                                                    with social networking services
VW Web Plug-In                         Standards for 3D content                     (AIM and Facebook)
                                                                                  • SONY: Providing VW community
                                                                                    service (HOME) based on the game
                                                                                    console Playstation3
                                                                                  • FORTERRA: Support for COLLADA*
                                                           Forterra’s Olive VW      into its open and standards-based
                      VIVATY Scene                         Platform supporting
                      for AIM                                                       VW platform, OLIVE
 SISA CSL “Innovate by doing”                                -2-
1.2 State of The Art: VW Clients for Mobile Platforms

 Trends:                                                                    Opportunities:
  1. Mobile 3D clients start providing users with permanent access to VWs   •Enhance the performance of 3D
  2. Connecting VW users in real-time from instant messaging client         graphic accelerating processor for
  3. Emergence of motion sensors and haptic feedback on mobile devices      mobile devices

 Challenges:                                                                •Integrate VoIP/messaging system
                                                                            with mobile immersive 3D client
 • Limited performance of remote rendering & streaming technologies
 • No voice/text enabled thin client connecting VW on mobile platform       •Develop highly-efficient mobile VW
 • Lack of new use models for sensor-mounted mobile devices                 client and its UI mechanism

Mobile VW client         State-of-Art Mobile Device for 3D Graphics
                                                                            Current status:
                                                                            • VOLLEE: Leader in mobile VW
                                                                              service with remote rendering &
                                                                              streaming technologies
                                                                            • VIVOX (w/ Linden Lab): First
                                                                              commercial voice-driven IM client
 Vollee’s SL client
                                                                              (SLim) for Second Life

Thin VW client                                                              • APPLE: iPhone is equipped with a
                         3D Motion Tracking                                   top-level chip (ARM1176JZF) with
                                                                              3D accelerating coprocessor
                                                                              (Samsung S3C6400)

Vivox’s SLim client      Apple’s iPhone with ARM1176JZF & Samsung S3C6400

SISA CSL “Innovate by doing”                               -3-
1.3 State of The Art: VW Server Technologies

 Trends:                                                                        Opportunities:
 1. Open-source approach to interoperable server platform                       •Develop software standards for
 3. Efforts for building MMOG environments in VW                                interoperable platforms for online 3D
 2. Both open and private-labeled virtual world (VW) services are increasing    applications and virtual worlds

 Challenges:                                                                    •Connect different hardware
 • Client-neutral, flexible and highly-trusted server platform is required      platforms (e.g. mobile devices and
 • Difficult to attract and hold a large volume of concurrent users             fixed systems) via VW applications
 • Server maintenance cost is still expensive (per user)                        •Rethink VW platform architecture
                                                                                (e.g. cloud computing)

Open-Source Community             Interoperable VW              MMOG* in VW
                                                                                Current status:
                                                                                • OpenSIM: Open-sourced and
                                                                                  flexible 3D application server
                                                                                  supporting various VW clients
                                                                                • IBM + Linden Lab: Releasing the
                                                                                  first interoperable VW service
                                                                                  platform (Open Grid Beta)
                                                                                • Multiverse: Providing open
                                                                                  development toolkit for MMOG-
                                                                                  type VW, including authoring tools
                                                                                  and source code

   OpenSimulator             Open Grid by IBM+Linden Lab           Multiverse
SISA CSL “Innovate by doing”                                  -4-
 2.1 Catalyst Elements: 3D Content

    1. Renaissance of 3D movies                                                   Opportunities:
       - 11 movie titles released (from 2005 to 2008) and many more in 2009       •Create innovative and intuitive
    2. Consistent popularity of games with 3D content                             user interaction model for immersive
    3. Explosive increase in medical imaging with 3D content ($1.4B by 2013)      3D VW client on consumer electronic
    4. Web-based VW and Mirror World (SL, Google Earth)                           devices
    5. Stereo Synthesis
                                                                                  •Provide mobile solutions for 3D
       - 2D to 3D conversion, depth capturing, multi-aperture imaging
                                                                                  content navigation
                                                                                  •3D interactive movies for VW
    •Displays: auto-stereoscopic displays are expensive and cause user fatigue
    and most of 3D viewing solutions require wearing awkward glasses              •Remote medical service and
    •UX: there is no good solution in the market for natural and intuitive user   training in VW
    interface for 3D content

                                                                                  Current status:
                                                                                  • Philips: Developed auto-
                                                                                    stereoscopic displays. Technical
                                                                                    issues such as narrow angle of
                                                                                    view and user fatigue. No
                                                                                    compelling UX
                                                                                  • 3DV Systems: Detection of body
                                                                                    gestures using depth cameras for
                                                                                    avatar control. Issues of user
                                                                                    fatigue and lack of resolution
                                                                                  • GestureTek: Control of digital
                                                                                    content by tracking gestures.
Source: 3D SMPTE standard activities                                                Could lead to user fatigue
  Wendy Aylsworth (Warner Bros.)                    Source: Insight Media
   SISA CSL “Innovate by doing”                                 -5-
2.2 Catalyst Elements: Natural UI for 3D

 1. Beyond keyboard and mouse: evolution towards alternative HCI tools        Opportunities:
 2. Multi-touch: Evolution towards gesture interfaces on surface              •Combine different technologies to
 3. Multitude of sensors: Progressive adoption of orientation, position and   provide a more immersive and
 motion sensing (optical sensors, inclinometers, accelerometers, compass)     engaging UX for 3D interaction
                                                                              •Create closed-loop systems for user
 Challenges:                                                                  experience and 3D object rendering
 •User fatigue: solutions that rely on mapping human body movements often     by dynamically displaying 3D object
 lead to user fatigue                                                         according to user position and
 •Lack of feedback: it is difficult to give appropriate feedback for 3D       interaction with virtual content

                                                                              Current status:
                                                                              • Seeingmachines: Developer of
                                                                                face API that tracks human face
                                                                                with normal webcam and displays
                                                                                images based on view-dependent-
        Seeingmachines           Softkinetic             Orange                 rendering
                                                                              • Softkinetic: Provides 3D real-time
                                                                                gesture recognition middleware
                                                                              • Mgestyk: Combines 3DV systems
                                                                                depth camera with hand-gesture


SISA CSL “Innovate by doing”                                -6-
2.3 Catalyst Elements: Artificial Intelligence in VW

  1. A.I. engines programmed to learn by interacting with humans in VW           •Virtual pets can be a killer
     - VW gives advantage of embodiment to A.I. engine without hassle or         application for A.I. in VWs
       costs that comes with physical robotics                                   •Adding ability to have pets genuinely
     - VW allows exponential increase in interaction for AI engines to learn     learn and respond to the
  2. There are efforts to make VW presence and interaction more engaging         environment will make them more
                                                                                 real to the user and increase the
  Challenges:                                                                    user/virtual pet bond
  •Recreate self-mind in VW: there is need to create a self-awareness as
  present in human interaction by appropriately planning                         •Autonomous avatars for creating
  •Pattern mining: finding complex patterns requires prohibitively inefficient   rich virtual environments
  •Self-modeling: creating learning algorithms capable to embody intelligence
  •Language and gesture processing: interpretation of language, gestures
                                                                                 Current status:
                                                                                 • Novamente IRC: methodology for
                                                                                     • Imitative learning: instructor shows
                                                                                       by example
                                                                                     • Reinforcement learning:– student
                                                                                       repeats movements with feedback
                                                                                     • Corrective learning:– instructor
                                                                                       actively corrects student’s

 SISA CSL “Innovate by doing”                               -7-
3. Mixed Reality as a Killer Application
 1. Increase in information overlay (Googlemaps, VWs)                          Opportunities:
 2. See-through devices as a platform for mixing VW and RW (iPhone App)
                                                                               •Develop technologies for an
                                                                               unobtrusive system with natural UX
 Challenges:                                                                   for mixed reality
 •Design of wearable displays: current solutions are bulky and too geeky
 •Need for an intelligent agent: user should not be overwhelmed by profusion   •Possible applications:
                                                                                 - Travel planning with shared experience
 of virtual data but system should have an embedded agent that filters
                                                                                   and enhanced participation
 relevant or preferred information                                               - Training of first responders
 •Lack of precision of location sensors: current location sensors need to        - Inter-reality physics: virtual reality
 improve precision in order to identify location for data augmentation             system coupled with real-world one

                 Virtual Continuum (by Paul Milgram, 1994)
                                                                               Current status:
                                                                               • BBC R&D: Have explored the
                                                                                 impact of mixed reality in media:

                                                                               • Nokia Research: Research on
                                                                                 mobile mixed reality system with
                                                                                 augmentation of real world using
                                                                                 mobile phones.
                                                                               • GaTech: Augmented Reality in SL

      Nokia Research                                 BBC R&D

SISA CSL “Innovate by doing”                              -8-
Backup slides

SISA CSL “Innovate by doing”
Mixed Reality Workgroup: Mission

  We believe that Mixed Realities have the
  potential to change the way we interact
  with the world in a radical way.
       Under Mixed Reality, we refer to the merging
       of real and virtual worlds to produce new
       environments and visualizations where
       physical and digital objects co-exist and
       interact in real time.

  But radical change can only happen if the
  technology is available on a broad basis.
       The more people who can interact with
       Mixed Realities, the clearer the advantages
       of such a technology will become.

  Therefore, our mission is to improve human interaction with real and
  virtual worlds by encouraging the mass adoption of Mixed Reality
  technologies, and the creation of an industry with very low barriers to
  innovation and business success.

SISA CSL “Innovate by doing”
Mixed Reality Workgroup: Goals

                          We are an Active Workgroup that develops
                          documents which frame the usages, as well as key
                          technical and business model barriers, to the success
                          of the Mixed Reality vision.

 We propose to provide information aimed at
 technology providers, investors, and business
 decision makers:
     • Casebook that documents past efforts, lessons
       learned, and best practices
     • Roadmap of key technical requirements for "pacing
       mass usages“
     • “Grand Challenges” collection to inspire innovation
       in the industry

SISA CSL “Innovate by doing”
Issues in 3D

 3D content creation
   - Quality x Quantity
   - Mastering 3D content for home x cinema
   - 3D creation and workflow training and education

 3D content storage, transmission and distribution
   - File formats, compression, transmission
   - Coordinate work in standard bodies (DCI, MPEG, IEEE 3D Consortium,
     3D4YOU, etc…)

  3D Promotion
    - Common naming conventions and vocabulary
    - Develop content for web distribution
    - End-user and retailer/reseller education and in-store demos

  3D Displays
    - 3D display quality metrics – ghosting, brightness, etc…
    - 3D human factors – eye fatigue, viewing time, viewing angle dependency, etc

SISA CSL “Innovate by doing”
3D Display Technologies

SISA CSL “Innovate by doing”
3DTV vs. 3D-ready TV

SISA CSL “Innovate by doing”

SISA CSL “Innovate by doing”
PDP display market forecast
 Samsung manufactures 3D-ready PDP-TV. This is the expected PDP forecast in
 the USA with 3D TVs dominating the market in the next years.

SISA CSL “Innovate by doing”
3D LCD display market forecast
 It is expected that 3D LCD is going to emerge in the market and grow

SISA CSL “Innovate by doing”
3D and 3D-ready display market

SISA CSL “Innovate by doing”
3D content – movie as a driving force

SISA CSL “Innovate by doing”
TV – How do you know it is 3D-ready?

SISA CSL “Innovate by doing”
3D-ready TV – 3D glasses and PC requirements

SISA CSL “Innovate by doing”
3D-ready TV – 3D glasses and PC requirements

SISA CSL “Innovate by doing”
3D-ready TV – PC requirements

SISA CSL “Innovate by doing”

Shared By: