3D Video for the production of visual effects and
Shared by: dcc48652
3D Video for the production of visual effects and interactive media Production of interactive media Oliver Grau A lot of progress has been made in developing methods to BBC Research & Innovation compute 3D video. Most approaches operate in a studio email@example.com environment under controlled conditions. This usually includes controlled lighting and keying facilities, like uni-coloured This contribution reviews several 3D video techniques that background for chroma keying. were investigated by BBC R&I for use in broadcast Much progress has been made towards high quality visual applications. Further it discusses unsolved production issues in reconstructions of scenes, in particular human beings. However new applications, like the use of 3D video in interactive media. the use of these models in interactive applications raises new issues: What kind of interactivity should be provided? If the 3D graphics in the media industry is traditionally used for the viewer has full control over the (virtual) viewpoint then the production of visual effects (e.g. animation in post-production content must provide a high quality level from all sides. Details for movies) and by the gaming industry. Both have different like the face of persons should not break up when moving production approaches that result in very different kinds of closer. media: Post-production aims at highest photo-realistic picture Another important issue is about how interactive content is quality; the processing is done off-line (rendering alone of a 30 produced. Many approaches demonstrated so far are deploying sec clip can take hours and days). The gaming industry only one or a few actors in a (virtual) studio scenario. From a achieves good visual quality of completely synthetic content at creative perspective it is very hard to establish interactions interactive rendering rates. The production cycle is very between the actor(s) and scene elements that are added as different to the film or broadcast industry and a production can virtual objects or actors since they are not visible on set. last several years. The lack of visual feedback for actors is also a problem in the 3D video has the potential to allow the production of new production of special effects and has been addressed by the visual effects, e.g. free-viewpoint video, relighting, and others. ORIGAMI project . The key idea is to provide the actor Further it can also provide new ways to produce interactive with visual feedback of any (virtual) objects he has to interact media, that is currently dominated by the gaming industry. with. This has been achieved by integrating a view-dependent projection system into the 3D capturing system. Fig. 2 shows 3D video for special effects an immersive actor feedback system working in parallel to the real-time modelling system. It is based on data projectors Special effects are usually making as much as possible use of driven from rendering PCs that project onto the special retro- footage that is captured by real cameras, because this is usually reflective cloth without interference with the chroma-keying the cheapest option and guarantees 'photo-realistic' results. facility, since the light from a LED ring mounted on the Computer generated imagery (CGI) is used in production when camera lenses dominates over the projector light. it is impossible or too expensive to capture a scene with a real The head position of the actor is derived from the captured 3D camera. model and used to synthesise view-dependent images of the 3D video can be used in many ways to allow the integration of virtual scene for each projector. real and virtual objects. BBC R&I developed a multi-camera The system is shown during a demo production that was set in system with 12 cameras in a studio set-up, as depicted in fig. 1. the entrance hall of the British Natural History Museum in Illustration 2: View-dependent projects gives actor a visual Illustration 1: Multi-camera studio set-up and application to queue to interact with virtual objects insert actor in different environment London. The boy was watching a Pterosaur flying through the entire length of the hall. Four projectors were used to generate 3D video is computed based on the visual hull concept. The this 180o view of the hall and the animation. The boy was able use of 3D data was then demonstrated for compositing of to establish exact eye-line contact with the Pterosaur. virtual and real scene components, including changing the lighting conditions [1,2]. References The ability to render the captured content from a different camera viewpoint is one of the most attractive features of 3D  O. Grau, ,3D in content and post-production", book chapter video. It allows the production of special effects or camera in Schreer, Kauff und Sikora, ,3D Communications", Wiley, moves that are impossible or too expensive to achieve with a 2005. real camera.  O. Grau, ,Multi-camera Radiometric Surface Modelling for This concept is very attractive to visualise actions in sport Image-Based Re-lighting", in Proc. of 28th Annual DAGM scenes. The recent collaborative 'iview' project, led by the BBC Symposium, Sept. 12-14 2006, Berlin, Germany. is developing a free-video system that allows the replay of  O. Grau, A. Hilton, J. Kilner, G. Miller, T. Sargeant, J. sport scenes, like football or rugby from any camera angle . Starck, 'A free-viewpoint video system for visualisation of The environment is less controlled than the studio. In particular sport scenes', Proc. of IBC 2006, 7-11 September 2006, camera calibration and segmentation are harder to solve. Amsterdam, NL. Robust 3D reconstruction techniques are required to overcome  O. Grau, ,A 3D production pipeline for special effects in this. TV and film", Proc. of Mirage 2005 conference. March, 1-2 2005, INRIA Rocquencourt, France.