Interactive Computer Graphics 01

Chapter 1 Graphics Systems and Models Professor: Jiung-yao Huang E-mail: jyhuang@mail.ntpu.edu.tw http://web.ntpu.edu.tw/~jyhuang/Course/ National Taipei University, Dept. CS & IE Contents 1.1 Applications of Computer Graphics 1.2 A Graphics System 1.3 Images: Physical and Synthetic 1.4 Imaging Systems 1.5 The Synthetic-Camera Model 1.6 The Programmer’s Interface 1.7 Graphics Architectures 1.8 Programmable Pipelines 1.9 Performance Characteristics 1 §1.1 Computer Graphics ÜComputer graphics deals with all aspects of creating images with a computer – Hardware – Software – Applications 2 Ch.1 -> Applications of Computer Graphics Example ÜWhere did this image come from? ÜWhat hardware/software did we need to produce it? 3 Ch.1 -> Applications of Computer Graphics Preliminary Answer ÜApplication: The object is an artist’s rendtion of the sun for an animation to be shown in a domed environment (planetarium) ÜSoftware: Maya for modeling and rendering but Maya is built on top of OpenGL ÜHardware: PC with graphics card for modeling and rendering 4 Ch.1 -> Applications of Computer Graphics §1.2 A Graphics System ÜThe General Model (Five major elements): Output Devices… Input Devices ‚ • „ ƒ Image formed in FB 5 FIGURE 1.1 A graphics system Ch.1 -> A Graphics System Pixels and the Frame Buffer ÜRaster -> Pixels – Pixels are stored in the frame buffer. – Resolution: number of pixels in the frame buffer. qIt determines the detail of the image. – Depth (Precision): number of bits for each pixel. qIt determines how many colors can be represented. q8 bits ⇒ 256 colors. q24 or more bits ⇒ 224 or more colors — full-color. 6 Ch.1 -> A Graphics System -> Pixels and the Frame Buffer Pixels and the Frame Buffer (Conti.) 15 pixels Frame Buffer 10 pixels Resolution = 15*10 pixels R G B 10000001 01101000 01100011 FIGURE 1.2 Pixels. (a) Image of Yeti the cat. (b) Detail of area around one eye showing individual pixels Depth = 24 (full-color, true-color, or RGB-color) 7 Ch.1 -> A Graphics System -> Pixels and the Frame Buffer Pixels and the Frame Buffer (Conti.) ÜFrame Buffer – It usually is implemented with special memory chips to redisplay contents fast. – If image can’t be produced in real time, it may be part of system memory. – It may holds complex information such as 3D data. – It may comprises multiple buffers including color buffers. – The terms “frame buffer” and “color buffer” can be used synonymously. Ch.1 -> A Graphics System -> Pixels and the Frame Buffer 8 Pixels and the Frame Buffer (Conti.) ÜGraphics Processing Units (GPUs) – Rasterization (scan conversion): The conversion of geometric entities to pixels. Ex. Red line from point A to B rasterize 0001 0110 0101 1100 0100 1011 …………. – GPU can carry out graphic functions such as rasterization to reduce the load of CPU. – GPU can be on the main board or graphic card and may include the frame buffer. 9 Ch.1 -> A Graphics System -> Pixels and the Frame Buffer Output Devices ÜCathode-Ray Tube (CRT) – Refresh rate qInterlaced: can double the rate; most for TV. qNoninterlaced: becoming more widespread. qViewer near the screen can tell their difference. ÜLight-Emitting Diode (LED) ÜLiquid-Crystal Display (LCD) ÜPlasma Panel ÜHard-Copy Devices: Printers, Plotters… 10 Ch.1 -> A Graphics System -> Output Devices CRT FIGURE 1.3 The cathode-ray tube (CRT). Can be used either as a line-drawing device (calligraphic) or to display contents of frame buffer (raster mode) 11 Ch.1 -> A Graphics System -> Output Devices Color CRT FIGURE 1.4 Shadow-mask CRT 12 Ch.1 -> A Graphics System -> Output Devices Generic Flat-panel Monitor Plasma panel是控制此 兩層的電極 13 FIGURE 1.5 Generic flat-panel display LED display, LCD display的差 異在此層的電極控制方式 Ch.1 -> A Graphics System -> Output Devices Additive and Subtractive Color ÜAdditive color – Form a color by adding amounts of three primaries – Primaries are Red (R), Green (G), Blue (B) – Form a color by filtering white light with cyan (C), Magenta (M), and Yellow (Y) filters qLight-material interactions qPrinting qNegative film qCRTs, projection systems, positive film ÜSubtractive color (印刷界使用) 14 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1950-1960 ÜComputer graphics goes back to the earliest days of computing – Strip charts – Pen plotters – Simple displays using A/D converters to go from computer to calligraphic CRT ÜCost of refresh for CRT too high – Computers slow, expensive, unreliable 15 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1960-1970 ÜWireframe graphics – Draw only lines ÜSketchpad ÜDisplay Processors ÜStorage tube wireframe representation of sun object Color Palate 2 16 Ch.1 -> A Graphics System -> Output Devices Sketchpad ÜIvan Sutherland’s PhD thesis at MIT – Recognized the potential of man-machine interaction – Loop qDisplay something qUser moves light pen qComputer generates new display – Sutherland also created many of the now common algorithms for computer graphics 17 Ch.1 -> A Graphics System -> Output Devices Display Processor ÜRather than have the host computer try to refresh display, a special purpose computer called a display processor (DPU) is used 18 ÜGraphics stored in display list (display file) on display processor ÜHost compiles display list and sends to DPU Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1970-1980 ÜRaster Graphics ÜBeginning of graphics standards – IFIPS qGKS: European effort – Becomes ISO 2D standard qCore: North American effort – 3D but fails to become ISO standard ÜWorkstations and PCs 19 Ch.1 -> A Graphics System -> Output Devices Raster Graphics ÜAllows us to go from lines and wire frame images to filled polygons Flat shading 20 Smoothing shading Color Palate 4 Ch.1 -> A Graphics System -> Output Devices Color Palate 3 PCs and Workstations ÜAlthough we no longer make the distinction between workstations and PCs, historically they evolved from different roots – Early workstations characterized by qNetworked connection: client-server model qHigh-level of interactivity – Early PCs included frame buffer as part of user memory qEasy to change contents and create images 21 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1980-1990 Realism comes to computer graphics Smoothing shading Color Palate 4 22 Environment mapping Color Palate 7 Bump mapping Color Palate 6 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1980-1990 ÜSpecial purpose hardware – Silicon Graphics geometry engine qVLSI implementation of graphics pipeline ÜIndustry-based standards – PHIGS – RenderMan ÜNetworked graphics: X Window System ÜHuman-Computer Interface (HCI) 23 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 1990-2000 ÜOpenGL API ÜCompletely computer-generated featurelength movies (Toy Story) are successful ÜNew hardware capabilities – Texture mapping – Blending – Accumulation, stencil buffers 24 Ch.1 -> A Graphics System -> Output Devices Computer Graphics: 2000ÜPhotorealism ÜGraphics cards for PCs dominate market – Nvidia, ATI, 3DLabs ÜGame boxes and game players determine direction of market ÜComputer graphics routine in movie industry: Maya, Lightwave ÜProgrammable pipelines 25 Ch.1 -> A Graphics System -> Output Devices Input Devices ÜPointing Devices – keyboard, mouse, joystick, data tablet… – to indicate a particular location on the display. ÜThree or more dimensional devices – for Games, CAD, and VR applications – 3D: laser range finders and acoustic sensors – higher-dimension: data gloves including many sensors; computer vision systems. 26 Ch.1 -> A Graphics System -> Input Devices §1.3 Images: Physical and Synthetic ÜIn computer graphics, we form images which are generally two dimensional using a process analogous to how images are formed by physical imaging systems – – – – 27 Cameras Microscopes Telescopes Human visual system Ch.1 -> Images: Physical and Synthetic Objects and Viewers Ü3D objects Ú(viewers)Ú 2D images ÜObjects + Viewers Ú Image-information process ÜObjects: – exist independent of any image and viewer – generally can be defined by a set of vertices ÜViewers: – form images from objects – human visual system and camera – the same object, different viewers, different images 28 Ch.1 -> Images: Physical and Synthetic -> Objects and Viewers Objects and Viewers (Conti.) FIGURE 1.6 Image seen by three different viewers. (a) A’s view. (b) B’s view. (c) C’s view. 29 Ch.1 -> Images: Physical and Synthetic -> Objects and Viewers Camera System FIGURE 1.7 Camera system. 30 Ch.1 -> Images: Physical and Synthetic -> Objects and Viewers Light and Images ÜElements of image formation – Objects – Viewer – Light source(s) ÜAttributes that govern how light interacts with the FIGURE 1.8 A camera system materials in the scene with an object and a light source. ÜNote the independence of the objects, the viewer, and the light source(s) 31 Ch.1 -> Images: Physical and Synthetic -> Objects and Viewers Light ÜLight is the part of the electromagnetic spectrum that causes a reaction in our visual systems – Generally these are wavelengths in the range of about 350-780 nm (nanometers) ÜLong wavelengths appear as reds and short wavelengths as blues FIGURE 1.9 The electromagnetic spectrum 32 Ch.1 -> Images: Physical and Synthetic -> Light and Images Global vs Local Lighting ÜCannot compute color or shade of each object independently – Some objects are blocked from light – Light can reflect from object to object – Some objects might be translucent FIGURE 1.10 Scene with a single point light source. 33 Ch.1 -> Images: Physical and Synthetic -> Light and Images Imaging Models ÜImage-formation techniques: – Ray tracing (Backward ray-tracing) qCan simulate complex physical effects closely qUnsuitable for real-time – Photon mapping (Backward and forward ray-tracing) – Radiosity qA two-pass global illumination algorithm that solves the rendering equation Based on ray interactions qWorks best for surfaces that scatter the incoming light equally in all directions qAlso can’t be implemented in real time for now Based on conversation of energy 34 Ch.1 -> Images: Physical and Synthetic -> Imaging Models Ray Tracing and Geometric Optics ÜOne way to form an image is to follow rays of light from a point source finding which rays enter the lens of the camera. – However, each ray of light may have multiple interactions with objects before being absorbed or going to infinity. – Figure 1.11 Ray A enters camera directly. Ray B goes off to infinity. Ray C is reflected by a mirror. Ray D goes trough a transparent sphere. 35 FIGURE 1.11 Ray interactions. Ch.1 -> Images: Physical and Synthetic -> Imaging Models Ray Tracing 36 Ch.1 -> Images: Physical and Synthetic -> Imaging Models Photon Mapping 37 Three teapots, primarily illuminated by indirect light reflecting from the back wall of the room http://www.pbrt.org/gallery.php Ch.1 -> Images: Physical and Synthetic -> Imaging Models Radiosity 38 Ch.1 -> Images: Physical and Synthetic -> Imaging Models §1.4 Imaging Systems ÜThe Pinhole Camera – simple ÜThe Human Visual System – extremely complex 39 Ch.1 -> Imaging Systems The Pinhole Camera ÜProjection of (x,y,z): zp = -d y yp = z/d x xp = z/d y x (x,y,z) h (xp,yp,zp) d θ z ÜDepth of field: – Ideal: infinite (every point is in focus) 40 FIGURE 1.12 Pinhole camera Ch.1 -> Imaging Systems -> The Pinhole Camera The Pinhole Camera (Conti.) ÜField (angle) of view: θ = 2 tan-1(h/2d) ÜDisadvantages: – Almost no light enter the camera – No different angle of view ÜSolution: – Replace pinhole with a lens 41 FIGURE 1.14 Angle of view Ch.1 -> Imaging Systems -> The Pinhole Camera The Human Visual System ÜIris: adjusts the amount of entering light ÜRetina: forms an image – Rods: monochromatic, night vision – Cones: color vision q 3 types (for 3 primary colors, most sensitive to green) q Only these 3 brightness (light intensity) values are sent to the brain ÜFinally, the visual cortex carries The human visual system out high-level functions such as object recognition. 42 – Rods and cones determine the resolution (visual acuity) FIGURE 1.15 Ch.1 -> Imaging Systems -> The Human Visual System §1.5 The Synthetic-Camera Model ÜThe actual process of an image system FIGURE 1.16 Imaging system 43 Ch.1 -> The Synthetic-Camera Model The Basic Principles FIGURE 1.17 Equivalent views of image formation. (a) Image formed on the back of the camera. (b) Image plane moved in front of the camera. 44 Ch.1 -> The Synthetic-Camera Model The Synthetic-Camera Model ÜForming image similar to optical system ÜSeparation of objects, viewer and light source ÜUsing simple Projector geometric calculations ÜClipping rectangle (clipping window) – Determine which objects will appear in the image p Projection plane Projection of p Center of Projection (COP) Ch.1 -> The Synthetic-Camera Model FIGURE 1.18 Imaging with the synthetic camera 45 Clipping Window ÜClipping window, or clipping rectangle, is a window on the projection plane that decide which objects will appear in the image FIGURE 1.19 Clipping. (a) Window in initial position. (b) Window shifted 46 Ch.1 -> The Synthetic-Camera Model Advantages ÜSeparation of objects, viewer, light sources ÜTwo-dimensional graphics is a special case of three-dimensional graphics ÜLeads to simple software API – Specify objects, lights, camera, attributes – Let implementation determine image ÜLeads to fast hardware implementation 47 Ch.1 -> The Synthetic-Camera Model §1.6 The Programmer’s Interface ÜUser interface is a method for the user to interact with a graphics system FIGURE 1.20 Interface for a painting program 48 Ch.1 -> The Programmer’s Interface §1.6 The Programmer’s Interface ÜThe programmer sees only the API – shield from the hardware and the implementation details FIGURE 1.21 Application programmer’s model of graphics system. 49 Ch.1 -> The Programmer’s Interface The Pen-Plotter Model ÜMainly for 2D; doesn’t extend well to 3D ÜEx1: moveto(0,0); lineto(1,0); lineto(1,1); lineto(0,1); lineto(0,0); (0,1) (1,1) (0,0) (1,0) ÜEx2: write_pixel(x,y,Red); 50 Ch.1 -> The Programmer’s Interface -> The Pen-Plotter Model The Pen-Plotter FIGURE 1.22 Pen plotter FIGURE 1.23 Output of pen-plotter program for (a) a square, and (b) a projection of a cube. 51 Ch.1 -> The Programmer’s Interface -> The Pen-Plotter Model Three-Dimensional APIs ÜFunctions in the API need to be specified – – – – Objects A viewer Light sources Material properties 52 Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs Three-Dimensional APIs (Conti.) ÜObjects – Usually defined by sets of vertices – OpenGL define primitives through lists of vertices: Ex: glBegin(GL_POLYGON); // object type // vertex A // vertex B // vertex C z x y glVertex3f(0.0, 0.0, 0.0); glVertex3f(0.0, 1,0, 0.0); glVertex3f(0.0, 0.0, 1.0); glEnd(); // end of object definition – More complex primitives, such as curves and surfaces, are approximated by a series of simpler primitives – Can also access pixels in the frame buffer 53 Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs Three-Dimensional APIs (Conti.) ÜViewer – Specifications qPosition qOrientation qFocal length qFilm plane – Can use a series of coordinatesystem transformation – Another type: two-point perspective – OpenGL functions: 54 FIGURE 1.25 Camera specification qgluLookAt(cop_x,cop_y,cop_z,at_x,at_y,at_z,up_x,up_y,up_z); qglPerspective(field_of_view,aspect_ratio,near,far); Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs Two-point Perspective One-point perspective of a cube FIGURE 1.26 Two-point perspective of a cube. 55 Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs Two-point Perspective 56 Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs Three-Dimensional APIs (Conti.) ÜLight sources – Location, strength, color, and directionality ÜMaterial properties – Characteristics or attributes of the objects – Specified through functions when each object is defined 57 Ch.1 -> The Programmer’s Interface -> Three-Dimensional APIs A Sequence of Images - Wireframe flat polygons Smooth shading of the polygons Render time Complexity Wireframe using NURBS surfaces Surface texture Bump mapping Environment map Anti-aliasing method + 58 Ch.1 -> The Programmer’s Interface -> A Sequence of Images The Modeling-Rendering Paradigm Ü Separate the modeler and the renderer to implement them with different software and hardware Ü Interface file: can be a text file that describes objects and information important to the renderer Ü Scene graph: models including the objects, lights, cameras, and material properties Modeler Interface file Renderer Modeler 1 Modeler 2 59 Interface file Renderer 1 Renderer 2 Ch.1 -> The Programmer’s Interface -> The Modeling-Rendering Paradigm Graphics Architectures ÜApplication program ---API---> hardware and functional software ÜEarly graphics systems: – CPU processes all instructions including graphics instructions – Would burden an expensive computer 60 FIGURE 1.28 Early graphics system Ch.1 -> Graphics Architectures Display Processors ÜFreeing the host for other tasks ÜAvoid flicker FIGURE 1.29 Display-processor architecture 61 Ch.1 -> Graphics Architectures -> Display Processors Pipeline Architectures ÜIncrease the throughput of the system ÜMust balance the latency of the system Application Program Display FIGURE 1.31 Geometric pipeline 62 Ch.1 -> Graphics Architectures -> Pipeline Architectures Vertex Processing ÜConvert object representations from one coordinate to another ÜCoordinate change = matrix transformation – We use 4x4 matrices ÜAlso computes vertex colors 63 Ch.1 -> Graphics Architectures -> Vertex Processing Clipping and Primitive Assembly ÜClipping volume – determine which objects are inside or straddle the edges in the image ÜAssemble sets of vertices into primitives before clipping 64 Ch.1 -> Graphics Architectures -> Clipping and Primitive Assembly Rasterization ÜConvert primitives from vertices to pixels in the frame buffer (fragments) ÜFragments carry with information including colors, location, and depth 65 Ch.1 -> Graphics Architectures -> Rasterization Fragment Processing ÜDetermine whether the fragments are visible ÜDetermine the final color of each fragment after blending and texture mapping 66 Ch.1 -> Graphics Architectures -> Fragment Processing §1.8 Programmable Pipelines ÜFor now, no render approach leads to realtime performance ÜPipeline architectures dominate the graphics field ÜPipeline processes are now programmable by application program and thus many timewasting techniques may be done in real time 67 Ch.1 -> Programmable Pipelines Performance Characteristics ÜTwo types of processing – Geometric processing qFocus: Matrix transformation qSuited for pipeline – Rasterization qDirect manipulation of bits in the frame buffer qFocus: Move blocks of bits quickly ÜPipelines are incorporated within GPUs ÜBalance: realism or real-time? 68 Ch.1 -> Performance Characteristics Questions?? 69