PresenterMouse LASER-Pointer Tracking System Evgeny Popovich (I.D.: 319204103) The Faculty of Computer Science – Technion I.I.T. Joint work with Zachi Karni Supervised by Prof. Craig Gotsman Figure 1: The PresenterMouse system at work. Abstract We present a novel LASER-Pointer tracking system for use in interactive presenta- tions. The LASER-Pointer’s red dot is meant to draw the audience’s attention to a spe- cific place in a slide. Our system enables presenters to use the LASER-Pointer as they would a regular mouse cursor. The system detects the red dot on the screen and automatically brings the cursor to it. This feature paves the way for highly interactive and dynamic presentations. 1. INTRODUCTION Computer-based presentations are today the most common way to teach courses and make scientific and business presentations. To support this, most university class- rooms and business conference rooms are equipped with high-resolution projectors and computers. In order to control their presentations, presenters must use the computer keyboard and mouse. Operations such as forwarding to the next slide or handling the presented ap- plication anchor presenters to their computers instead of permitting them to face and address audiences. The development of cordless mice has given presenters some de- gree of freedom by allowing them to control presentations from a distance, yet they are still required to place the mouse on a hard surface in order to use it. LASER-Pointers are small gadgets that are able to generate a narrow red light beam and project a red dot onto almost any surface. Presenters usually use them to draw the audience’s attention to important issues on the screen. Logitech® Cordless-Presenter combines a LASER-Pointer with a cordless mouse. Although using such a device en- ables the presenter to control a simple presentation (going back and forward between slides), it cannot be used for complex applications, for example: manipulating 3D ob- jects in 3D viewers. Our project enables the presenter to perform any mouse task using only the LASER- Pointer’s red beam and the mouse buttons, by moving the mouse cursor to the position of the red LASER point. This document is organized as follows: Section 2 lists the system components, Section 3 briefly describes the system algorithm, Section 4 describes the installation proce- dure, Section 5 describes the PresenterMouse daemon initialization and Section 6 briefly describes the puzzle application, which was developed to demonstrate the Pre- senterMouse system. Figure 2: Zachi Karni is using the PresenterMouse system. 2. SYSTEM COMPONENTS The PresenterMouse system includes the followings components: Computer: It must run a Windows®1 operating system and host the presentation and the PresenterMouse daemon application. The PresenterMouse daemon is optimized and on Pentium III 1000Mhz processor, it uses less than 20% of the CPU time. Projector: A wide-screen projector must be connected to the computer. USB-Camera: A simple web-cam must be connected to the computer using a USB interface. We used two web-cams: 3Com®HomeConnectTM and IBM UltraPort. Each of them uses the USB 1.1 interface and can deliver up to five 320×240 True-Colors images per second. Modern cameras use the USB 2.0 or FireWire interface. Such cameras are able to deliver high-resolution images at high frame rates. Using such de- vices will enhance the PresenterMouse performance and will increase its accuracy. LASER-Pointer: Any simple red-beam LASER-Pointer commonly used in presenta- tions is suitable. However, we used the Logitech® Cordless-Presenter, which com- bines a LASER pointer together with a cordless mouse. This allowed us to use the mouse buttons in order to control the presentation. (a) (b) (c) (d) Figure 3: PresenterMouse components: (a) The complete system (b) the projector screen (c) Logi- tech® Cordless-Presenter (d) IBM ThinkPad T22 with the IBM UltraPort Camera 1 PresenterMouse was tested under Windows XP Pro, XP Home and 2000. 3. ALGORITHM The heart of the PresenterMouse system is the LASER red dot recognition algorithm, which is based on a high-pass filter. We consider only the red component of each frame image. (We found out that although the LASER point is red, its center pixels are usually captured as white (due to surface reflection and camera saturation)). Each new frame image, I rt , is differentiated from a mean frame M rt , which is the mean of the previous frames (we used ten previous frames): 10 Drt = I rt − M rt , M rt = 1 10 ∑I i =1 t −i r We consider the pixel with the highest value in Drt as the candidate for the new posi- tion of the LASER point. We found that when a LASER point exists, the highest value in Drt locates it. However, even when there are no LASER points in the image the highest value locates arbitrary points. In order to distinguish between the two cases we set the mouse cursor to the candidate position only if the following two conditions take place: The difference between the highest value in Drt and the mean of the ten successive highest values is higher than a threshold (the user specified sensitivity value): Drt = sort − descending ( Drt ) 11 Drt (1) − 10 ∑ Drt (i ) ≥ Threshold 1 i =2 The distance between the candidate point position and the previous LASER point po- sition is smaller than 20 pixels. After locating the LASER point in the captured image, it must be correlated to its po- sition in the computer screen space. Since, usually, the image on the computer screen fills only a portion of the captured images (the camera’s lens takes in more than the computer screen), a registration-process must be preformed to correlate between the two (as described in Section 5.4). In the Calibration stage, the user specifies the posi- tions of the four corners of the computer screen in the captured image and the correla- tion is done using the following two equations: a⋅ x + b⋅ y + c d ⋅ x + e⋅ y + f u= v= , g ⋅ x + h ⋅ y +1 g ⋅ x + h⋅ y +1 where (x,y) is the LASER point in the captured image and (u,v) is the same point on the computer screen. In order to find the constants a to f, the following linear equation needs to be solved after the Calibration stage. x0 y0 1 0 0 0 0 0 a 0 x1 y1 1 0 0 0 −W ⋅ x1 −W ⋅ y1 b W x2 y2 1 0 0 0 −W ⋅ x2 −W ⋅ y2 c W x3 y3 1 0 0 0 0 0 d 0 ⋅ = 0 0 0 x0 y0 1 0 0 e 0 0 0 0 x1 y1 1 0 0 f 0 0 0 0 x2 y2 1 − H ⋅ x2 − H ⋅ y2 g H 0 0 0 x3 y3 1 − H ⋅ x3 − H ⋅ y3 h H where (xi,yi) i=0..3 are the calibration points the user specifies during this stage and (W,H) are the computer screen’s resolution (width and height). 4. INSTALLATION 4.1. Download The PresenterMouse wizard and demo applications can be downloaded from then pro- ject web page: http://www.cs.technion.ac.il/~zachik/projects/presentermouse or from the FTP site: ftp://ftp.cs.technion.ac.il/pub/misc/zachik/projects/PresenterMouse.exe. 4.2. Unpack After downloading PresenterMouse.exe, double click on the file and unpack its con- tents into a temporary folder of your choice. 4.3. Install After unpacking PresenterMouse, double click on the setup.exe file to start the Instal- lation wizard. Follow the wizard instructions through to the end of the installation. You can abort the installation by pressing Cancel during each of the wizard's stages. 4.3.1. Welcome Stage Greets new users. 4.3.2. Select Installation Folder Stage Select the folder in which to install the PresenterMouse wizard and the demo applica- tions. You can also choose to make the the wizard and applications visible only to you or enable all the computer users to see it. 4.3.3. Confirm Installation Stage This is the final stage before the installation. Selecting Next will start the installation. Selecting Cancel will abort. 4.3.4. Installation Complete Stage The PresenterMouse wizard and applications were successfully installed. Selecting Close will end the Installation wizard. 4.4. Uninstall Use Add and Remove Programs from the Control Panel to remove PresenterMouse and its demo applications. 5. ACTIVATION Once installed PresenterMouse can be activated by choosing the PresenterMouse ap- plication from the Start menu. The following screen will appear: 5.1. Choosing and Setting the Camera Pressing Next in the About page will take users to the camera selection and setting page. 5.2. Choosing the Camera Clicking on Choose Camera will provide a list of all installed cameras in the com- puter. Choose the one you wish to use. 5.2.1. Setting the Video Format and Properties Each camera has its own Video Format and Video Properties wizards. Selecting one of them will open the proper window. In most of our tests we used a 320x240 color im- age format with automatic camera gain control. It was our experience that using the default camera parameters gave us acceptable results. 5.3. Positioning the Camera In order for PresenterMouse to work well, the camera needs to see the entire com- puter screen. To best position the camera, follow the stages listed on the Camera Posi- tion page. 5.4. Calibration In most cases the camera will display more than the computer screen. In order to im- prove PresenterMouse performance and to position the mouse correctly on the screen, it is necessary to specify the screen boundary in the camera image. To do so, follow the instruction on the Calibration page. 5.5. Finalizing the Wizard Pressing Start on the final wizard page will start the PresenterMouse application. Fol- low the instructions on this page to fine-tune PresenterMouse performance. 5.6. Enabling and Disabling PresenterMouse Ctrl+Shift+E and Ctrl+Shift+D can be used to enable and disable PresenterMouse activities. It does not stop the application from running, but only causes the cursor (temporarily) from moving to the new LASER point on screen. 5.7. Setting the Sensitivity After pressing Start on the final wizard page, an icon will be placed on the Task Bar. Right-clicking on it will open the application menu. In order to set the proper sensitiv- ity we recommend Enabling the application and testing it with a LASER-pointer. In case it does not function properly (the LASER's red point is not recognized), Disable the application, increase the sensitivity and Enable it again. Repeat these stages until the application is working properly. 6. DEMO APPLICATIONS The puzzle demo application will demonstrate the PresenterMouse system as it was used together with the Logitech® Cordless Presenter. The application receives an im- age and partitions it into rectangular pieces (the user can choose the number of pieces). The user is required to rearrange the pieces by dragging and dropping them into their original place in the image. Beside presentations and the puzzle demo application, PresenterMouse can be used to handle any Windows®-based application. For example in the Virtue®Optimizer applica- tion, PresenterMouse can perform 3D object manipulations such as rotate, translate and scale.