Normal User Interface by 7doJ10


									                         Presentation Mouse
                                CE Senior Design Project #2

Name: Ryan Martin, John Bidwell
Course: CE Design Projects 1 – 0306 654 01
Submit Date: September 4th 2008


Operational Objectives…………………………………………...…………3

Similar Products…………………………………………………………….4

Normal User Interface………………………………………………………5

Hardware Component Overview..…………………………………………..7

Software Component Overview..…………………………………………...8

Expected Cost..…………………………………………………………….. 9

Performance Analysis………………………………………………………10


Project Timeline…..………………………………………………………....12

Team Member Information……………………………………....………....13

Operational Objectives
        The proposed design is a wireless remote computer input device and a corresponding
software system. The objective of the system is to give the user the ability to perform standard
mouse input operations using a wirelessly connected remote controller. It will allow a user to
wirelessly perform the functions of a normal computer mouse using a laser as an indicator for
mouse position and wireless transmissions to indicate left and right click mouse clicks. Unlike a
conventional computer mouse, the remote controller does not require a flat surface to operate.
The system frees the user from the podium by allowing mouse actions to be controlled directly
using the wireless remote controller. Typical presentation remote controllers are not capable of
controlling mouse inputs which makes the proposed system so versatile.

Similar Products
       There are similarities in the proposed system to products already introduced to the
consumer market. However, during our investigation no products were discovered that take
advantage of image processing technology for allowing the user to control mouse functions

        Our investigation has revealed that most of the wireless presentation remotes on the
market use either a built in directional pad or joystick to control mouse cursor position and
movement. Information gathered from consumer reviews of similar products shows that
directional pads and joysticks are often awkward and difficult to use. Reviews also suggest that
these mouse cursor manipulation devices are often times inaccurate and slow to respond.

        By taking advantage of image processing technology, the desired outcome of the project
is to provide a quicker and more accurate way for a user to perform mouse functions wirelessly.

       Included in Appendix A are specification sheets of currently released wireless
presentation pointer devices. More mouse cursor manipulation devices were found during
proposal research however, only devices deemed most relevant to the proposed system were
attached. Details on other similar devices can be provided upon request.

Normal User Interface

Diagram 1. Project overview.

         Shown above is a diagram of the proposed system. After initial setup of the system, the
user will only be interacting with the wireless remote and Windows based PC components. After
initial setup of the Projector and Camera components, the user will not need to interact with them
except for making minor adjustments to projection or capture fields.

        Once the system is setup the user will interact with the Windows based PC using the
wireless remote controller. By pointing the laser at the projected display output and squeezing
the laser trigger button located on the wireless remote controller, the user will be able to control
the location of the mouse cursor. Pointing the laser at a specific point on the projected display
output should move the mouse cursor to that respective position. The mouse cursor position
change should be reflected on the projected display output.

        Moving the laser over the projected display output while the laser trigger button is
squeezed will alter the position of the mouse cursor displayed on the projection. However, when
the laser trigger is released the system will not alter the position of the mouse cursor. Pointing
the laser outside of the projected display area, while squeezing the laser trigger, will have no
effect on the current mouse cursor position. Only when the laser is pointed within the projected
display area, while squeezing the laser trigger, will the mouse cursor move.

        Pressing the left button located on the wireless remote controller will send to the
Windows based PC a command which will simulate a left mouse click. Pressing the right button
located on the wireless remote controller will send to the Windows based PC a command which
will simulate a right mouse click. Double pressing the left mouse button located on the wireless
remote controller will simulate a double left mouse click.

         The left and right click buttons located on the wireless remote controller can be pressed
anytime while the system is active. Anytime a button is pressed while the system is active, a
mouse click simulation command is sent to the Windows based PC regardless of whether the
laser trigger button is squeezed or released.

        System setup is discussed in more detail in the Hardware Component Overview section
of this document.

Hardware Component Overview
       Depicted in Diagram 1. are the proposed project components. As shown, the proposed
system will consist of 4 main hardware components. Below is a description of each component
and the role it plays in the system.

          1. Windows based PC – This component will be used for processing the images
             received by the Camera. Initial setup of the system would require that the DSP
             and mouse movement software be executed. Due to the power required by
             filtering techniques, this component must have a processor of 1Ghz or greater.

          2. Projector – This component will be connected to the Windows based PC. It will
             be used to project display output of the Windows based PC onto a projector
             screen. The projector screen can then be used as a reference for re positioning the
             mouse cursor. Initial setup of the system would require that the projected image
             be displayed evenly and fully on a flat white surface or projector screen to ensure
             maximum accuracy in calculating desired mouse cursor position. After initial
             setup of this component, the user should not alter the position except for making
             minor fine tuning adjustments to the projection area.

          3. Camera – This component will be connected to the Windows based PC. It will be
             used to capture images of the projector screen which will be sent to the Windows
             based PC for image processing. The camera should be placed nearly
             perpendicular to the projector screen to ensure that the desired mouse position can
             be accurately calculated from the captured images. Initial setup of the system
             would require that the camera be positioned so that all of the edges of the
             projected display output can be seen by the camera. After initial set up that the
             camera position should remain untouched except for making fine tuning
             adjustments to the image capture area.

          4. Wireless Remote – This component is connected to the Windows based PC via
             wireless a wireless Bluetooth connection. The purpose of this component is to
             send left and right click mouse commands to the system and to indicate, using the
             attached laser, the desired location of the mouse cursor. The wireless remote will
             consist of a laser pointer for indicating mouse position and a set of buttons which
             will be used for executing left and right click mouse functions. These components
             will be placed inside an ergonomic package with a trigger like switch for
             operating the laser and separate function buttons. It should be noted that laser, and
             function buttons on the wireless remote be independently operated. Until an
             independent study is done to confirm the possibility of shared power, the wireless
             function buttons and the laser will both had independent power supplies. The
             independent power will come in the form of small and compact lithium ion

Software Component Overview

        For the proposed system, DSP and mouse movement simulation software must be written
to perform the required task of calculating desired mouse position and executing mouse

        The DSP analysis software component will obtain streaming image data from the camera
and will process it to locate a laser dot within the streaming image data. As noted previously, the
user must set up the camera so that each of the corners of the projected display output are visible
to the camera. The DSP analysis will use a brightness threshold to detect the location of the laser
dot in the streaming images. If a bright area above the set threshold value is detected the position
will be marked and handled by the mouse movement simulation software. However, if no bright
areas above the set threshold value are detected, then no action is taken.

        The mouse driver will take in relative coordinates and use them to move the mouse to the
corresponding position. This interaction will not be a smooth motion but rather an instant
movement to a location, similar to how the mouse interacts on a tablet PC. This system will be
installed on the host PC before the system can be used. The implementation of a smooth mouse
movement transition will investigated in the future.

        Since the proposed design will be using a Windows based PC, mouse movement
simulation can be performed using the Win32 API through Java, C#, or C++. It should be noted
that investigation of this issue has revealed that the Java libraries contain much more mouse
simulation functionality than the C# and C++ libraries. However, in order to make the image
processing software and mouse driver emulator software more easily integrate able, C# will be
used for both software packages..

Diagram 2. Software block diagram.

Expected Costs

   Red Laser Module - Focusable Dot (3.5V~4.5V 16mm 5mW)
       o $4.30

   MoGo X54 Pro 4 Buttons Bluetooth Wireless 800 dpi Mouse
      o $73.99

   ViewSonic PJ503D 800 x 600 DLP Projector 1,500 ANSI lumens
       o $489.99 - $100.00 MIR = $389.99

   Logitech QuickCam Communicate Delux
       o $84.99

    Total Cost= $553.27

Performance Analysis

               Graph 1. Filtering Delay vs. CPU speed

        One of the initial concerns of the project was the delay due to frame processing of the PC.
The delay between when a frame is sent by the Quickcam and when the connected PC detects the
location of the laser dot and moves the mouse to the position. It was already determined through
independent investigation that the delay of moving the mouse to a position was insignificant but
the processing delay remained a concern. The prupose of this analysis was to determine the
severity of the delay due to processing and to determine the minimum recommended processing
power required for the Wireless Presentation Mouse system.

        For analysis, the webcam which was used was the Quickcam Communicator Deluxe
which was manufactured by Logitech. The PC which performed the processing on video input
was used was the Everex SA2503T laptop computer. The SA2503T is equiped with a 1.6Ghz
intel Core Duo processor. However, for the purposes of this experiment the processing power of
the CPU was reduced to a range of 400 Mhz and 800 Mhz. It should also be noted, that that the
experiment was performed using only a YCrCb filter for finding the green laser dot. No blob
detection or location finding algorithms were used during this experiment as these portions of the
project were not yet completed.

        The data shown in Graph 1. shows performance measurements, or delay in milliseconds,
of the filtering software at specific cpu speeds. Analysing the results has shown that using an
800Mhz processor yielded a filter processing delay of approximately 27 milliseconds. It was
determined that in order to provide enough processing power to run a minimum of 2 windows
applications, as well as the filtering software simultaneously, a PC with a minimum CPU speed
of 1 Ghz would be necessary.

        The largest foreseeable difficulty in implementing the proposed design is the DSP. It was
mentioned that there may be difficulty in determining laser position when colors matching that of
the laser are used on the Windows based PC.

        This issue was investigated in some detail with Dr. Bowman of the Electrical Engineering
department of RIT. Although there is still a concern, Dr. Bowman believes that due to the
brightness and intensity of the laser light, that there should very little difficulty in determining
laser position when the same color is used on the Windows based PC. It was decided that for the
time being, using a green colored laser pointer of normal intensity will be acceptable for use with
the proposed design.

       If trouble occurs with this decision in the future, brighter, more intense lasers or the IR
spectrum may be investigated as possible solutions.

        Due to the design limitations, the brightness of the laser is a part of the detection criteria.
As a result the system will be sensitive to the ambient light in the environment. The system is
expected to work in most normal presentation situations, with the exception of barely being able
to see the projected image or other extreme direct light situations.

        Along with sensitivity to light, this application may be somewhat processing intensive.
Some initial analysis has been performed on an alpha version of the application allowing some
insight to the resource requirements of the hardware needs of the application (see Appendix).
The final design will work with both a power point presentation and a web browser or word
processor active with little effect on the performance of the computer.

        Another minor foreseeable difficulty is wireless remote. The wireless remote may be
difficult to construct to a comfortable form factor with the selected hardware components.

        It is suspected that difficulties may occur when attempting to implement the mouse
cursor manipulation software. The proposed design for the mouse cursor manipulation software
is to use available Windows libraries through Java, C++ or C# to control mouse cursor
movement. However, neither team member has any previous experience in this area of software
programming. It was therefore decided that this concern is significant enough to be investigated
further in the independent investigations.

        Refer to the Independent Investigation section of this document for more detail on
investigation topics and approaches.

Project Timeline

                                                                                                                                               Sep 2008                                                                                                                                           Oct 2008
ID               Task Name                     Start       Finish      Duration
                                                                                  1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16    17   18   19   20   21   22   23   24   25   26   27   28   29   30   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15     16       17   18   19   20   21   22   23   24   25   26   27   28   29   30   31

1 Laser Interpretation                        9/1/2008    10/5/2008      5w

2      Learn AForge                           9/1/2008     9/1/2008       1d
       Interface AForge with Streaming
3                                             9/2/2008     9/5/2008       4d
4      Filter Projector Illumination          9/6/2008     9/7/2008       2d

5      Process Laser Dot Location            9/14/2008    9/24/2008     1w 4d

6      Apply Filters for Image Skewing       9/25/2008    10/5/2008     1w 4d

7 Mouse Driver Emulation                      9/1/2008     9/2/2008       2d

8      Learn Windows.h                        9/1/2008     9/1/2008       1d

9      Develop Mouse Control Methods          9/2/2008     9/2/2008       1d

10 GUI Development                            9/4/2008    9/11/2008     1w 1d

11     Design GUI                             9/4/2008     9/7/2008       4d

12     Develop Moveable Buttons               9/8/2008    9/11/2008       4d

13 Handheld Development                      9/12/2008    9/30/2008     2w 5d

14     Select Laser Diode                    9/12/2008    9/13/2008       2d

15     Modify Bluetooth Mouse                9/14/2008    9/16/2008       3d

16     Build Enclosure                       9/17/2008    9/30/2008       2w

17 System Integration                        10/6/2008    10/25/2008    2w 6d
       Integrate Laser Interpretation with
18                                           10/6/2008    10/8/2008       3d
       Mouse Driver Emulator
       Integrate Laser Interpretation with
19                                           10/9/2008    10/17/2008    1w 2d
20     Testing                               10/18/2008   10/22/2008      5d
       Develop Utility into Background
21                                           10/23/2008   10/25/2008      3d
22 Website and Presentation                   9/1/2008    10/30/2008    8w 4d

23     Website Design and Development         9/1/2008    10/15/2008    6w 3d

24     Poster Design                         10/26/2008   10/28/2008      3d

25     Project Write-up                      10/26/2008   10/30/2008      5d

Team Member Information

   Ryan Martin
       o Phone: 774-272-0908
       o E-Mail:
       o Address: 193 Farnum Lane, Rochester NY 14623

   John Bidwell
       o Phone: 585-978-2939
       o E-Mail:
       o Address: 202 Bennington Hills Court, Rochester NY 14623


To top