Learning Center
Plans & pricing Sign in
Sign Out
Get this document free

Automatic Color Balls Sorter


									Automatic Colored Ball Sorter

                  Team members:
                         Ka Fai Ng
                         Woo Jin Park
                         Mihir Daftari
                  Date: September 7, 2006
             Contact Information:

     Name         Ka Fai Ng
Mailing Address   778 Kimball Dr. Rochester NY
Phone Number      718 986 6868

     Name         Woo Jin Park
Mailing Address   128 Beech Street
                  Maywood, NJ 07607
Phone Number      551 265 2073

     Name         Mihir Daftari
Mailing Address   22 Fairwood Drive, Apt #1k
                  Rochester, NY 14623
Phone Number      585 200 1564
The objective of this project is to design and implement a sorting device that classifies
balls based on their color. The system will dispense a ball from a container on to a sloped
surface. The sloped surface consists of four servomotors that are used to steer the ball to
appropriate bin. The first servomotor will stop the ball under a camera. The camera will
scan the ball and the information will be processed using HCS12. Once the information
is processed, the HCS12 will signal the other servos to turn to „open‟ or „closed‟ positions
and open the first servo to let the ball go to the appropriate bin. A graphical user
interface will be provided to the user to control the system and communicate the system
status. The system will start in a paused state and the user will need to start the system
using the „start‟ override.

Preliminary System Diagram:

                  Pivot                                           Moving
                                  Servomotor                      direction
                                    G                                          Stepper motor

            bin           D

                              R             B



                                                    HC12                      Port B – Step Motor
                                  Port P        microcontroller



           Figure 1: Top View of the System layout (figure not drawn to scale)

User Interface
The user interface of the system will consist of a user-friendly java program that will run
on a computer (or laptop). The java program will be the only communication interface by
which the user can control the system and its settings. The HC12 microcontroller will be
connected with the computer via the serial/RS-232 port. As for java programming, the
"javax.comm" package will be used in order to initiate the communication ports, receive
and transfer data between the microcontroller and the program.

The java swing package will be used to create the graphical user interface, which will be
the primary controller of the whole user interface protocol. The user will select the
options from the graphic user interface using keyboard or mouse. The program will
display the current status of the system to the user. The following figure shows how the
GUI will look like.

Graphical User Interface

                   Figure 2: Prospective user interface for the system

The following table summarizes all the overrides that will be provided to the user along
with proposed bit codes:
    Command               Internal                   Description                     Error Causing
                         Encoding                                                       Condition
   Start System          10000000       The system starts up in a paused           If the camera
                                        state when powered up. This                does not respond
                                        command is used to start the actual        within two
                                        dispensing and sorting of balls.           seconds, it will
                                        Internally this command                    cause an error.
                                        synchronizes the camera and the
                                        HCS12. It also gives about two
                                        seconds to the camera to warm-up.
   Stop System           01000000       This command will stop the system
                                        immediately. It will stop the
                                        dispenser from dispensing more balls
                                        and clear all the balls on the track by
                                        „opening‟ up all the servomotors. It
                                        also puts the camera in sleep mode.
                                        A „Start System‟ command will be
                                        needed to restart the sorting process.
   Pause System          00100000       This command will pause the system
                                        after the last ball dispensed is sorted.
  Resume System          00010000       This will resume the ball dispensing
                                        and sorting process.
    Clear balls          00001000       This command will pause the ball
                                        dispenser and then „open‟ all the
                                        servomotors to clear all the balls on
                                        the track. A „Resume System‟
                                        command will be needed to start the
                                        system again.

         Table 1: Description of user commands and their proposed bit-patterns

Stepper Motor Specification
The stepper motor will control the ball dispenser portion of the project. The stepper
motor is used for this application because of its precise rotor position. The operation of
the stepper motor will consist of turning 90 to push one ball on to the sloped surface.

Since the rotor position of the stepper motor is very precise and accurate, it is estimated
the offset for each step is very small, assuming that the ball is light and it would not
require a lot of torque from the stepper motor.

Ball Dispense Control
The rotating step control will be controlled using ULN2064B 1.5 Quad Darlington



                     Figure 3: Mechanism to dispense one ball to tube
                     (top down view – ball dispensing tube not shown)

The idea of using two motor arms instead of four because of timing issue involve with the rest of
the system. Furthermore, with only two motor arms, it provides enough time for the color
recognition and servo positioning to be completed before the next ball dispenses.

                   V-shape dent

                      Figure 4: Ball dispenser mechanism – side view

**Note: The purpose of the V-shape dent is to hold the ball in place without bouncing
away from the motor arm because of its spherical surface.
                                                  Stepper Motor


     Figure 5: Infinitely different directions the ball can go when it has been pushed

*Note that the trapezoidal shape shown in the figures is actually a sloped platform with boundary
on the sides. The purpose of such configuration is to compensate the possibility that the ball
might move toward a different direction other than straight ahead. With the boundary on the side,
the ball is guaranteed to go into the tube regardless of the direction it was pushed.

                               Step Degree Step times
                               45             12
                               90             24
                               180            48
                               360            96
            Table 2: Stepper motor step degree and step times for half stepping

0 Step – Initial state (Ball B is on top of ball A)

                                                   Stepper Motor


24 Steps – Turn Counterclockwise 90 (Ball A is pushed onto the tube and Ball B falls)
                                                      Stepper Motor



Another 24 Step – Turn Counterclockwise 90 (Ball B remains stationary)
*Note that depend on the amount of balls in the dispensing tube, the torque that requires
to push the ball will be different, hence the ball may pushed off in a different direction.

                                                    Stepper Motor


Another 24 Step – Turn Counterclockwise 90 (Ball B is pushed onto the tube)

                                                      Stepper Motor



                           Figure 6: Steps for the stepper motor
Camera and Color Detection Algorithm

The C328-7640 camera module (Manufacturer: Inc) can perform as a
video camera or a JPEG compressed still camera. It can be attached to a wireless or PDA

                     Figure 7: System block diagram of the camera

The C328-7640 is a small, low cost, low powered solution for high-resolution image
capture making it a perfect choice for an embedded application like this project. The
following shows the extremely small size of the camera:

               Figure 8: C328-7640 board layout and serial interface pin

The camera does operate at 3.3 V and thus needs a RS-232 transceiver setup as a
communication interface.
                                 Figure 9: System block diagram

The RS-232 transceiver chips are available from Maxim and many other semiconductor

The camera has a lot of plug and play features like built-in color conversion circuits for 4
gray/16 gray/256 gray/12-bit RGB/16-bit RGB images, RS-232 interface with a wide
range of baud rates and auto detection of baud rates. One of the best features of the
camera is the on-board EEPROM that provides a command-based interface to external
host via RS-232.
The command set for the module has a total of 11 commands for interfacing to host as

  Command          ID         Parameter1       Parameter2          Parameter3           Parameter4
     Initial     AA01h            00h           Color Type       RAW Resolution            JPEG
                                                                 (Still image only)      Resolution
  Get Picture     AA04h       Picture Type         00h                  00h                 00h
   Snapshot       AA05h         Snapshot       Skip Frame          Skip Frame               00h
                                  Type          Low Byte            High Byte
  Set Package     AA06h            08h        Package Size      Package Size High           00h
      Size                                      Low Byte               Byte
  Set Baudrate    AA07h       1st Divider      2nd Divider              00h                 00h
      Reset       AA08h       Reset Type           00h                  00h                xxh*
   Power Off      AA09h           00h              00h                  00h                 00h
      Data        AA0Ah       Data Type       Length Byte 0       Length Byte 1        Length Byte 2
     SYNC         AA0Dh           00h              00h                  00h                 00h
      ACK         AA0Eh      Command ID       ACK counter        00h / Package ID      00h / Package
                                                                      Byte 0             ID Byte 1
       NAK          AA0Fh           00h         NAK counter       Error Number              00h
       Light        AA13h        Frequency           00h                00h                 00h
    Frequency                      Type
* If the parameter is 0xFF, the command is a special Reset command and the firmware responds to it
 The camera has a very simple protocol to use the above commands. Complete detail
about the parameters for the command and the protocol is given in the user manual for
the camera. The following diagram shows the protocol to take an uncompressed picture
using the camera:

            Figure 10: Command protocol to take an uncompressed snapshot

The camera has been bought and interfaced with the PC. Using serial communications, a
picture has also been taken with the camera.

The camera and the color detection algorithm selected perfectly complement each other.
The camera is able to give out the image in RGB format and the algorithm uses the same
format for detecting color. The details of the camera are attached in the appendix.
The following is the description of the algorithm:
Suppose that each pixel is a vector of bits. The length of the vector will be 1 byte or 16
Each pixel = [R G B]
RGB[5, 6, 5] will be the vector of bits retrieve from the camera.
Red represents by the first 5 bits of the vector.
Green represents by the next 6 bits.
Blue represents by the last 5 bits.

Each reference color combination is another 8 bits vector shown below:
Red reference color = [255 0 0]
Green reference color = [0 255 0]
Blue reference color = [0 0 255]

Here is where we can have different "tolerances/thresholds" for different colors.
The Euclidian distance between those 2 vectors is:

   D      R ref    Ri   Gref  Gi   Bref  Bi 
                         2                 2                2

Rref is the reference code for the color RED.
Gref is the reference code for the color GREEN.
Bref is the reference code for the color BLUE.
Ri, Gi, and Bi are the retrieved vector from the camera.

Algorithm Pseudo code:
for each pixel
 1. compute the distance between that pixel and the reference color (D)
 2. if D<Threshold
           current pixel is accepted
           current pixel is NOT accepted
Threshold is a constant value and has to be set manually according to the "tolerance" in
the algorithm. A high value of threshold means that a less “clear” green pixel will be
considered as green, whereas a lower value of threshold will cause the range of
determining the color to decrease, and hence a stricter rule is imposed on the algorithm.

Algorithm State Diagram
In this case, the algorithm is verifying the color against green reference color.

                                           Read a pixel
                                          Pixel = [R G B]

                                        Calculate Distance
                                         from “Threshold”
                                       D = sqrt((R-0)^2 + (G-
                                         255)^2 + (B-0)^2)

                                        Is D < “Threshold”


                                        The color is Green.
                                       Increment counter for

                                            More Pixels?

                                          Ready to take
                                            the next

                 Figure 11: State diagram for Green color detection

Futaba S3003
Supply Voltage: 5V
Angle of movement: 180º


                  Figure 12: Servomotor’s method of sorting out the ball

The servomotor turns to the different position according to the pulse widths sends from
the HC12 microprocessor. For example, if a red ball is recognized, the servomotor will
move to position 1 in order for the ball to slide down to its corresponding bin.

The servomotor has a lot of offset when it pivots from on position to another
instantaneously. Hence, it is predicted that by given enough time for it to settle on the
position, the offset should be minimal.

Overall System Interconnection
                                                 5V Power

         Servo Motor X 4

                                                     RS 232
              Servo Input

                                                                            8       Step Motor
                                        Port P                  Port B
                                                                         Output           1


                                                                                   Stepper Motor

                                                     RS 232

                                                                                   5V Power

                                           User Interface (Laptop)

                           Figure 13: Preliminary system interconnection

Performance Analysis (Estimation)
The estimation below is approximated using their each component‟s maximum value.
 The approximate time for the ball to dispense from the container to the sensor is
   approximately 1 second at maximum.
 The color detecting and recognition algorithm, RS-232 transmission with baud rate of
   115.2 kbps and HC12 processing time is approximately 300 ms.
 The length of the tube is about 8 cm and approximately about 3 to 4 seconds to go to
   the end of the tube.
 The maximum time for the servomotor to pivot from one position to another is at
  maximum of 1 second.

Operation time:
Hence, the total estimated time from the ball dispensing (1 second), color recognition and
processing (300 ms), time to get to the end of the tube is 4 seconds maximum and
servomotor pivoting position for sorting is 1 second maximum. Hence, the total time for a
ball to dispense to its corresponding bin is about 6.3 seconds maximum.

Testing Strategy:
There will be independent testing strategies implemented for testing each module.
The color detection will be tested under different light conditions. It should work under
all kinds of „reasonable‟ lighting conditions (it will not work if it is absolutely dark).
The ball dispensing mechanism will be tested with different number of balls in the
container (or no ball in the container). After integrating the whole system, the user
overrides and the system status indication will be tested.
The system will start in a paused state and will need a „start‟ override to start. Pausing
the system in the middle of color detection process and then resuming should yield proper
result. If the system is „stopped‟ after releasing a ball, the system will stop only after
sorting that ball.

Budget Estimation:
                                   Approximate       Price Paid /
      Equipment         Quantity                                        Vendor/Source
                                      Cost          Expected Cost
  Custom HCS12                                                      Discount from RIT
                           1            $175              $0
  Microcontroller                                                   CE Dept.
  Futaba S3003 Servo
                           4          $10 each            $0        RIT CE Dept.
  Stepper Motor            1            $10               $0        RIT CE Dept.
                                                                    Borrowed from
  Laptop                   1              -               $0
                                                                    team member
  C328-7640 JPEG
  Compression VGA          1            $50              $47
  Camera Module
  RS-232 Transceiver       1             $5               $0
  (tube, balls, bins,      -              -              $50        -
                                        $280          ($32.3 per

-   Camera and microcontroller interfacing and communication at a high baud rate
    could be difficult. The camera is „supposed‟ to talk at a very high baud rate and
    the HC12 can also be fast. A „worst‟ case speed is proposed for the project, but
    the actual speed should be considerably faster. The camera IS able to talk to the
    computer at 9600 baud. So the expectation of it working at least at that speed is
-   Ball dispenser jamming problem could jam the balls in the dispenser itself. The
    plan is to use balls of appropriate weight and sort this problem by trial and error

Camera Detail:
Camera – C238-7640 (Manufacturer:, Inc.)
Supply Voltage: Range from +3.0 to +3.6 V
Resolution: 160x120 pixels
Built-in color conversion circuits for 2-bit, 4-bit, 8-bit gray scale, 12-bit and 16-bit RGB
and JEPG CODEC compression capability
Include On-board EEPROM

Stepper Motor Detail
Airpax L82701-P1 Uni-polor permanent magnet stepper motor
Supply Votlage: 5 V
Full Step degree: 7.5

Independent Research Project Papers of all three team members attached

To top