Automatic Colored Ball Sorter
Ka Fai Ng
Woo Jin Park
Date: September 7, 2006
Name Ka Fai Ng
Mailing Address 778 Kimball Dr. Rochester NY
Phone Number 718 986 6868
Name Woo Jin Park
Mailing Address 128 Beech Street
Maywood, NJ 07607
Phone Number 551 265 2073
Name Mihir Daftari
Mailing Address 22 Fairwood Drive, Apt #1k
Rochester, NY 14623
Phone Number 585 200 1564
The objective of this project is to design and implement a sorting device that classifies
balls based on their color. The system will dispense a ball from a container on to a sloped
surface. The sloped surface consists of four servomotors that are used to steer the ball to
appropriate bin. The first servomotor will stop the ball under a camera. The camera will
scan the ball and the information will be processed using HCS12. Once the information
is processed, the HCS12 will signal the other servos to turn to „open‟ or „closed‟ positions
and open the first servo to let the ball go to the appropriate bin. A graphical user
interface will be provided to the user to control the system and communicate the system
status. The system will start in a paused state and the user will need to start the system
using the „start‟ override.
Preliminary System Diagram:
G Stepper motor
HC12 Port B – Step Motor
Port P microcontroller
Figure 1: Top View of the System layout (figure not drawn to scale)
The user interface of the system will consist of a user-friendly java program that will run
on a computer (or laptop). The java program will be the only communication interface by
which the user can control the system and its settings. The HC12 microcontroller will be
connected with the computer via the serial/RS-232 port. As for java programming, the
"javax.comm" package will be used in order to initiate the communication ports, receive
and transfer data between the microcontroller and the program.
The java swing package will be used to create the graphical user interface, which will be
the primary controller of the whole user interface protocol. The user will select the
options from the graphic user interface using keyboard or mouse. The program will
display the current status of the system to the user. The following figure shows how the
GUI will look like.
Graphical User Interface
Figure 2: Prospective user interface for the system
The following table summarizes all the overrides that will be provided to the user along
with proposed bit codes:
Command Internal Description Error Causing
Start System 10000000 The system starts up in a paused If the camera
state when powered up. This does not respond
command is used to start the actual within two
dispensing and sorting of balls. seconds, it will
Internally this command cause an error.
synchronizes the camera and the
HCS12. It also gives about two
seconds to the camera to warm-up.
Stop System 01000000 This command will stop the system
immediately. It will stop the
dispenser from dispensing more balls
and clear all the balls on the track by
„opening‟ up all the servomotors. It
also puts the camera in sleep mode.
A „Start System‟ command will be
needed to restart the sorting process.
Pause System 00100000 This command will pause the system
after the last ball dispensed is sorted.
Resume System 00010000 This will resume the ball dispensing
and sorting process.
Clear balls 00001000 This command will pause the ball
dispenser and then „open‟ all the
servomotors to clear all the balls on
the track. A „Resume System‟
command will be needed to start the
Table 1: Description of user commands and their proposed bit-patterns
Stepper Motor Specification
The stepper motor will control the ball dispenser portion of the project. The stepper
motor is used for this application because of its precise rotor position. The operation of
the stepper motor will consist of turning 90 to push one ball on to the sloped surface.
Since the rotor position of the stepper motor is very precise and accurate, it is estimated
the offset for each step is very small, assuming that the ball is light and it would not
require a lot of torque from the stepper motor.
Ball Dispense Control
The rotating step control will be controlled using ULN2064B 1.5 Quad Darlington
Figure 3: Mechanism to dispense one ball to tube
(top down view – ball dispensing tube not shown)
The idea of using two motor arms instead of four because of timing issue involve with the rest of
the system. Furthermore, with only two motor arms, it provides enough time for the color
recognition and servo positioning to be completed before the next ball dispenses.
Figure 4: Ball dispenser mechanism – side view
**Note: The purpose of the V-shape dent is to hold the ball in place without bouncing
away from the motor arm because of its spherical surface.
Figure 5: Infinitely different directions the ball can go when it has been pushed
*Note that the trapezoidal shape shown in the figures is actually a sloped platform with boundary
on the sides. The purpose of such configuration is to compensate the possibility that the ball
might move toward a different direction other than straight ahead. With the boundary on the side,
the ball is guaranteed to go into the tube regardless of the direction it was pushed.
Step Degree Step times
Table 2: Stepper motor step degree and step times for half stepping
0 Step – Initial state (Ball B is on top of ball A)
24 Steps – Turn Counterclockwise 90 (Ball A is pushed onto the tube and Ball B falls)
Another 24 Step – Turn Counterclockwise 90 (Ball B remains stationary)
*Note that depend on the amount of balls in the dispensing tube, the torque that requires
to push the ball will be different, hence the ball may pushed off in a different direction.
Another 24 Step – Turn Counterclockwise 90 (Ball B is pushed onto the tube)
Figure 6: Steps for the stepper motor
Camera and Color Detection Algorithm
The C328-7640 camera module (Manufacturer: 123Electronics.com Inc) can perform as a
video camera or a JPEG compressed still camera. It can be attached to a wireless or PDA
Figure 7: System block diagram of the camera
The C328-7640 is a small, low cost, low powered solution for high-resolution image
capture making it a perfect choice for an embedded application like this project. The
following shows the extremely small size of the camera:
Figure 8: C328-7640 board layout and serial interface pin
The camera does operate at 3.3 V and thus needs a RS-232 transceiver setup as a
Figure 9: System block diagram
The RS-232 transceiver chips are available from Maxim and many other semiconductor
The camera has a lot of plug and play features like built-in color conversion circuits for 4
gray/16 gray/256 gray/12-bit RGB/16-bit RGB images, RS-232 interface with a wide
range of baud rates and auto detection of baud rates. One of the best features of the
camera is the on-board EEPROM that provides a command-based interface to external
host via RS-232.
The command set for the module has a total of 11 commands for interfacing to host as
Command ID Parameter1 Parameter2 Parameter3 Parameter4
Initial AA01h 00h Color Type RAW Resolution JPEG
(Still image only) Resolution
Get Picture AA04h Picture Type 00h 00h 00h
Snapshot AA05h Snapshot Skip Frame Skip Frame 00h
Type Low Byte High Byte
Set Package AA06h 08h Package Size Package Size High 00h
Size Low Byte Byte
Set Baudrate AA07h 1st Divider 2nd Divider 00h 00h
Reset AA08h Reset Type 00h 00h xxh*
Power Off AA09h 00h 00h 00h 00h
Data AA0Ah Data Type Length Byte 0 Length Byte 1 Length Byte 2
SYNC AA0Dh 00h 00h 00h 00h
ACK AA0Eh Command ID ACK counter 00h / Package ID 00h / Package
Byte 0 ID Byte 1
NAK AA0Fh 00h NAK counter Error Number 00h
Light AA13h Frequency 00h 00h 00h
* If the parameter is 0xFF, the command is a special Reset command and the firmware responds to it
The camera has a very simple protocol to use the above commands. Complete detail
about the parameters for the command and the protocol is given in the user manual for
the camera. The following diagram shows the protocol to take an uncompressed picture
using the camera:
Figure 10: Command protocol to take an uncompressed snapshot
The camera has been bought and interfaced with the PC. Using serial communications, a
picture has also been taken with the camera.
The camera and the color detection algorithm selected perfectly complement each other.
The camera is able to give out the image in RGB format and the algorithm uses the same
format for detecting color. The details of the camera are attached in the appendix.
The following is the description of the algorithm:
Suppose that each pixel is a vector of bits. The length of the vector will be 1 byte or 16
Each pixel = [R G B]
RGB[5, 6, 5] will be the vector of bits retrieve from the camera.
Red represents by the first 5 bits of the vector.
Green represents by the next 6 bits.
Blue represents by the last 5 bits.
Each reference color combination is another 8 bits vector shown below:
Red reference color = [255 0 0]
Green reference color = [0 255 0]
Blue reference color = [0 0 255]
Here is where we can have different "tolerances/thresholds" for different colors.
The Euclidian distance between those 2 vectors is:
D R ref Ri Gref Gi Bref Bi
2 2 2
Rref is the reference code for the color RED.
Gref is the reference code for the color GREEN.
Bref is the reference code for the color BLUE.
Ri, Gi, and Bi are the retrieved vector from the camera.
Algorithm Pseudo code:
for each pixel
1. compute the distance between that pixel and the reference color (D)
2. if D<Threshold
current pixel is accepted
current pixel is NOT accepted
Threshold is a constant value and has to be set manually according to the "tolerance" in
the algorithm. A high value of threshold means that a less “clear” green pixel will be
considered as green, whereas a lower value of threshold will cause the range of
determining the color to decrease, and hence a stricter rule is imposed on the algorithm.
Algorithm State Diagram
In this case, the algorithm is verifying the color against green reference color.
Read a pixel
Pixel = [R G B]
D = sqrt((R-0)^2 + (G-
255)^2 + (B-0)^2)
Is D < “Threshold”
The color is Green.
Increment counter for
Ready to take
Figure 11: State diagram for Green color detection
Supply Voltage: 5V
Angle of movement: 180º
Figure 12: Servomotor’s method of sorting out the ball
The servomotor turns to the different position according to the pulse widths sends from
the HC12 microprocessor. For example, if a red ball is recognized, the servomotor will
move to position 1 in order for the ball to slide down to its corresponding bin.
The servomotor has a lot of offset when it pivots from on position to another
instantaneously. Hence, it is predicted that by given enough time for it to settle on the
position, the offset should be minimal.
Overall System Interconnection
Servo Motor X 4
8 Step Motor
Port P Port B
User Interface (Laptop)
Figure 13: Preliminary system interconnection
Performance Analysis (Estimation)
The estimation below is approximated using their each component‟s maximum value.
The approximate time for the ball to dispense from the container to the sensor is
approximately 1 second at maximum.
The color detecting and recognition algorithm, RS-232 transmission with baud rate of
115.2 kbps and HC12 processing time is approximately 300 ms.
The length of the tube is about 8 cm and approximately about 3 to 4 seconds to go to
the end of the tube.
The maximum time for the servomotor to pivot from one position to another is at
maximum of 1 second.
Hence, the total estimated time from the ball dispensing (1 second), color recognition and
processing (300 ms), time to get to the end of the tube is 4 seconds maximum and
servomotor pivoting position for sorting is 1 second maximum. Hence, the total time for a
ball to dispense to its corresponding bin is about 6.3 seconds maximum.
There will be independent testing strategies implemented for testing each module.
The color detection will be tested under different light conditions. It should work under
all kinds of „reasonable‟ lighting conditions (it will not work if it is absolutely dark).
The ball dispensing mechanism will be tested with different number of balls in the
container (or no ball in the container). After integrating the whole system, the user
overrides and the system status indication will be tested.
The system will start in a paused state and will need a „start‟ override to start. Pausing
the system in the middle of color detection process and then resuming should yield proper
result. If the system is „stopped‟ after releasing a ball, the system will stop only after
sorting that ball.
Approximate Price Paid /
Equipment Quantity Vendor/Source
Cost Expected Cost
Custom HCS12 Discount from RIT
1 $175 $0
Microcontroller CE Dept.
Futaba S3003 Servo
4 $10 each $0 RIT CE Dept.
Stepper Motor 1 $10 $0 RIT CE Dept.
Laptop 1 - $0
Compression VGA 1 $50 $47
RS-232 Transceiver 1 $5 $0 ic.com
(tube, balls, bins, - - $50 -
$280 ($32.3 per
- Camera and microcontroller interfacing and communication at a high baud rate
could be difficult. The camera is „supposed‟ to talk at a very high baud rate and
the HC12 can also be fast. A „worst‟ case speed is proposed for the project, but
the actual speed should be considerably faster. The camera IS able to talk to the
computer at 9600 baud. So the expectation of it working at least at that speed is
- Ball dispenser jamming problem could jam the balls in the dispenser itself. The
plan is to use balls of appropriate weight and sort this problem by trial and error
Camera – C238-7640 (Manufacturer: Electronics123.com, Inc.)
Supply Voltage: Range from +3.0 to +3.6 V
Resolution: 160x120 pixels
Built-in color conversion circuits for 2-bit, 4-bit, 8-bit gray scale, 12-bit and 16-bit RGB
and JEPG CODEC compression capability
Include On-board EEPROM
Stepper Motor Detail
Airpax L82701-P1 Uni-polor permanent magnet stepper motor
Supply Votlage: 5 V
Full Step degree: 7.5
Independent Research Project Papers of all three team members attached