Test Plan for binutils porting on BF533
Version No. 1.7
Released On june 9, 2004
Author(s) Omkar Murthy
Reviewed By Pradip Shah
Table of Contents
1 Introduction
2 Test Items
3 Features To Be Tested
4 Features Not To Be Tested
5 Approach
o 5.1 Code Inspections
o 5.2 Unit Testing
o 5.3 System Testing
o 5.4 Regression Testing
o 5.5 Acceptance Testing
6 Item Pass/Fail Criteria
7 Suspension Criteria and Resumption Requirements
8 Test Deliverables
9 Testing Tasks
10 Environmental Needs
11 Responsibilities
12 Staffing and Training Needs
13 Schedule
14 Risks and Contingencies
15 Approvals
To Do List
Revision History
1 Introduction
Analog Devices Inc., hereinafter referred to as “ADI”, is a world leader in high
performance signal processing solutions. ADI positions the ADSP-BF53x family, also
known as the Blackfin Processor, as a full-fledged highly integrated system-on-a-chip
(SoC) solution for the next generation of digital communication and connected media
applications.
For the first member of this family – the ADSP BF535 – ADI has completed a port of the
uCLinux embedded operating system. It now intends to make available uCLinux for
other processors in the family, namely the ADSP BF531/2/3 processors.
This work involves three main components:
1. Linux Port, Bootloader and Library support
2. Peripheral device drivers for the Board.
3. Compiler tool chain
The objectives and tasks of the test effort as stated as follow:
Objectives
- Perform tests for binutils ported for on BF533 platform.
Tasks
- Test environment setup
- Regression Test
2 Test Items
The test plan for binutils details the various tests to be executed, test procedure,
schedule and deliverables.
Following tests will be performed:
1. Regression tests are carried out to ensure that the functionality and behavior
of the binutils hasn‟t changed while changes were done in the source code.
3 Features To Be Tested
Following tools will be tested under binutils
All the test cases will be maintained under respective sub directories named same as tool
name like gas, bfd, ld.
Test cases in correspondig subdirectories will be executed using dejagnu.
Tools Test cases and Description
1. gas create subdirectory bfin and include BF533 architecture and
instruction specific testcases.
Following categories of tests will be created.
1. 533 instruction set
Arithmetic
Logical
Input/Output
Interrupt
External Hardware Sync/control
Transfer instructions(conditional, unconditional)
Shift
Rotate
String
Iteration(loop, while..,)
Delay
Flag Related
General purpose transfer instructions(copy, move.,)
2. Relocations.
2. ld Special test cases will be created for linker.
relocation tests are listed in a table below
3. bfd Test case “bfin_bfd.c” test for bfd changes
4. binutils-all This subdirectory contains test cases for nm, size,
Objdump,Objcopy, readelf and ar.We will run all of the
existing test cases in this subdirectory.
Also include new test case for objdump, objcopy and
readelf, and nm accordingly.
5.Opcodes A Special tool „asmtesttool‟ is written to test opcodes.
Section 9 below describes the details of testing using this
tool.
Following assembly test cases will be executed:
Functionality/instruction type Testcase(s)
Arithmetic Arith-1.s
Arith-2.s
Arith-3.s
Arith-4.s
Arith-5.s
Arith-6.s
Arith-7.s
Arith-8.s
Arith-9.s
Control code (CC) ControlCode-1.s
ControlCode-2.s
ControlCode-3.s
ControlCode-4.s
Cache control CacheCtrl-1.s
CacheCtrl-2.s
CacheCtrl-3.s
CacheCtrl-4.s
Transefer Transfer-1.s
Transfer-2.s
Transfer-3.s
TestAndSet.s
Return Ret-1.s
Registers Register-1.s
Move Move-1.s
Move-2.s
Move-3.s
Move-4.s
Move-5.s
Logical-1.s
Logical-2.s
Logical-3.s
Logical-4.s
Logical-5.s
Logical-6.s
Logical-7.s
Interrupt Interrupt-1.s
Interrupt-2.s
Interrupt-3.s
Iteration-1.s
Delay Delay.s
Exception Exception-1.s
Exteral control ExtControl-1.s
Following relocation test cases will be executed manually:
Steps:
1.Create object file from the .s files using bfin assembler and linker
Example : bfin-elf-as main.s –o main.o
bfin-elf-as main1.s –o main1.o
bfin-elf-ld main.o main1.o –o main
2.Run bfin-elf-objdump tool on the object file.
Example: bfin-elf-objdump –D main
bfin-elf-objdump –r main
3.Verify whether all the relocations are resolved correctly or not.
Relocation type Test case(s)
Call call_main.s
call_main1.s
if.cc.jump ccjump_main.s
ccjump_main1.s
jump.l jumpl_main.s
jumpl_ain1.s
jump.s jumps_main.s
jumps_main1.s
Lsetup lsetup_main.s
lsetup_main1.s
preg=target pregeq_main.s
pregeq_main1.s
rN=target rNeq_main.s
rNeq_main1.s
Arithmetic relocations Arith_main.s
Arith_main1.s
4 Features Not To Be Tested
- C++ features will not be tested.
- Following tools under binutils will not be tested: gprof, dlltool and gasp.
5 Approach
5.1 Unit Testing
- Relocation testing. VDSP compatible relocations are being introduced in the
port. Relocation testing will test these relocations. Refer Section 8 for specific
tests that will be performed.
- Assembler testing. Assembler will be tested at the unit level. Refer Section 8
for details of the tests that will be run.
5.2 Integration Testing
All regression tests will be run on following platforms during acceptance
- SuSe Linux
- cygwint Linux.
5.3 System Testing
No Separate System testing is planned for binutils.
Note : Assembler and linker will be tested along with the gcc during system testing.
Binutils utilities will also undergo testing individually as described in this document.
5.4 Regression Testing
Regression tests are carried out regularly to ensure that the functionality and
behavior of the binutils hasn‟t changed while changes were done in the source code
Regression testing involves following steps
Get the latest sources for binutils from the CVS repository.
Compile and Build the binutils.
Run binutils test suites using dejagnu tools. (Section 15 below describes the test
execution)
5.5 Performance and Stress Testing
No specfic performance or stress testing is planned for binutils. Gcc stress testing
does some testing of assembler and linker.
5.6 Documentation Testing
Document changes will be reviewed.
5.7 Beta Testing
NA.
5.8 Additional Testing
NA.
6 Item Pass/Fail Criteria
Regression Test:
There will be a set of expected results for these tests, which will be used by the tool
to decide whether the test is fail or pass.
Analyse the logs produced by dejagnu tools. Look at fails and cross check the
dejagnu tool‟s result by reproducing the error.
All the test cases flagged as unexpected fail or pass by dejagnu are considered as
fail.
All other tests are considered as having passed.
Opcode test:
‘asmtesttool’ reads each assembly instruction and compares with the expected opcode.
Depending on the results of comparison, the tools print the instruction and pass or fail against
them. An example is given as follows:
Example: Testing Opcode
„asmtesttool‟ test for all BF533 opcodes. It takes input file as command parameter.
Typically format of ininput file will look like
assembly code @ op code value
e.g.
IF CC R0 = SP; @ 0746
opcode value must be in hex.
No line should carry more then one set of assembly code. However there is no limitation
for number of lines in any input file.
OPTIONS
Other optional parameters are (preceded by '-' charecter):
'd': Show lots of debug info.
'e': Show only error (Supress the OK statement).
's': Suppress assembler error messages.
'h': Show this help statement
Running ‘asmtesttool’
set the envirnment variable BFIN_ASM_PATH to blackfin assembler path.
e.g export BFIN_ASM_PATH=/usr/bfin/binutils/bin/bfin-as
cd into the directory where your asmtesttool and input file is present.
Command : ./asmtesttool myinputfile
Above command will generate output for every line of inputfile, whether it fais or
succeed.
Alternatively the command : ./asmtesttoo –es myinputfile
will only show output for line which will get failed and also supress the assembler
messages.
Test results
Error at line 5: Assembler unable to generate op code for CALL (P1 ;
[omkar@linux testworkshop]$ ./asmtesttool test
Analyzing NOP and expecting opcode value 0(0x0000): OK
Analyzing RTS and expecting opcode value 16(0x0010): OK
Analyzing CLI R0 and expecting opcode value 48(0x 0030): OK
Analyzing JUMP (P0) ; and expecting opcode value 80(0x0050 ): OK
Analyzing CALL (P1 ; and expecting opcode value 28769(0x 07061):__temp.s:
Assembler messages:
__temp.s:3: Error:
parse errorInput text was `'
__temp.s:3: Error:
Parse error.
FAILED
Error at line 5: Assembler unable to generate op code for CALL (P1 ;
Analyzing IF CC R0 = SP ; and expecting opcode value 1862(0x 0746 ): OK
Exit Criteria:
All the test cases (listed above - function to be tested) have been executed and
results are verified, bugs are reported and approved.
There should not be any unexpected failures.
7 Suspension Criteria and Resumption Requirements
N/A.
8 Test Deliverables
Deliverables are listed below.
Test plans
Test reports
9 Testing Tasks
N/A.
10 Environmental Needs
The test environment shall be as defined below:
ADSP BF-
Null Modem / Ethernet 531/2/3
cable
JTAG
Host Computer
Hardware
ADI EZ-KIT Lite BF-533 Development board
JTAG
Host computer
Software
uCLinux Kernel Version 2.4.18 or greater for ADI EZ-KIT Lite BF-533
Development board
SuSe Linux
cygwin
Network drivers
Testing Tools
The following tools will be used:
1. Dejagnu
2. Tracker
Dejagnu
Dejagnu will be used for test execution. Regression testing for binutils will be very
similar to gcc testing. The following procedure will be followed:
1. Organize test cases in the respective subdirectories like gas, ld, bfin etc.,
2. Create a directory „bfin‟ under each of the following directories gas, ld, and
bfin.
3. Copy copy corresponding test cases into each folder.
Test plan RRAP-ADI-TP-001 explains the steps to executed regression tests using
dejagnu.
Tracker
Tracker will be used as the bug-tracking tool. The tool will be exposed to ADI over
the Internet, giving the ability to file bugs and query status.
Each of the bugs shall be classified into „High‟, „Medium‟ and „Low‟ priority
bugs in the test reports.
All bugs shall be assigned a unique number and entered into the bug-tracking
system.
The bugs shall be assigned to the respective module owner. Once the cause of
the bug is identified and its solution has been implemented, the bug will be
moved over to the „Fixed‟ status.
During subsequent testing, the bug shall either be „Closed‟ or be revoked as
„Open‟.
The test iterations shall continue until all bugs have been tracked to closure.
11 Responsibilities
- Pradip shah: Project Management
- Omkar: Testing, Test plan creation and execution, Reporting bug(s).
12 Staffing and Training Needs
N/A.
13 Schedule
Binutils tests will be execution start from on4/01/04 and should
complete on 04/30/04
14 Risks and Contingencies
The project depend on the following:
- Availability of ADI EZ-KIT Lite BF-533 Development board.
- Availability of connectivity to the EZ-KIT lite boards.
So we assume that:
- ADI EZ-KIT Lite BF-533 Development board with all necessary interfaces is
available in time.
- ADI EZ-KIT Lite BF-533 Development board is up with uCLinux and support
network features required for testing.
15 Approvals
Pradip shah, project manager
To Do List
# Who Due What
1 Groucho 4/7/2003 Add code inspection plans
2 Zeppo 4/1/2003 Add acceptance test plans
Revision History
Version/Date Additions/Modifications Prepared/Revised By
0.9 Original put into this format Omkar
1.0 Reviewed by Pradip Shah
1.1 Section 8, Functions to be tested Omkar
1.3 Section for opcode test Omkar
1.4 Elaborated pass/fail criteria Omkar
1.5 Incorporated ADI feedback Omkar
1.6 Accepted ADI changes Omkar
1.7 Changes in section 8 - Added a table which lists Omkar
relocation tests