Test Plan - Download as DOC by xscape

VIEWS: 553 PAGES: 11

									Test Plan for binutils porting on BF533
Version No.         1.7
Released On         june 9, 2004
Author(s)           Omkar Murthy
Reviewed By         Pradip Shah

Table of Contents

      1 Introduction
      2 Test Items
      3 Features To Be Tested
      4 Features Not To Be Tested
      5 Approach
           o 5.1 Code Inspections
           o 5.2 Unit Testing
           o 5.3 System Testing
           o 5.4 Regression Testing
           o 5.5 Acceptance Testing
      6 Item Pass/Fail Criteria
      7 Suspension Criteria and Resumption Requirements
      8 Test Deliverables
      9 Testing Tasks
      10 Environmental Needs
      11 Responsibilities
      12 Staffing and Training Needs
      13 Schedule
      14 Risks and Contingencies
      15 Approvals
      To Do List
      Revision History
1 Introduction

Analog Devices Inc., hereinafter referred to as “ADI”, is a world leader in high
performance signal processing solutions. ADI positions the ADSP-BF53x family, also
known as the Blackfin Processor, as a full-fledged highly integrated system-on-a-chip
(SoC) solution for the next generation of digital communication and connected media

For the first member of this family – the ADSP BF535 – ADI has completed a port of the
uCLinux embedded operating system. It now intends to make available uCLinux for
other processors in the family, namely the ADSP BF531/2/3 processors.

This work involves three main components:
   1. Linux Port, Bootloader and Library support
   2. Peripheral device drivers for the Board.
   3. Compiler tool chain

The objectives and tasks of the test effort as stated as follow:


                      -   Perform tests for binutils ported for on BF533 platform.


                      -   Test environment setup
                      -   Regression Test

2 Test Items

The test plan for binutils details the various tests to be executed, test procedure,
schedule and deliverables.
Following tests will be performed:
    1. Regression tests are carried out to ensure that the functionality and behavior
       of the binutils hasn‟t changed while changes were done in the source code.
3 Features To Be Tested

Following tools will be tested under binutils
All the test cases will be maintained under respective sub directories named same as tool
name like gas, bfd, ld.
Test cases in correspondig subdirectories will be executed using dejagnu.

         Tools             Test cases and Description
         1. gas            create subdirectory bfin and include BF533 architecture and
                           instruction specific testcases.
                           Following categories of tests will be created.
                                   1. 533 instruction set
                                    External Hardware Sync/control
                                    Transfer instructions(conditional, unconditional)
                                    Iteration(loop, while..,)
                                    Flag Related
                                    General purpose transfer instructions(copy, move.,)
                                   2. Relocations.

         2. ld             Special test cases will be created for linker.
                           relocation tests are listed in a table below
         3. bfd            Test case “bfin_bfd.c” test for bfd changes
         4. binutils-all   This subdirectory contains test cases for nm, size,
                           Objdump,Objcopy, readelf and ar.We will run all of the
                           existing test cases in this subdirectory.
                           Also include new test case for objdump, objcopy and
                           readelf, and nm accordingly.
         5.Opcodes         A Special tool „asmtesttool‟ is written to test opcodes.
                           Section 9 below describes the details of testing using this

   Following assembly test cases will be executed:

   Functionality/instruction type               Testcase(s)
   Arithmetic                                     Arith-1.s

Control code (CC)                              ControlCode-1.s
Cache control                                  CacheCtrl-1.s

Transefer                                      Transfer-1.s

Return                                      Ret-1.s
Registers                                   Register-1.s
Move                                          Move-1.s

Interrupt                                      Interrupt-1.s

Delay                                          Delay.s
Exception                                      Exception-1.s
Exteral control                                ExtControl-1.s

Following relocation test cases will be executed manually:
1.Create object file from the .s files using bfin assembler and linker
     Example : bfin-elf-as main.s –o main.o
               bfin-elf-as main1.s –o main1.o
               bfin-elf-ld main.o main1.o –o main

2.Run bfin-elf-objdump tool on the object file.
    Example: bfin-elf-objdump –D main
                 bfin-elf-objdump –r main

  3.Verify whether all the relocations are resolved correctly or not.

  Relocation type                               Test case(s)
  Call                                          call_main.s
  if.cc.jump                                    ccjump_main.s
  jump.l                                        jumpl_main.s
  jump.s                                        jumps_main.s
  Lsetup                                        lsetup_main.s
  preg=target                                   pregeq_main.s
  rN=target                                     rNeq_main.s
  Arithmetic relocations                        Arith_main.s

4 Features Not To Be Tested

   -   C++ features will not be tested.
   -   Following tools under binutils will not be tested: gprof, dlltool and gasp.

5 Approach

5.1 Unit Testing

   -   Relocation testing. VDSP compatible relocations are being introduced in the
       port. Relocation testing will test these relocations. Refer Section 8 for specific
       tests that will be performed.
   -   Assembler testing. Assembler will be tested at the unit level. Refer Section 8
       for details of the tests that will be run.

5.2 Integration Testing

All regression tests will be run on following platforms during acceptance
    - SuSe Linux
    - cygwint Linux.
5.3 System Testing

No Separate System testing is planned for binutils.
Note : Assembler and linker will be tested along with the gcc during system testing.
Binutils utilities will also undergo testing individually as described in this document.

5.4 Regression Testing

Regression tests are carried out regularly to ensure that the functionality and
behavior of the binutils hasn‟t changed while changes were done in the source code

Regression testing involves following steps
      Get the latest sources for binutils from the CVS repository.
      Compile and Build the binutils.
      Run binutils test suites using dejagnu tools. (Section 15 below describes the test

5.5 Performance and Stress Testing

No specfic performance or stress testing is planned for binutils. Gcc stress testing
does some testing of assembler and linker.

5.6 Documentation Testing

Document changes will be reviewed.

5.7 Beta Testing


5.8 Additional Testing


6 Item Pass/Fail Criteria

Regression Test:
There will be a set of expected results for these tests, which will be used by the tool
to decide whether the test is fail or pass.
Analyse the logs produced by dejagnu tools. Look at fails and cross check the
dejagnu tool‟s result by reproducing the error.
All the test cases flagged as unexpected fail or pass by dejagnu are considered as
All other tests are considered as having passed.

Opcode test:
‘asmtesttool’ reads each assembly instruction and compares with the expected opcode.
Depending on the results of comparison, the tools print the instruction and pass or fail against
them. An example is given as follows:

Example: Testing Opcode
     „asmtesttool‟ test for all BF533 opcodes. It takes input file as command parameter.
Typically format of ininput file will look like
     assembly code @ op code value
     IF CC R0 = SP; @ 0746
 opcode value must be in hex.
No line should carry more then one set of assembly code. However there is no limitation
for number of lines in any input file.

       Other optional parameters are (preceded by '-' charecter):
       'd': Show lots of debug info.
       'e': Show only error (Supress the OK statement).
       's': Suppress assembler error messages.
       'h': Show this help statement

   Running ‘asmtesttool’
   set the envirnment variable BFIN_ASM_PATH to blackfin assembler path.
   e.g export BFIN_ASM_PATH=/usr/bfin/binutils/bin/bfin-as

   cd into the directory where your asmtesttool and input file is present.
   Command : ./asmtesttool myinputfile
   Above command will generate output for every line of inputfile, whether it fais or

   Alternatively the command : ./asmtesttoo –es myinputfile
   will only show output for line which will get failed and also supress the assembler

   Test results
   Error at line 5: Assembler unable to generate op code for CALL (P1 ;
   [omkar@linux testworkshop]$ ./asmtesttool test
   Analyzing NOP and expecting opcode value 0(0x0000): OK
   Analyzing RTS and expecting opcode value 16(0x0010): OK
   Analyzing CLI R0 and expecting opcode value 48(0x 0030):        OK
   Analyzing JUMP (P0) ; and expecting opcode value 80(0x0050 ): OK
   Analyzing CALL (P1 ; and expecting opcode value 28769(0x 07061):__temp.s:
   Assembler messages:
   __temp.s:3: Error:
    parse errorInput text was `'
   __temp.s:3: Error:
  Parse error.
  Error at line 5: Assembler unable to generate op code for CALL (P1 ;
Analyzing IF CC R0 = SP ; and expecting opcode value 1862(0x 0746 ): OK

Exit Criteria:
All the test cases (listed above - function to be tested) have been executed and
results are verified, bugs are reported and approved.
There should not be any unexpected failures.

7 Suspension Criteria and Resumption Requirements


8 Test Deliverables

Deliverables are listed below.
       Test plans
       Test reports

9 Testing Tasks


10 Environmental Needs

The test environment shall be as defined below:

                                                            ADSP BF-
                           Null Modem / Ethernet             531/2/3
   Host Computer


          ADI EZ-KIT Lite BF-533 Development board
          JTAG
          Host computer


          uCLinux Kernel Version 2.4.18 or greater for ADI EZ-KIT Lite BF-533
           Development board
          SuSe Linux
          cygwin
          Network drivers

Testing Tools
The following tools will be used:
  1. Dejagnu
  2. Tracker

Dejagnu will be used for test execution. Regression testing for binutils will be very
similar to gcc testing. The following procedure will be followed:
   1. Organize test cases in the respective subdirectories like gas, ld, bfin etc.,
   2. Create a directory „bfin‟ under each of the following directories gas, ld, and
   3. Copy copy corresponding test cases into each folder.
Test plan RRAP-ADI-TP-001 explains the steps to executed regression tests using

Tracker will be used as the bug-tracking tool. The tool will be exposed to ADI over
the Internet, giving the ability to file bugs and query status.
        Each of the bugs shall be classified into „High‟, „Medium‟ and „Low‟ priority
           bugs in the test reports.
          All bugs shall be assigned a unique number and entered into the bug-tracking
          The bugs shall be assigned to the respective module owner. Once the cause of
           the bug is identified and its solution has been implemented, the bug will be
           moved over to the „Fixed‟ status.
          During subsequent testing, the bug shall either be „Closed‟ or be revoked as
          The test iterations shall continue until all bugs have been tracked to closure.
11 Responsibilities

   -   Pradip shah: Project Management
   -   Omkar: Testing, Test plan creation and execution, Reporting bug(s).

12 Staffing and Training Needs


13 Schedule

              Binutils tests will be execution start from on4/01/04 and should
               complete on 04/30/04

14 Risks and Contingencies

   The project depend on the following:
    - Availability of ADI EZ-KIT Lite BF-533 Development board.
    - Availability of connectivity to the EZ-KIT lite boards.

   So we assume that:
    - ADI EZ-KIT Lite BF-533 Development board with all necessary interfaces is
       available in time.
    - ADI EZ-KIT Lite BF-533 Development board is up with uCLinux and support
       network features required for testing.

15 Approvals

Pradip shah, project manager
To Do List

<List of items to be completed in THIS artifact.>

#       Who       Due           What
1       Groucho   4/7/2003      Add code inspection plans
2       Zeppo     4/1/2003      Add acceptance test plans

Revision History

Version/Date         Additions/Modifications                            Prepared/Revised By
0.9                  Original put into this format                      Omkar
1.0                  Reviewed by Pradip Shah
1.1                  Section 8, Functions to be tested                  Omkar
1.3                  Section for opcode test                            Omkar
1.4                  Elaborated pass/fail criteria                      Omkar
1.5                  Incorporated ADI feedback                          Omkar
1.6                  Accepted ADI changes                               Omkar
1.7                  Changes in section 8 - Added a table which lists   Omkar
                     relocation tests

To top