IDL Source Generator and IDL Compiler Testing Tool by Levone

VIEWS: 41 PAGES: 43

									   Charles University in Prague
Faculty of Mathematics and Physics




MASTER THESIS



           Pavel Strnad

      IDL Source Generator and
      IDL Compiler Testing Tool


  Department of Software Engineering (KSI)

     Supervisor: Ing. Petr Tůma, Dr.
             Study branch: Computer Science
                                                                 




       I would like to thank very much my supervisor Petr T ma for his advice.




       This is to certify that I wrote this thesis on my own and that the references
include all the sources of information I have utilized. This thesis is freely available for
study purposes.

Prague, April 6, 2002                                                Pavel Strnad




                                             ii
Table of Contents


Preface .................................................................................................................................. 1
1.     Introduction ................................................................................................................... 2
     1.1.     Motivation ............................................................................................................. 2
     1.2.     Thesis goals ........................................................................................................... 2
2.     Work structure and typographical conventions............................................................... 3
     2.1.     Work structure ....................................................................................................... 3
     2.2.     Typographical conventions .................................................................................... 3
3.     Analysis of problem....................................................................................................... 4
     3.1.     How does the IDL compiler work .......................................................................... 4
       3.1.1.         Lexical analyser ............................................................................................. 4
       3.1.2.         Syntactical analyser........................................................................................ 4
       3.1.3.         Semantic analyser........................................................................................... 5
       3.1.4.         Code generator ............................................................................................... 5
     3.2.     What are the possible failures of the IDL compilers ............................................... 5
       3.2.1.         Possible failures of the lexical analyser........................................................... 5
       3.2.2.         Possible failures of the syntactical analyser .................................................... 5
       3.2.3.         Possible failures of the semantic analyser ....................................................... 5
       3.2.4.         Possible failures of the code generator............................................................ 5
     3.3.     Short analysis of the IDL language......................................................................... 6
     3.4.     What should be tested ............................................................................................ 6
       3.4.1.         Front-end tests................................................................................................ 6
       3.4.2.         Back-end tests ................................................................................................ 6
4.     Possible ways of IDL compiler testing ........................................................................... 7
     4.1.     The IDL generator.................................................................................................. 7
     4.2.     Possible ways of output testing............................................................................... 7
       4.2.1.         Analysing of the IDL compiler output ............................................................ 7
       4.2.2.         Usage of mapping produced by the IDL compiler........................................... 9
     4.3.     Pros and cons of described variants .......................................................................11
       4.3.1.         Analysing of the IDL compiler output ...........................................................11
       4.3.2.         Usage of mapping produced by the IDL compiler..........................................12


                                                                iii
     4.4.     Chosen solution.....................................................................................................13
5.     Brief description of chosen solution ..............................................................................15
     5.1.     How are the IDL compilers tested .........................................................................15
     5.2.     How does the testing tool work .............................................................................15
6.     Description of implementation......................................................................................17
     6.1.     Platform ................................................................................................................17
       6.1.1.        Testing tool ...................................................................................................17
       6.1.2.        Target language.............................................................................................17
     6.2.     Approach to test the IDL compilers.......................................................................17
       6.2.1.        Understanding of IDL....................................................................................17
       6.2.2.        Correctness of generated code .......................................................................17
       6.2.3.        Running of the generated code.......................................................................18
     6.3.     Testing tool structure.............................................................................................18
       6.3.1.        The IDL generator .........................................................................................18
       6.3.2.        Tester generator for C++ ...............................................................................23
       6.3.3.        Connector......................................................................................................25
     6.4.     User’s guide ..........................................................................................................26
       6.4.1.        System requirements .....................................................................................26
       6.4.2.        Configuration files.........................................................................................26
       6.4.3.        Header files ...................................................................................................28
       6.4.4.        Running the testing tool and using of generated code.....................................29
7.     Summary of the work ...................................................................................................31
     7.1.     Fulfilling the goals ................................................................................................31
     7.2.     Comparison with other known methods.................................................................31
     7.3.     Future work...........................................................................................................31
8.     Conclusion....................................................................................................................33
9.     Bibliography.................................................................................................................34
Appendix A           Discovered bugs in the IDL compilers ...........................................................35
       Understanding of IDL ...................................................................................................35
       Conformance with C++ specification............................................................................36
       Execution errors............................................................................................................37




                                                              iv
Abstract



            Czech
         Název práce: Generátor IDL a testovací nástroj pro IDL kompilátory
         Autor: Pavel Strnad
         Katedra (ústav): Katedra softwarového inženýrství (KSI)
         Vedoucí diplomové práce: Ing. Petr T ma, Dr. 




         Email vedoucího: petr.tuma@mff.cuni.cz
       Abstrakt: N které kompilátory jazyka IDL se nechovají správn na všech
                     ¡                                                                   ¡




p edložených vstupech. IDL kompilátory mohou selhat na n kolika úrovních. Tato
 ¢                                                                   ¡




diplomová práce odhaluje potenciální problémy IDL kompilátor na r zných úrovních.
                                                                                  




Obsahem této diplomové práce je také implementace testovacího nástroje pro IDL
kompilátory, který je schopen najít velkou adu chyb. Architektura testovacího nástroje
                                             ¢




umož uje testování všech typ IDL kompilátor nezávisle na cílovém jazyce.
     £                                                        




Nezávislost na používané platform je zaru ena použitím programovacího jazyka Java.
                                        ¡        ¤




Implementace je hotova pro IDL kompilátory, které p ekládají do C++. B hem
                                                                 ¢                           ¡




testování samotné aplikace bylo objeveno n kolik chyb IDL kompilátor , které byly
                                                         ¡                            




oznámeny jejich výrobc m. Mohou jim pomoci v tom, aby se po et chyb v jejich
                                                                             ¤




kompilátorech zmenšil.
         Klí ová slova: IDL, kompilátor, generátor, test, selhání
            ¥




              English
         Title: IDL Source Generator and IDL Compiler Testing Tool
         Author: Pavel Strnad
         Department: Department of Software Engineering (KSI)
         Supervisor: Ing. Petr T ma, Dr
                                 




         Supervisor’s email address: petr.tuma@mff.cuni.cz
        Abstract: Some IDL compilers do not behave correctly on all IDL definitions.
They can fail at several levels. This thesis discovers potential problems of the IDL
compilers at different levels. The thesis contains the implementation of the IDL compiler
testing tool, which is able to detect the wide range of bugs. The architecture of the
testing tool allows testing of all IDL compilers independently on the target language.
The independence on used platform is ensured by usage of Java language. The
implementation is done for IDL to the C++ language compilers. Several bugs were
discovered during the testing of the testing tool and they were reported to IDL compiler
vendors. It can help the vendors to improve their IDL compilers.
         Keywords: IDL, compiler, generator, test, failure




                                                 v
Preface


       There are several systems which are used for the development of distributed
applications. Three mainly used systems are Java Remote Method Invocation (RMI) by
Sun Microsystems, Distributed Component Object Model (DCOM) by Microsoft and
Common Object Request Broker Architecture (CORBA) by OMG.
        Development of CORBA-compliant systems and programming tools is ensured
by many standards and specifications published by OMG (see [1], [2] and [3]).
CORBA-compliance is needed for interoperability of the systems from different
vendors and development of correct CORBA applications. The software is not 100%
bug-free in general. The same problem is with the IDL compilers. Finding the technique
for testing the IDL compilers and developing of the IDL compiler testing tool is a main
goal of this thesis.




                                          1
1. Introduction



   1.1. Motivation
       Some IDL compilers do not support all features, which are specified in the
CORBA standard or they contain several bugs. The measure of CORBA “non-
compliance” is variable in different implementations of the IDL compilers. If there is a
bug in the IDL compiler the process of IDL translation can stop and no output is
generated. There can also be another type of incorrect behaviour of the IDL compilers.
The generated code does not need conform to the target language grammar. The
behaviour of the generated code can differ from programmer’s intentions. As a result,
the CORBA programmer can encounter different errors at different levels.

   1.2. Thesis goals
        The first goal of this thesis is finding an effective way, which can be used for
testing behaviour of the IDL compilers. Testing is mainly focused on the correct
translation from IDL to the target language (e.g., C, C++, Java, etc). Testing of other
characteristics of the IDL compilers is outside the scope of this thesis.
       The second goal is creation of the IDL generator. The IDL generator should
generate complex IDL definitions. The complexity of generated definitions should be
adjustable. The complexity of IDL definitions means, which syntactical constructs are
used in the IDL definitions, and how much they are nested. The complexity of literals
and identifiers should also be adjustable.
       The third goal is a creation of the tool, which can test different implementation
of the IDL compilers from different vendors. This part of work uses the outcomes,
which are achieved during serving of the first and second goal. The output of the IDL
generator will be used as an input for the IDL compilers during their testing.




                                           2
2. Work structure and typographical conventions



  2.1. Work structure
    The first thing to do is to analyse possible failures of the IDL compilers. This
    analysis is described in chapter 3. The information obtained from this analysis is
    used in further stages. The next thing is specification of testing tool functions,
    which are able to verify correctness of the IDL compilers. The specification is
    described in chapter 4. Searching for similar projects is also an important stage,
    which helps to avoid duplicating already implemented solutions. Currently
    (2002/02) completed project (CORVAL2) is described in section 7.2. Next stage
    is searching for possible ways how to implement the testing tool. Some possible
    ways are described in chapter 4. Choosing the best solution follows in section
    4.4. The analysis of the IDL language is needed for efficient implementation of
    the IDL generator. Short analysis is described in section 3.3. The
    implementation of the IDL generator and its testing is one of the most time
    consuming activities. The description of implementation of the IDL generator is
    in section 6.3.1. The last stage is an implementation of the C++ tester generator
    and its tests, which is described in section 6.3.2.

  2.2. Typographical conventions
    In the text, the following conventions are used:
   The IDL compiler
                  Normal text

   TGCPPTesterGenerator
                Name of existing Java class, method, parameter, …

   char
                    Name of IDL or C++ keyword, name of IDL grammar terminal
                    or non-terminal, …

   Cgxyz
                    Name of the class which is used in examples; It can be also used
                    for the group of classes with similar name.

   throws TesterGeneratorException
               Testing tool source code written in Java language

   #define
                    Tester source code in C++

   module {
                    IDL definitions

   #GCinterface
               Text in configuration file


                                        3
3. Analysis of problem



   3.1. How does the IDL compiler work
      The IDL compilers have usually similar structure as the general structure of
compiler shown in Figure 1.




                                         Figure 1
       Process of compiling is usually split into two parts: front-end and back-end. The
lexical analysis, syntactical analysis and semantic analysis are in the front-end.
Generating of the target language code is in the back-end.

       3.1.1. Lexical analyser
        The lexical analyser transforms the input into a row of lexical units. Each lexical
unit cannot be split into parts. The lexical units are sometimes called atoms. These
lexical units are basic language units.
       E.g., long is an atom.

       3.1.2. Syntactical analyser
       The syntactical analyser verifies that the input conforms to the IDL grammar.
All syntactical constructs are identified during this verification. Each syntactical
construct is composed from one or more lexical units.
       E.g., constant declaration: const long myLong = 38;




                                            4
       3.1.3. Semantic analyser
        The semantic analyser verifies that all relations among syntactical constructs are
correct. It means that only already defined constructs are used and that they are used at
the right places. The semantic analyser also verifies that all forward declaration have
their definitions.
       E.g., invalid constant declaration: const short myShort = myLong; long
constant cannot be used as a value of short constant.

       3.1.4. Code generator
        The code generator is last but important component of the IDL compiler. The
code generator uses information from previous phases and generates the target language
code (mapping of the IDL definitions to the target language). The generating of a code
depends on the target language. The IDL compilers from different vendors have usually
different code generators. The CORBA system must conform to its specification (see
[2]) but it allows many forms of implementation.

   3.2. What are the possible failures of the IDL compilers
        The IDL compiler can fail in some of the four components described in section
3.1. All possible failures can be divided into two groups: “front-end failures” and “back-
end failures”. Misunderstanding of IDL definitions causes the front-end failures. The
cause of the back-end failures is usually a bug in the code generator.

       3.2.1. Possible failures of the lexical analyser
        At least three types of failures can appear in the lexical analyser. The first one is
acceptance of a wrong lexical unit. The second one is misunderstanding of regular
lexical unit. The third one is understanding of the IDL definitions in the wrong way.
E.g., long long can be understood as a double type specification. The first and third
failure usually confuse the syntactical analyser. The second failure stops the execution
of the lexical analyser.

       3.2.2. Possible failures of the syntactical analyser
        Correct syntactical analyser should accept the only IDL definitions, which
conform to the IDL specification [1]. Incorrect behaviour of the syntactical analyser can
have at least two forms. The syntactical analyser can accept the incorrect definitions or
it can refuse the correct IDL definitions.

       3.2.3. Possible failures of the semantic analyser
        The semantic analyser should verify correct usage of the syntactical constructs.
The verification fails if the semantic analyser does not announce any errors but at least
one forward declaration has not its definition or at least one syntactical construct is used
but not defined etc.

       3.2.4. Possible failures of the code generator
        The code generator should produce the correct target language code
corresponding with the IDL definitions. Incorrect behaviour can cause at least two types
of failures. The produced code does not conform to the target language grammar. The


                                             5
second type of failure is hidden in the target language code, which conforms to the
target language grammar, but the execution of a code has a different behaviour than
intended.

   3.3. Short analysis of the IDL language
       The IDL grammar has 99 derivation rules as it is written in [1]. The grammar is
quite simple. One non-terminal can derive up to five other non-terminals except the
unlimited list of non-terminals. The only problem is recursive character of grammar.
E.g. module can derive to module and there is not a terminating rule. The IDL
generator, which produces IDL definitions recursively, should fix the recursion by some
mechanism.

   3.4. What should be tested

       3.4.1. Front-end tests
        All front-end failures can be divided into two groups. The first group contains
failures when the correct IDL definitions are used as an input. The second group
contains failures when the incorrect IDL definitions are used as an input. Both types of
failures are important but testing the incorrect refusing of correct IDL definitions is
more useful for the CORBA programmers than testing the acceptance of the incorrect
IDL definitions. The implementation of the IDL generator, which produces correct IDL
definitions, is one of the goals of this thesis.
       The IDL generator should focus on three possible front-end failures described in
section 3.2.
        The IDL generator should generate IDL definitions with variable lexical
complexity. The lexical complexity of definitions should be adjustable by users. The
lexical complexity includes complexity of the identifiers and complexity of the literals
and their combinations.
       The syntactical complexity should also be adjustable by users. The syntactical
complexity includes a number of used syntactical constructs, measure of their nesting,
depth of their nesting etc.
       The last but not the least attribute of generated IDL definitions is their semantic
complexity. The adjustability of the semantic complexity includes measure of usage of
the forward declarations; measure of usage already defined syntactical constructs etc.

       3.4.2. Back-end tests
        There are two main types of the back-end failures, as it is written in section
3.2.4. The target language grammar conformance has to be tested as the first thing. The
second test should verify that the target language code conforms to the intentions
written in the IDL definitions.




                                            6
4. Possible ways of IDL compiler testing



   4.1. The IDL generator
        The implementation of the IDL generator should follow the recommendations
written in section 3.4.1. The IDL definitions produced by the generator have to conform
to the IDL specification [1]. The three main attributes of generated definitions (lexical
complexity, syntactical complexity and semantic complexity) have to be adjustable.

   4.2. Possible ways of output testing
        In the following paragraphs there is a description of several possible ways how
the IDL compilers can be tested. They are tested for a correct translation from IDL to
the target language. The following list of described ways is not complete. Some other
ways are also possible but these look more realistic than the others do.

       4.2.1. Analysing of the IDL compiler output




                                        Figure 2
       The first approach is a direct one. It looks like an automatization of checking of
the target language code generated by the IDL compiler by hand. The testing tool
analyses the input of the IDL compiler. It also analyses the target language code
generated by the IDL compiler. The analysis is performed in the same way as analysis
of IDL. The analysis of IDL is mainly focused on identification of the syntactical
constructs. The analysis of the target language code is also focused on the conformance
with the target language grammar. The comparison of both sources can start after the

                                           7
end of these two analyses. The corresponding syntactical constructs are searched during
this comparison. Some unmatched constructs can be found in both sources. The whole
process is shown in Figure 2.
       The comparator should have two components. The first one is the IDL analyser
and the second one is target language code analyser. This decomposition allows a
simple replacement of the target language analyser for different target languages.
        The Analysis of IDL and the analysis of the target language code are performed
in the same way as the front end of compiler does it. The source is decomposed into
simple parts and the same method is used on these parts. This decomposition ends when
all the parts become atoms. These atoms cannot be decomposed to more than one part.
No derivation rule is described in language grammar for the atoms. This analysis can be
implemented in Perl or awk because they have good support for text manipulation.
        Already existing tools can be used for the analysis of source in different
languages. These tools usually need some adaptation for usage with the comparator of
the IDL definitions and the target language code. The tools distributed with a source
code are more useful for this reason than other tools. The GNU compiler of C++ is a
good example of a useful tool. The front end of this compiler can be used for the
analysis of code conformance with the C++ grammar. The front end of the C++
compiler also decomposes the input C++ code to the atoms as it is written above. The
compilers for other languages, which are distributed with source code, can be used for
other target languages. Other useful tools are bison, flex and gperf.
        There is a high probability that mapping produced by the IDL compiler is correct
if all IDL constructs have their equivalents in the target language code and the target
language code conforms to the target language grammar.




                                        Figure 3




                                           8
        There is also another possible implementation of the comparator. The
comparator can be connected with the IDL compiler through the interface and receives
the information about the IDL definitions directly from the IDL generator. The
architecture is shown in Figure 3. This variant does not need the IDL analyser but it can
be used only together with the IDL generator.

       4.2.2. Usage of mapping produced by the IDL compiler
        The second way one can verify the correctness of generated code is quite
different from the one described in section 4.2.1. The generated test application
corresponds with the input IDL definitions. The target language code produced by the
IDL compiler and test application are compiled with appropriate compiler. Produced
object files are linked to the client and server parts and both parts are then started.
        The code written in some scripting language is not compiled but an interpreter
interprets it together with test applications. The Java language code is not linked but the
classes are split into a client and server part. The compiler (interpreter) of target
language verifies that the code produced by the IDL compiler conforms to the target
language grammar. Existence of all language constructs corresponding to the constructs
defined in the IDL definitions is verified during the link or it is verified by the runtime
during the execution of test application. The correct target language compiler
(interpreter) and linker should announce all errors in the target language code. When the
executable code is available it can be run. The compilation and link without any errors
does not guarantee that the code does not contain any bugs. The test application can fail
immediately after its start or at any time during its execution. The test application can
be also terminated by the operating system because of some fundamental error. The
execution of an incorrect code can cause the system failure on some operating systems.
        Next type of target language code verification can be performed when the
executed code does not fail because of some error. Correct behaviour of the executed
code is checked during this verification. The executed code can run without any error
but it can do something else, which is not expected. The client side of the test
application calls methods, which should be implemented in the server side of the test
application. These calls should be performed on the instances of classes, which were
generated by test application generator (tester generator). Optional return values are
returned when the call is correctly performed and all parameters of called method are
transferred to the server side of the test application. The test is successfully finished
when the calls for every method are performed correctly.
       The implementation of the tester generator can have several forms. Two of
possible variants are described in the following paragraphs.
       The first variant is a combination of the IDL generator and the tester generator
into one application. This generator produces the IDL definitions and appropriate test
applications in the target language. There are several possibilities of implementation.
The decomposition into two components connected though the well-defined interface
looks like the most suitable architecture of the generator. This architecture allows
simple replacement of the tester generator component for different target languages.
The architecture is shown in Figure 4.




                                            9
                                         Figure 4
        The second variant is a two-stage tester generator. The first part is the IDL
analyser and the second part is the real tester generator. The IDL definitions analyser
detects, which syntactical constructs are contained in the IDL definitions. These
syntactical constructs are announced to the real tester generator. The architecture can be
very similar to the previous variant but the real tester generator is connected to the IDL
analyser instead direct connection to the IDL generator. The architecture is shown in
Figure 5. This two-stage tester generator can generate test application from all IDL
definitions.




                                           10
                                       Figure 5


   4.3. Pros and cons of described variants
       The architecture of all described variants allows replacement of any component
without a need for modification of remaining components. It is caused by the
independence of all components. Some are used separately and others are connected
through the well-defined interface.
       All described variants verify that the target language code produced by the IDL
compiler conforms to the target language specification. All ways of testing IDL
compilers can verify correctness of generated target language code but used ways and
achieved results differ. Detailed description of the differences is in the following
paragraphs.

       4.3.1. Analysing of the IDL compiler output
       This solution is very straightforward. There is no doubt about the way, how it
works. It works like comparison of the IDL definitions and the target language code by
hand. The first variant (independent comparator) can be used on all IDL definitions and
its mapping to the target language. There is no dependency on the implementation of the
IDL generator in an independent comparator. The second variant (online comparator)


                                          11
does not need to identify the syntactical constructs in the IDL definitions. The online
comparator obtains all information about the IDL definitions directly from the IDL
generator. There is a very good identification of the location of incorrect mapping in
both variants. On the other hand this solution does not detect errors in the target
language code, which conforms to the target language specification but it has only
supplementary function and does not have an equivalent in the IDL definitions. E.g.,
some interface defined in the IDL definitions is mapped to the class in the C++ code,
which uses a null pointer in its constructor. The failure can be detected during the
execution. Another example follows:
       if(…)
            statement;
       else
            // missing statement
       statement; // this statement should be executed every
time
       Similar bugs can stay hidden it there is no failure of the test application.
       Summary:
                   + Good localization of incorrect mapping
                   + No dependence on the third party programs
                   − Does not detect some bugs

          Independent comparator
       Summary:
                   + Straightforward solution
                   + All IDL definitions
                   − Needs the IDL analyser

          Online comparator
       Summary:
                   + No need for the IDL analyser
                   − Cannot be used on all IDL definitions
                   − Cooperation with the generator of IDL definitions

       4.3.2. Usage of mapping produced by the IDL compiler
        The verification of the conformance of the target language code with the target
language specification is done by the third party application (compiler, interpreter).
Many developers use the compiler (interpreter) of the target language. For this reason
the probability of existence of a bug in it is lower than the probability of existence of a
bug in a newly developed application, which is used only for these purposes. The
possible change of the target language specification does not pose any new claim on the
testing tool. The new version of compiler (interpreter) has to be used. Only enormous
changes (e.g., renaming of main function in C++) in the target language specification
can lead to modification of the tester generator.
       Test of other attributes of the target language code is performed by its execution.


                                            12
       Summary:
                  + Usage of already implemented application (some tests are for
                    nothing, automatic upgrade)
                  + Demonstrative test of mapping correctness (execution of test
                    application)
                  + Detection of well-hidden bugs (null pointer usage, etc.)
                  − No localization of these bugs
                  − Usage of already implemented application (trust in its correct
                    behaviour)
        The tester generator implementation can have at least two forms as it is written
in section 4.2.2. Their pros and cons follow in following two sections.

           Direct generating of the tester
       The big advantage of this solution is that there is no need for analysis of any
source in any language. The IDL generator informs the tester generator about all
generated syntactical constructs. The tester generator generates test applications
corresponding with the received information from the IDL generator.
       The disadvantage is that the tester generator and the IDL generator must
cooperate.
       Summary:
                  + No need for source analyser
                  + Faster tester generating
                  − Cannot be used on all the IDL definitions
                  − Cooperation with the IDL definitions generator

             Post-generating of the tester
        The advantage of this variant is that it can generate the tester from all IDL
definitions. There is no direct link between the tester generator and the IDL generator.
        The disadvantage is that the tester generator has to contain the IDL analyser,
which detects what syntactical constructs are used in the IDL definitions. The previous
variant of the tester generator obtains this information directly from the IDL compiler.
The progress of the tester generator is the same as the progress of the tester generator
from previous variant after obtaining the information about the IDL definitions.
       Summary:
                  + All the IDL definitions
                  − Need for IDL analysis

   4.4. Chosen solution
        The direct tester generating was chosen because it can achieve the best results at
a reasonable price from described variants. It can detect well-hidden bugs. The
verification of correctness of the target language code is very demonstrative. There is no


                                           13
need for the source analyser to check whether the target language code conforms to the
target language specification. The compiler (interpreter) of the target language is used
instead. This solution is limited for using only these IDL definitions, which are
generated with the appropriate IDL generator. The need for usage with IDL definitions,
which are obtained in another way, is not very big because the IDL generator allows
generating of definitions with demanded complexity.
        The other reason is, that this implementation can be used together with master
thesis called “IIOP Engine For Testing CORBA Implementation Robustness”, which is
also worked at the Faculty of Mathematics and Physics. The main goal of the master
thesis is: design and implement an IIOP engine capable of interfacing with a CORBA
client and a CORBA server through IIOP and generating malformed IIOP requests with
a goal of testing the robustness of the CORBA implementation.




                                          14
5. Brief description of chosen solution


       The details about implementation of chosen solution can be found in chapter 5.2.
User’s guide is available in section 6.4.
        The following description is influenced by choice of C++ as the target language.
There are some differences in some other languages. The Java binary code is not linked,
script language code is not compiled and so on.

   5.1. How are the IDL compilers tested
       The IDL compilers are tested in three phases. The IDL compilers are verified for
correct understanding of input IDL definitions in the first “IDL understanding” phase.
Generated mapping is verified for correctness during the second “basic mapping check”
phase. The behaviour of generated target language code is tested during the third
“execution” phase. All phases are shown in Figure 6.




                                        Figure 6


   5.2. How does the testing tool work
      The IDL generator generates the IDL source and informs the tester generator
about it. The tester generator generates the test application after obtaining all
information about the IDL definitions. The tester generator is prepared for direct access


                                           15
to all generated syntactical constructs. Because of this there are only few constraints for
the IDL generator behaviour. The IDL generator generates syntactical constructs
recursively (e.g., method is generated during the generating of interface). The connector
adapts the IDL generator calls to the appropriate form. The architecture is shown in
Figure 6.
        The process of testing can start after the end of generating of the IDL definitions
and the test application source code. Generated IDL definitions are used as an input of
the IDL compiler. The IDL compiler should translate the input into the target language
code (stubs & skeletons). The IDL compiler does not pass “IDL understanding” phase if
there is a problem during the translation.
        The target language code is used as an input of the target language compiler. The
target language compiler should compile the target language code to the object files.
The IDL compiler does not pass the “basic mapping check” phase if there is some
problem during the compilation process. The test application source code is compiled to
the object files after successful passing the “basic mapping check” phase. Appropriate
object files are linked together and client and server test applications are ready to be
run. The “execution” phase follows.
        The server part of the test application is being run. If the application fails it may
indicate the problem in the generated mapping. The incorrect mapping does not cause
all failures. E.g. incorrect mapping usually causes errors like STACK OVERFLOW,
NULL POINTER ASSIGNMENT or MARSHALL EXCEPTION (after start of the
client). Some other errors like OUT OF MEMORY can be caused by system
configuration. Naturally OUT OF MEMORY error can be caused by memory leak in
the target language code.
        The client part of the test application is run after successful running of the
server. The failure of the client or server can follow. As it is written above the incorrect
mapping does not cause all failures. The user should identify, which failures are caused
by incorrect mapping.
        There is a high probability that the mapping is correct if there are no failures at
all during all phases.




                                             16
6. Description of implementation



   6.1. Platform

       6.1.1. The testing tool
       CORBA implementations and the IDL compilers are available for several
platforms. The Java platform was chosen for its portability. The API documentation is
available at world wide web (see [5]). The testing tool was developed using JDK 1.3.0
and was tested on Windows 2000, Red Hat Linux 7.2 and Sun Solaris 2.7. The Java
platform should provide the same interface to all applications on every operating system
with the Java runtime environment. For this reason, the testing tool should run on every
Java virtual machine.

       6.1.2. Target language
       The IDL compilers can translate IDL to many languages. OMG releases
mapping specifications for: Ada, C, C++, COBOL, CORBA Scripting Language, Java,
Lisp, PL/1, Python and Smalltalk.
       The Current implementation of the testing tool can fully test IDL to C++
compilers. Other IDL compilers can also be tested but there are some constraints.
Compilers are tested at three levels (see section 6.2). All compilers can be tested at the
first and second levels. An implementation of method TesterGeneratorDirect
AccessInterface.generateClientTestingApplication is needed at the third level for other
compilers. The implementation of this method depends on the target language.
       As it is written in section 4.2.2 the process of testing depends on the target
language (Java classes are not linked, scripting languages are interpreted). The
following text is written for tests of IDL to C++ compilers.

   6.2. Approach to test the IDL compilers

       6.2.1. Understanding of IDL
        Understanding of all syntactical constructs, possible identifiers or literals of the
IDL language described in its specification [1] is tested in the first “IDL understanding”
test. Some compilers are not fully CORBA-compliant and do not pass this test.
      The first component (IDL generator) of the testing tool is needed for this test.
The IDL generator generates complex definitions, which are used as an input of the IDL
compilers. The complexity of the generated definitions is adjustable.

       6.2.2. Correctness of generated code
      The generated code must conform to the target language specification. The
conformance with the target language specification is verified in this test.
       There is no guarantee that the IDL compilers produce correct target language
code even if they do not announce any warning or error. For this reason, the generated


                                            17
code is tested for correctness. It is not useful to implement verification application. The
compiler of target language (C++) can be used instead. The compiler tests a code
correctness and prepares object files for next test.

       6.2.3. Running of the generated code
       The executable code is tested for correct behaviour in the third “execution” test.
        Even if generated code conforms to the target language specification, it does not
guarantee correct behaviour. For this reason, test applications (client and server) are
generated during the generating of IDL definitions. These test applications are linked
together with the compiled code. Both parts of the application are being run. The
application needs the ORB to be installed. The server creates instances of all classes and
activates them. The client obtains references to all objects and tries calling their
methods. Some method calls can fail or raise an exception.

   6.3. Testing tool structure
       Every target language needs an appropriate generator of test application. For this
reason, the testing tool is composed of two main components. These components are the
IDL generator and the tester generator. The IDL generator generates IDL definitions
and informs the tester generator about its content. The tester generator generates test
applications based on received information from the IDL generator. The IDL generator
can stay unchanged for different target languages. Another IDL generator can also be
implemented and all tester generators can be used with it. The architecture is shown in
Figure 7.




                                         Figure 7
        The starting class of the testing tool is IDLCompilerTestingTool. This class
loads configuration file, creates an instance of CodeGenerator (main class of the IDL
generator) and an appropriate main class of the tester generator. Name of this class is
variable and is read from configuration file. IDLCompilerTestingTool generates IDL
definitions and test application after initialisation.

       6.3.1. The IDL generator


       Following terms will be used in the description of code generating.




                                            18
     “terminal”
                       Cannot derive to any item. E.g., literal char

     “non-terminal”
                       Can derive to several terminals and non-terminals. E.g.,
                       interface_dcl   consists  of    interface_header    and
                       interface_body.

     “strict non-terminal” (special kind of non-terminal)
                       Derives to exactly one non-terminal. E.g., declarator derives to
                       simple_declarator OR complex__declarator.

        The generating of IDL definitions is implemented in many classes, which have
corresponding terminals or non-terminals in IDL grammar. These classes can generate
and print its corresponding terminal or non-terminal. There are also some other
supporting classes, which are used by the IDL generator. A brief description of
important classes follows. The process of code generating is described after description
of the classes. The IDL generator uses several groups of classes with similar semantics.
Classes belonging to the same groups are not described individually if their behaviour is
not different from other classes in the group. Names of groups are underlined. More
detailed description of classes is available in “javadoc” documentation.
     CodeGenerator
                       Main class of the IDL generator; an instance of this class is
                       responsible for generating and printing of items.

     LexicalManager
                       Used for two reasons: generating of identifiers and generating of
                       literals. Identifiers must be unique inside its scope. Current
                       implementation of LexicalManager returns globally unique
                       identifiers.

     SemanticManagement
                   Holds the semantic managers (see below).

     Semantic managers
                    Used for storing generated items. They are accessible through an
                    instance of the class SemanticManagement using method
                    getXYZManage, where XYZ indicates type of manager. Some
                    managers can be used by other managers. E.g.,
                    ParameterTypeManager does not hold interfaces but uses
                    InterfaceManager for it.

     ManagerInterface
                    Interface for all semantic managers

     CommonManager (abstract)
                 Parent of all semantic managers; contains common
                 functionality; uses inner class Data for storing generated items.
                 Instances of class Data are held in tree. Searching key is prefix
                 of name (E.g., name of module). Appropriate part of prefix is
                 used in every level of tree.


                                           19
ConstTypeManager
             Holds all constant types except integer, char and boolean

ParameterTypeManager
              Holds types, which can be used as parameter types in methods
              declarations.

SwitchConstTypeCommonManager
               Holds integer, char and boolean. These types can be used as
               switch types in unions or constant types.

SwitchTypeManager
              Holds enums.

TypeManager
                  Holds types, which cannot be stored in another manager.

Semantic managers for constants
               Used for storing constants. Name of manager                     is
               ConstXYZManager where XYZ indicates type of constant.

ConstPositiveIntegerManager
                 It is special type of ConstIntegerManager. Used for storing
                 small (up to 100) positive integers. These constants are used as
                 array sizes and the like. Small numbers are required because of
                 limited memory used in the tester generator. E.g., char[1 000
                 000 000] needs 1 GB of memory.

GeneratorClassInterface
               Interface for all GCxyz and GCEXxyz classes.

GCxyz classes
                  Have corresponding terminals or non-terminals in IDL
                  specification [1]. Name of class is GCxyz where “xyz” is name
                  of terminal or non-terminal of IDL grammar.

GCEXStringHolder
             Holds string; used for holding of basic types (e.g., char).

GCEXCommonxyz abstract classes
            Parents of GCxyz or GCEXxyz classes. Add some common
            functionality to all subclasses. E.g., performing with IDs and
            names.

GCEXxyz classes
                  Similar to terminals or non-terminals in IDL specification [1].
                  Name of class is GCxyz where “xyz” is name of terminal or non-
                  terminal of IDL grammar. Some of these classes

HoldedItem and subclasses
                Stored in managers; holds information about generated item.



                                    20
        The generating of IDL definitions is implemented in CGxyz and CGEXxyz
classes (see above). Generating of items is recursive. It means that new item generating
can start inside generating of other item. E.g., generating of method starts inside
generating of interface. The process of generating is shown in Figure 8.




                                        Figure 8
        Items are generated in method generate of an appropriate object. When item is
generated, method print can be called. This method prints content of the object into
specified file and cleans the object.




                                        Figure 9



                                          21
        The architecture of the IDL generator is shown in Figure 9. Generating starts in
the method generateCode of the class CodeGenerator. Instance of the class
GCspecification is created and its method generate is called. Non-terminal
specification is the starting point of IDL grammar. This non-terminal can derive only to
a list of non-terminals definition, which can derive to six other non-terminals. For the
description of IDL grammar see [1].
       There are several possible ways of an item xyz generating.
           1. The item xyz is a terminal. String representation of the terminal is stored
              inside the instance of the class GCxyz or in a supplemental class
              GCEXStringHolder.
           2. The item xyz is a non-terminal and consists of one or more other items.
              E.g., it consists of xyz1 and xyz2. Instances of GCxyz1 and GCxyz2 are
              created and methods generate are called on these instances.
           3. The item xyz is a non-terminal and consists of a list of xyz1 items.
              Instances of GCxyz1 is created for all members of a list and methods
              generate are called. The list can be empty.
           4. The item xyz is a strict non-terminal. Instance of sub-item is created and
              method generate is called on it.
           5. The item xyz is mixture of types 1, 2 or 3.


        When the control is returned to the method generateCode a method print is
called on instance of GCspecification. This method prints content of items, which have
not yet been printed because the method print is called recursively on sub-items. E.g.,
instance of GCinterface_dcl calls method print on its header (instance of the class
GCinterface_header) then prints opening brace (terminal {) then call the method print
on its body (instance of GCinterface_body) and then it prints closing brace (terminal }).
At the end of the method generateCode a method generatingCompleted is being called
on the tester generator. It announces that all the items are generated.
      Some items can be printed during the generating process. All pre-generated
items (see below) are announced to CodeGenerator using method
newItemPreGenerated. CodeGenerator calls the method print on these items. The
method generate in the class GCspecification calls a method compactBlockDone on the
CodeGenerator every time when the control is returned from a method
GCdefinition.generate. This method announces to the CodeGenerator that the compact
block of the IDL definitions had been generated and can be printed. The method
compactBlockDone returns the file into which the code can be printed.
        Almost every generated item is stored inside the instance of a semantic manager.
E.g., information about the generated exception can be stored only in the instance of the
ExceptionManager and so on.
       Every semantic manager has at least two basic methods: insertItem and getItem.
The method insertItem is used for storing the item and getItem is used for returning the
item. Returned items can be used as a part of some other item. E.g., name of enum is
used as a switch type in an union definition. Its enumerators are used in case labels.
getItem must return information about any item. A new item is generated if there is no
item stored in an instance of the manager. This process is called “pre-generating”.



                                           22
        There are at least two instances of a class SemanticManagement. One instance
is used for pre-generated items and one instance is used for items generated in a normal
way. Separating of these two types of items is a crucial rule. Some pre-generated items
can be used as a part of a normal item but any normal item must not be used as a part of
a pre-generated item. All pre-generated items used by a normal item are printed before
this normal item.

       6.3.2. The tester generator for C++
       A short description of some important classes follows. The tester generator uses
one group of classes with similar semantics. Classes, which belong to this group, are not
described individually if their behaviour is not different from other classes in the group.
Name of each group is underlined. More detailed description of classes is available in
“javadoc” documentation.
      TGTesterGenerator (abstract)
                     Main class of the tester generator; contains code independent on
                    target language.

      TGCPPTesterGenerator
                   Main class of the C++ tester generator. Programming language
                   of generated code is C++.

      TGGeneratedxyzInfo classes
                     Hold information about generated items. All of them extend
                     TGGeneratedItemInfo or some of its abstract subclass. xyz
                     indicates type of item.

      TGGeneratedItemInfo
                    Parent of all TGGeneratedxyzInfo classes; holds name, ID and
                    other common information.

      TGGeneratedItemTypedSubitemInfo(abstract)
                    Extends TGGeneratedItemInfo; parent of items which have
                    some sub-item. E.g., constant has type of constant.

       TGGeneratedItemWithArrayOfSubitemsInfo(abstract)
                        Extends TGGeneratedItemInfo; parent of items which have
                        array of sub-items. E.g., module has array of definitions.

      TGGeneratedLimitedFixedTypeInfo
                    Extends TGGeneratedLimitedStandardTypeInfo; fixed has size
                    of integer part and size of decimal part.

      TGGeneratedLimitedStandardTypeInfo
                    Extends TGGeneratedItemInfo; holds information about limited
                    types. E.g., string<20>.

       The mapping to C++ is specified in [3]. The tester generator is implemented in
an abstract class TGTesterGenerator and its non-abstract subclass called
TGCPPTesterGenerator.       Both       classes      implement       method      from
TesterGeneratorDirectAccessInterface. It works in two phases. Information about


                                            23
generated items is collected in the collecting phase. The generating phase begins when
all items are generated. The test applications (client and server) are generated during
this phase. The architecture of the tester generator is shown in Figure 10.




                                        Figure 10


            Phase 1 - collecting information
       This phase is independent on the target language and is completely implemented
in abstract class TGTesterGenerator. Method init is called at the beginning of this
phase. This method must be called before all other method calls. Method
generatingCompleted is called at the end of generating.
      Beginning of       every item generating is announced using                method
itemGeneratingStart.     End of generating is announced using                    method
itemGeneratingStop.
       void itemGeneratingStart(int itemID)
            throws TesterGeneratorException;

       The parameter itemID is unique for every generated item and is used in every
method which influences an item during its generating. The IDL generator also uses ID
when it refers to an item during the generating of another item. E.g., already defined
type is used in a member of a structure. There is only one exception. If an item is a
“forward declaration”, the IDL generator uses a method itemGeneratingStart_Forward
Declaration. Later defining of item begins using method itemGeneratingStart
_ForwardDefinition. Calling one of “start” methods creates a new instance of
TGGeneratedItemInfo. Items with special type announce its type by a method
setTypeOfItem. New instance of specific TGGeneratedxyzInfo is created
       Many method calls accessing the item can be performed between “start” call and
“stop” call of this item. These methods inform the tester generator about all significant
actions made with it. E.g., switch type of union is announced using method
setIDOfSubitem_Type. Description of methods from TesterGeneratorDirectAccess
Interface is available in a documentation produced by javadoc tool.




                                           24
             Phase 2 – generating of test applications
         The second phase begins in a method generateTargetLanguageCode. Abstract
class TGTesterGenerator does not implement this method. The method is implemented
in a class TGCPPTesterGenerator. Implementation of the method consists of opening
of the server file, calling a method generateServerTestingApplication, opening the
client file and calling a method generateClientTestingApplication.
        The method generateServerTestingApplication creates implementation of all
classes, which correspond to interfaces in the IDL definitions. Instances of all these
classes are created in the main function of the server. They are also announced to the
client side of application. This process is performed using references written in files.
Form of announcing is very similar to the form used in [4]. Serving of requests is started
at the end of the main function.
        The method generateClientTestingApplication generates the main function of a
client. References to all objects are obtained in an initial part of client’s main function.
These references are read from files created by the server side of application. The form
of obtaining the references is very similar to the form used in [4]. The method calls are
performed on all objects at the second part of client’s main function. The IDL attributes
are implemented as a pair of methods. These methods are also called during the second
part of client’s main function.

       6.3.3. Connector


       A connector adapts TesterGeneratorDirectAccessInterface for usage with
current implementation of the IDL generator. Current IDL generator produces items
recursively but the tester generator is implemented for direct access to any generated
items. The reason for direct access is a generality of an interface for most of possible
IDL generators. Better performance will be achieved if the tester generator is prepared
only for a current implementation of the IDL generator but there must not be many
requirements for behaviour of the IDL generator.
       The connector remembers ID of currently generated item and delegates
method’s calls on the tester generator using it. When a new item generating is started,
ID of currently generated item is stored on the top of the stack. When item is finished,
ID of currently generated item is restored from the stack. The architecture is shown in
Figure 11.




                                         Figure 11




                                            25
   6.4. User’s guide

       6.4.1. System requirements
       Operating system: The testing tool is written in the Java language. Java runtime
environment (JRE) 1.3.0 or its equivalent is required for running the testing tool. JRE is
available for many operating systems. See http://java.sun.com for details.
       Computer performance: PC with Pentium 150 MHz processor and 32 MB RAM
is an adequate configuration for most of the projects. Real requirements depend on
configuration settings. Simple projects (<20 KB of IDL definitions) require less than 5
MB of RAM. Complex projects (3 MB of IDL definitions) require less than 30 MB of
RAM. 500 MHz processor is recommended for projects with higher complexity.

       6.4.2. Configuration files

            General configuration file
       By default, config.txt in current directory is configuration file. Configuration file
can also be specified at command line.
       Required content of configuration file:
      debugOutput(boolean)=false
                     Required only for debugging purposes. Let this setting to
                     "false".

      idlFilename(text)
                          Name of file with generated IDL definitions.

      testerGeneratorClassName(text)="TGCPPTesterGenerator"
                       Classname of the tester generator. Current C++ tester generator
                       is implemented in TGCPPTesterGenerator.

      testerGeneratorConfigurationFile(text)
                      Name of configuration file for the tester generator

      identifierPrefix(text)
                         Prefix for all identifiers used in IDL definitions. “” means no
                         prefix. Using prefix is not required by the testing tool but can be
                         used for code lucidity for human reader.

      GeneratingControllerConfigFile(text)
                      Name of configuration filename for GeneratingController. “”
                      disables GeneratingController.

      randomSeed(integer)
                     Starting point of random number generator. This value can be
                     used for recurrent generating of same code.

      ignoreRandomSeed(boolean)
                    Set to true, if random generating is required




                                             26
     lexicalComplexity(integer)
                     Value from 1 to 10; controls complexity of IDL literals and
                     identifiers.

     syntacticalComplexity(integer)
                     Value from 1 to 10; controls syntactical complexity (which
                     types of syntactical constructs are generated) of IDL definitions.
                     GeneratingController can control what syntactical constructs
                     are generated.

     codeSize(integer)
                         Value from 1 to 10; size of generated code (number of
                         generated items, sizes of arrays and enums, etc)

     recursionDepth(integer)
                      Maximal depth of recursion during generating. E.g., how many
                      modules are nested.

     semanticComplexity(integer)
                    Value from 1 to 10; semantic complexity of code. E.g. using of
                    previously defined items from other modules instead of same
                    module, measure of generating of forward declarations.


           Configuration file for the C++ tester generator
     serverIncludeFile(text)
                      Name of header file included in server code

     serverOutputFile(text)
                      File with server code

     clientIncludeFile(text)
                       Name of header file included in client code

     clientOutputFile(text)
                       File with client code

     identifierPrefix(text)
                        Prefix for all identifiers used in C++ code. “” means no prefix.
                        Using prefix is not required by the testing tool but can be used
                        for code lucidity for human reader.


           Configuration file for GeneratingController
       This file is optional. It can contain values, which control the direction of the
generating process. It is necessary to choose which sub-item will be generated during
the generating of an item. It can be chosen randomly or it can be controlled by the
values contained in this configuration file.
       Example for the switch in class GCinterface follows.
       #GCinterface
       interfaceSettings_interface_dcl=70


                                          27
       interfaceSettings_forward_dcl=30

       6.4.3. Header files
        Both parts of test application require header files. The compulsory content of
these header files follows in the following paragraphs. There is no need for two header
files. The compulsory content of the server and client header files can be placed into
one header file, which is used together. Modification of the header files generated by
the IDL compiler can also be used.

         Compulsory content of both header files
     #define URI_MAX_SIZE
                 This value is used for the size of the array, which contains URI.

     #define REFERENCES_DIRECTORY
                 This directory is used for storing and loading the URIs. Similar
                 system is used in [4].

     #define MAX_PATH
                 This value is used for the size of the array, which contains path
                 of some file.

     #define BOOLEAN_VALUE_FALSE
                 False value of boolean type.

     #define BOOLEAN_VALUE_TRUE
                 True value of boolean type.

       Both header files must contain “null value” definitions for following types.
     #define      NULL_VALUE_WCHAR
     #define      NULL_VALUE_FLOAT
     #define      NULL_VALUE_DOUBLE
     #define      NULL_VALUE_LONGDOUBLE
     #define      NULL_VALUE_SHORT
     #define      NULL_VALUE_LONG
     #define      NULL_VALUE_UNSIGNEDSHORT
     #define      NULL_VALUE_STRING
     #define      NULL_VALUE_WSTRING
     #define      NULL_VALUE_CHAR
     #define      NULL_VALUE_BOOLEAN
     #define      NULL_VALUE_OCTET
     #define      NULL_VALUE_LONGLONG
     #define      NULL_VALUE_UNSIGNEDLONG
     #define      NULL_VALUE_UNSIGNEDLONGLONG

            Compulsory content of client header file
        The client header file must contain stub definitions. It can be #include of header
files generated by the IDL compiler.

            Compulsory content of server header file
       The server header file must contain skeleton definitions. It can be #include of
header files generated by the IDL compiler.



                                           28
           Example of header files for Orbacus IDL compiler
       The version of the IDL compiler is 4.1.0. It is used on Sun Solaris 2.7.
       Header file for client:
       #ifndef _includeC_h
       #define _includeC_h

       #include <OB/CORBA.h>
       #include "test.h"
       // Only for client
       #define URI_MAX_SIZE                         800

       // Client & Server
       #define REFERENCES_DIRECTORY
       "/home/strnad/references"
       #define MAX_PATH             300
       #define BOOLEAN_VALUE_FALSE                                        false
       #define BOOLEAN_VALUE_TRUE                                         true

       #define     NULL_VALUE_WCHAR                                       L'0'
       #define     NULL_VALUE_FLOAT                                       0.0
       #define     NULL_VALUE_DOUBLE                                      0.0
       #define     NULL_VALUE_LONGDOUBLE                                  0.0
       #define     NULL_VALUE_SHORT                                       0
       #define     NULL_VALUE_LONG                                        0L
       #define     NULL_VALUE_UNSIGNEDSHORT                               0
       #define     NULL_VALUE_STRING                                      "000"
       #define     NULL_VALUE_WSTRING                                     L"000"
       #define NULL_VALUE_CHAR
       (CORBA::Char)'0'
       #define NULL_VALUE_BOOLEAN        BOOLEAN_VALUE_FALSE
       #define NULL_VALUE_OCTET
       (CORBA::Octet)0
       #define NULL_VALUE_LONGLONG            OB_LONGLONG(0)
       #define NULL_VALUE_UNSIGNEDLONG             0U
       #define NULL_VALUE_UNSIGNEDLONGLONG
       OB_ULONGLONG(0)
       #endif

       Header file for server
       #ifndef _includeS_h
       #define _includeS_h
       #include "includeC.h"
       #include "test_skel.h"
       #endif

       6.4.4. Running the testing tool and using of generated code
        Test of the IDL compiler is performed in three levels (see section 6.2). Next
instructions are for 100% CORBA-compliant IDL compilers, which pass all tests. Next


                                           29
test phase cannot be performed if there are some errors in the previous phase. If some
test phase fails it usually indicates a bug in the IDL compiler.
          1) Generating of IDL definitions and C++ code are started with command
             “jre IDLCompilerTestingTool” or “java IDLCompilerTestingTool”.
             Three files are created as an output. Their names depend on
             configuration settings. E.g., they can be “test.idl”, “client.cpp” and
             “server.cpp”. File test.idl is used as an input for the IDL compiler.
             Starting command of the IDL compiler is variable and depends on its
             vendor. Starting of the IDL compiler depends on an operating system.
          2) Several files are produced as an output of the IDL compiler. There is one
             or more C++ source files. These files have to be compiled to the object
             files.
          3) Files client.cpp and server.cpp have to be compiled to the object files too
             (e.g., client.o and server.o). Client object files have to be linked together
             and server object files have to be linked together. Server and client are
             being run.




                                          30
7. Summary of the work



   7.1. Fulfilling the goals
        No universal testing tool is able to find all the bugs in the IDL compilers. The
definition of mapping is quite strict but there are many possible ways how to implement
a supplemental code. Only these tools, which are specialized on a concrete
implementation of the IDL compiler, have a chance to discover all the bugs in the IDL
compiler. In spite of universality of the testing tool the used method of testing the IDL
compilers is very effective. It found several bugs in commercially distributed IDL
compilers. These bugs are described in Appendix A .
        The IDL generator produces a code with variable complexity. The complexity of
the generated code is adjustable in four ways: lexical complexity, syntactical
complexity, semantic complexity and depth of recursion. The GeneratingController
class can be used for generating of specified syntactical constructs. The IDL generator
is portable because it is implemented in the Java language.
        The compiler of the target language verifies that the mapping produced by the
IDL compiler conforms to the target language specification. The tester generator
generates test application source code, which tests the code for appropriate content and
its behaviour during execution.

   7.2. Comparison with other known methods

            CORBA validation project (CORVAL2)
       The CORBA validation project is the project focused on CORBA
implementation testing. The CORVAL2 is focused on three main areas: IDL compiler,
syntax and semantics of CORBA APIs and GIOP and IIOP Protocols. The information
about this project can be found in [6], [7] or at http://www.opengroup.org/corval2.
        The CORVAL2 uses similar technique of testing the IDL compilers. The IDL
compiler translates the IDL definitions. The Stubs and skeletons are compiled using the
target language compiler. The compiled code is used by test application.
       There is a difference between the CORVAL2 and the implemented testing tool.
The CORVAL2 does not contain any IDL generator or the tester generator. There is a
set of tests containing the IDL files and corresponding server and client files. The
CORVAL2 can be used only for tests of IDL to C++ or IDL to Java compilers.

   7.3. Future work
        Current implementation of the testing tool allows full testing of the IDL to C++
compilers. The architecture of the testing tool allows testing of all IDL compilers
independently on the target language. The tester generator has a form, which allows
quite an easy implementation of a new tester generator for other target language. The
collecting phase of the tester generator does not need any modification and can be used
for all target languages. Using of this part in the new tester generator is not necessary.



                                           31
The only constraint for the tester generator is demand for implementation of
TesterGeneratorDirectAccessInterface.
       The IDL generator does not generate all syntactical constructs, which are
described in the IDL specification [1]. Main generated constructs are: const_dcl,
except_dcl, interface, module, union, struct, enum, operation and attribute in
interface etc. All not generated syntactical constructs are: ValueBase, any, object,
fixed, context_expr, value, abstract and local interfaces, unsigned integers,
interface_inheritance_spec, bounded (w)string, “native” type_dcl and typedef.




                                        32
8. Conclusion


       The correctness of the IDL compiler behaviour is one of the key requirements
for the development of distributed applications on the CORBA platform. The
implemented testing tool can help the vendors of the IDL compilers to improve their
software and fulfil this requirement. Other key requirements are the CORBA-
compliance of the ORBs and their interoperability. Testing of these requirements is
outside the scope of this thesis but the testing tool can help a bit. There is a possibility to
use this tool together with IIOP Engine For Testing CORBA Implementation
Robustness as it is written in section 4.4.




                                              33
9. Bibliography



[1]   OMG (2001): OMG IDL Syntax and Semantics (http://www.omg.org/…/01-12-
  41.pdf)

[2]   OMG (1999): The Common Object Request Broker: Architecture and
  Specification (http://www.omg.org/…/99-10-07.pdf)

[3]    OMG (1999): C++ Language Mapping Specification
  (http://www.omg.org/…/99-07-41.pdf)

[4]    Distributed Systems Research Group (2001): Open CORBA Benchmarking -
  server and client code recommendation (http://nenya.ms.mff.cuni.cz/~bench)

[5]    Sun (1993-2001): Java 2 Platform, Standard Edition, v 1.3.1 API Specification
  (http://java.sun.com/j2se/1.3/docs/api/index.html)

[6]    The Open Group (2001): Validating Multi-Vendor CORBA Conformance and
  Interoperability in Heterogeneous Environments
  (http://www.opengroup.org/corval2/D29_Jan2002.pdf)

[7]    The Open Group (2001): The Conformance Test Suite for CORBA v2.3
  (http://www.opengroup.org/corval2/D32-f_13Nov01.ppt)

[8]   OMG (2001): OMG IDL Syntax and Semantics (http://www.omg.org/…/99-07-
  07.pdf)

[9]   OMG (2001): OMG IDL Syntax and Semantics (http://www.omg.org/…/00-10-
  39.pdf)




                                         34
Appendix A                Discovered bugs in the IDL compilers


        The testing tool was used mainly on two compilers: MICO and ORBACUS.
Other compilers were not tested enough. As it is written above bugs can be detected at
three levels. All following bugs were reported to the vendors and I hope they help
improve the implementation of their IDL compilers.

    Understanding of IDL

           MICO and attributes
       Mico 2.3.6 is confused with more than one read-only attribute declaration at
same line.
         // IDL
         interface TestInterface {
                 readonly attribute short A1, A2;
         };

         Output of the IDL compiler:
         test.idl:2: parse error before ','


         Mico-mt 2.3.6 has same problem with both types (readonly, read-write) of
         attributes.
         Mico 2.3.7 does not have any problem with multi-attributes.

            MICO and unions
         Mico 2.3.6 and 2.3.7 is confused with the type declaration in more than one case
label.
         union MyUnion switch(char) {
                 case 'J':
                 case '5':
                         struct MyStructure {
                                 long myVariable0;
                         } myVariable1;
                       default :
                               double myVariable2;
         };

         Output of the IDL compiler:
         test.idl:6: identifier 'MyStructure' defined twice in this scope




                                              35
             ORBACUS and forward declarations of constructed types
        Orbacus 4.1.0 is confused with the forward declaration of constructed type
(structure, union). The forward declaration of structures and unions are introduced in
the IDL specification from the year 2000 [9] at the first time. The previous IDL
specification [8] does not know it.

            ORBIX 2000 and forward declarations of constructed types
        Orbix 2000 1.2.1 is also is confused with the forward declaration of constructed
type (structure, union).

          OMNIORB and long double
       Omniorb 4.0.0-020101 does not know long double. Output of the compiler
follows:
       test.idl:1466: Warning: long double constant truncated to double. Sorry.

   Conformance with C++ specification

          MICO and multidimensional arrays
      Mico 2.3.6 and 2.3.7 generates code, which cannot be compiled by C++
compiler.
       exception TestException{
               long testArray[37][26];
       };
       is mapped to:
     TestException::TestException( const CORBA::Long _m0[
37 ][ 26 ] )
     {
       for( int _iter_testArray = 0; _iter_testArray < 962;
_iter_testArray++ )
         testArray[ _iter_testArray ] = _m0[
_iter_testArray ];
     }

       Output of the C++ compiler:
       incompatible types in assignment of `const long int[26]' to `long int[26]'


       Suggested solution:
     TestException::TestException( const CORBA::Long _m0[
37 ][ 26 ] )
     {
       memcpy(testArray, _m0, 962 /*962 = 37 * 26*/ *
sizeof(CORBA::Long));
     }

          ORBACUS and unions (1)
       Orbacus 4.1.0 generates code, which cannot be compiled by C++ compiler.


                                           36
       if(_ob_d_ == OB_LONGLONG(754081661))
               else if(_ob_d_ == OB_LONGLONG(2142194374))
               else if(_ob_d_ == OB_LONGLONG(1055031698))

CORBA::string_free(_ob_v_.IDL_identifier1065);

       There is no statement between two else if.
       This error occurs only when some union is defined in the IDL definitions but not
always.

   Execution errors

           ORBACUS and unions (2)
       This is the same error as the previous but the C++ compiler does not detect this
occurrence because it conforms to the C++ specification.
       if(_ob_d_ == OB_LONGLONG(754081661))
               else if(_ob_d_ == OB_LONGLONG(1055031698))

CORBA::string_free(_ob_v_.IDL_identifier1065);
             else

                   _ob_i_ = false;

        The last line is not the statement in else but it has to be performed always. This
error causes server fail. The output follows:
       non-compliant application detected:
       uninitialized union used
       assertion failed: _ob_i_

            MICO and some methods calls
       Running of complex application fails during some method calls. Methods with
wstring return type usually fails. The Orbacus has no problem with the same code but
Mico announces the exception:
       IDL:omg.org/CORBA/MARSHAL:1.0 (0, maybe-completed)




                                           37
Appendix B         Content of enclosed CD


   /docs
                   This folder contains this text and the javadoc documentation.

   /src
                   This folder contains all source codes of the testing tool.

   /classes
                   This folder contains all implemented classes used during
                   execution of the testing tool. There are also examples of
                   configuration files.

   /bibliography
                   This folder contains most of the sources from chapter 9.




                                       38

								
To top