plugin by stariya

VIEWS: 15 PAGES: 39

									Table of Contents
Part 1:      A simple plug-in architecture pattern for C++ applications on Win32 [1] .................................... 4
  The Problem .............................................................................................................................................. 4
  The Solution .............................................................................................................................................. 4
  High-level design ....................................................................................................................................... 5
  Low-level design........................................................................................................................................ 5
  The code.................................................................................................................................................... 6
  Other enhancements .............................................................................................................................. 12
Part 2:      DynObj - C++ Cross Platform Plugin Objects [2] ......................................................................... 13
  Introduction ............................................................................................................................................ 13
     Background - Problem Area ................................................................................................................ 14

     A Plugin Approach ............................................................................................................................... 14

     Middle Ground .................................................................................................................................... 15

  A C++ Class .............................................................................................................................................. 15
     VTables ................................................................................................................................................ 16

     C++ Inheritance ................................................................................................................................... 16

     Common Ground Defined ................................................................................................................... 18

     The Run-Time Link............................................................................................................................... 18

     Plugin Object Creation ........................................................................................................................ 18

     Plugin Object Destruction ................................................................................................................... 19

     Linking Revisited ................................................................................................................................. 20

  Solution ................................................................................................................................................... 20
     Cross-Platform .................................................................................................................................... 20

     Cross Compiler .................................................................................................................................... 20

     C++ Classes used Across DLL/SO Boundary ........................................................................................ 20

     Object Model ...................................................................................................................................... 21

     C++ Type Query ................................................................................................................................... 21
   Arbitrary Types and DynI Derived Types............................................................................................. 21

   Simple Type Identifiers ....................................................................................................................... 22

   Plugin Role .......................................................................................................................................... 22

   Light-Weight........................................................................................................................................ 22

   Facilities............................................................................................................................................... 22

   Source Code Preprocessor .................................................................................................................. 23

   With other Languages ......................................................................................................................... 23

   Requirements ...................................................................................................................................... 23

A Sample: Plugin + Application ............................................................................................................... 24
   Creating an Interface File .................................................................................................................... 24

   Creating an Implementation File ........................................................................................................ 26

   A Main Application.............................................................................................................................. 29

The DynObj Library ................................................................................................................................. 30
   VObj, DynI, DynObj and Friends.......................................................................................................... 31

   The VObj Class..................................................................................................................................... 31

   The DynI Class ..................................................................................................................................... 32

   The DynObj Class................................................................................................................................. 33

   The DynSharedI Class .......................................................................................................................... 33

Building and Using the Library ................................................................................................................ 34
   Build Model ......................................................................................................................................... 35

   Compiler Defines................................................................................................................................. 35

   Building the DynObj library ................................................................................................................. 35

   Building a Main Application (Using Plugins/Modules)........................................................................ 36

   The DynStr Library............................................................................................................................... 36

   Building the Samples ........................................................................................................................... 36

   The pdoh Tool ..................................................................................................................................... 37
Part 3:      Send data to/from C++ and C# Windows processes [3] ............................................................. 37
  Introduction ........................................................................................................................................... 37
  Why use WM_COPYDATA? .............................................................................................................. 37
  Platforms supported ............................................................................................................................. 38
  Other facts ............................................................................................................................................. 38
   Part 1: A simple plug-in architecture
    pattern for C++ applications on Win32 [1]
By George Mihaescu



Summary: This article presents a simple and elegant solution to creating components that can be
dynamically deployed and loaded by a C++ Win32 application without the need of any
framework / infrastructure or technology (such as COM). It relies on basic C++ mechanisms and
two commonly used Win32 API calls.



Download the code here (VC++ projects + source).



Note: code snippets in the document and code available for download are now based on VS .Net
2005 / CRT 8.0

The Problem
You have a C++ Win32 application to which you want to be able to “attach” components
dynamically, developed either by yourself or by other parties. In the context of this document I
will call those components plug-ins, by analogy with the well-known web browser add-on
components. In the same manner with the browsers, you want your application to be aware of
such plug-ins that were developed and deployed possibly long after the application itself was
developed and deployed. Ideally, you don’t want the application to even need to restarted – what
you’d like is to drop the plug-in at a know location (e.g. somewhere under the application
installation directory) even as the application is running, and the application all of a sudden has
enhanced functionality. Can this be done in a simple and reliable way?

The Solution
Many readers will immediately dismiss the problem and this article by saying “of course, that’s
what COM is all about”. But I’m not in favor of using COM unless there is a very compelling
reason for it. Generally, if it can be done without COM (without making the code overly
complex – I don’t want to re-implement COM), then why bother with COM? Say COM and you
say code complexity, registration issues (such as registration requiring certain user security
privileges), dependency on registry, application usually needing to be re-started, etc. I argue
below that I can meet the requirements of this problem without COM, in a much simpler and
reliable way.
High-level design
My solution is based on marrying the C++ polymorphic mechanism with the Win32 APIs
LoadLibrary and GetProcAddress.



The principle is that the main application publishes a contract with the plug-ins in the form of an
interface that the plug-ins are expected to implement in order to meet the contract. Then the main
application will scan a known location on disk (e.g. a “plugins” directory relative to its
installation directory) and attempt to load all plug-ins (using LoadLibrary API) that export
implementations of this interface (determined by using the GetProcAddress API). Each plug-in
is packaged in its own DLL (or multiple DLLs) that must be deployed at the location the
application expects them in ordered to be detected and loaded by the application.

Low-level design
As C++ does not offer a language construct to model the concept of an interface, we will use the
next best thing available: an abstract base class.

Also, because the Win32 API GetProcAddress uses a function name as a parameter, while our
plug-ins are C++ (because they need to provide a concrete derivative of the abstract base class)
and because C++ does function name mangling, our plug-ins will need to export as a minimum
one C-style function to act as the class factory. To keep things balanced and because the
application has (in theory) no way of knowing what allocation strategy each plug-in factory
function uses, it only makes sense to ask plug-ins to also export the counter-part of the factory
function, another C-style function to act as the plug-in clean-up / tear-down procedure.



So, to sum it up:

 program has an abstract class through which it will use all dynamically
            The main
  loaded plug-ins. It also has a few lines of code to scan a known location and look for DLLs
  that export two known C-style functions: the plug-in factory and the plug-in clean-up.

  Each plug-in has a class implementing the abstract class in the main program, and
  is packaged as a Win32 DLL exporting two C-style functions: the plug-in factory and the
  plug-in clean-up.
This is illustrated in the diagram below. As you can see, there is no registration required, no need
for the user to have special privileges on the machine, no framework / runtime dependency other
than what you already have: C++ and Win32.




The code
Below is a sample “contract” (abstract class) in the main program that plug-ins will need to
implement. Of course, the methods of this class are going to be specific to what your plug-ins
need to do:



//////////////////////////////////////////////////////////////////////////

// Abstract base class ("interface") for the concrete plugin implementations




class IPlugin

{

public:

      //Add whatever functions each plugin needs to implement

      //Those below are just dummy examples to illustrate the principle
      //returns the name of the concrete plugin

      virtual const char* Get_Name () const = 0;




      //does the actual data processing

      virtual void Process_Data () = 0;

};




/////////////////////////////////////////////////////////////////////////

//Extern "C" functions that each plugin must implement in order to be

//recognized as a plugin by us.




// Plugin factory function

//extern "C" IPlugin* Create_Plugin ();




// Plugin cleanup function

//extern "C" void Release_Plugin (IPlugin* p_plugin);




Below is the code in the main program that scans for plug-ins, determines that they are indeed
exporting the two C-style functions it expects from a plug-in, then loads and executes each plug-
in found:



#include "IPlugin.h"               //for the IPlugin abstract base




//convenience typedef for the pointers to the 2 functions we

//expect to find in the plugins
typedef IPlugin* (*PLUGIN_FACTORY)();

typedef void (*PLUGIN_CLEANUP)(IPlugin*);




int main(int argc, char* argv[])

{

    //get the program's directory

    char dir [MAX_PATH];

    ::GetModuleFileName (NULL, dir, MAX_PATH);




    //eliminate the file name (to get just the directory)

    char* p = ::strrchr (dir, '\\');

    *(p + 1) = 0;




    //find all DLLs in the plugins subdirectory

    char search_parms [MAX_PATH];

    ::strcpy_s (search_parms, MAX_PATH, dir);

    ::strcat_s (search_parms, MAX_PATH, "plugins\\*.dll");




    WIN32_FIND_DATA find_data;

    HANDLE h_find = ::FindFirstFile (search_parms, &find_data);

    BOOL f_ok = TRUE;

    while (h_find != INVALID_HANDLE_VALUE && f_ok)

    {

         //load each DLL and determine whether it is exporting

                      //the functions we care about

         char plugin_full_name [MAX_PATH];

         ::strcpy_s (plugin_full_name, MAX_PATH, dir);
::strcat_s (plugin_full_name, MAX_PATH, "plugins\\");

::strcat_s (plugin_full_name, MAX_PATH, find_data.cFileName);




HMODULE h_mod = ::LoadLibrary (plugin_full_name);

if (h_mod != NULL)

{

    PLUGIN_FACTORY p_factory_function =

 (PLUGIN_FACTORY) ::GetProcAddress (h_mod, "Create_Plugin");

    PLUGIN_CLEANUP p_cleanup_function =

 (PLUGIN_CLEANUP) ::GetProcAddress (h_mod, "Release_Plugin");




    if (p_factory_function != NULL &&

                        p_cleanup_function != NULL)

    {

        //yes, this DLL exposes the 2 functions we need,

                                //it is a plugin we can use!




        //invoke the factory to create the plugin object

        IPlugin* p_plugin = (*p_factory_function) ();




        //show which plugin it is, and let the plugin

                                //do the processing

        printf ("Now working with plugin: %s\n",

                                                      p_plugin ->Get_Name ());

        p_plugin ->Process_Data ();




        //done, cleanup the plugin by invoking its
                                                         //cleanup function

                        (*p_cleanup_function) (p_plugin);

                  }




                  ::FreeLibrary (h_mod);

            }




            //go for the next DLL

            f_ok = ::FindNextFile (h_find, &find_data);

      }




      return 0;

}




And finally, here is the code for one such plug-in:



#include "stdio.h"




#include "..//MainProgram//IPlugin.h"




////////////////////////////////////////////////////////////////////////

// A concrete plugin implementation

////////////////////////////////////////////////////////////////////////




// Plugin class

class Plugin1 : public IPlugin
{

public:

     //returns the name of the concrete plugin

     const char* Get_Name () const

     {

          return "Plugin1";

     }




     //does the actual data processing

     virtual void Process_Data ()

     {

          for (int i = 0; i < 3; i++)

          {

              printf ("Plugin 1 is processing....\n");

          }

          printf ("Plugin 1 processing done!\n");

     }




};




extern "C"

{

     // Plugin factory function

     __declspec(dllexport) IPlugin* Create_Plugin ()

     {

          //allocate a new object and return it
        return new Plugin1 ();

    }




    // Plugin cleanup function

    __declspec(dllexport) void Release_Plugin (IPlugin* p_plugin)

    {

        //we allocated in the factory with new, delete the passed object

        delete p_plugin;

    }




}




But wait: what about the promise that the user won’t even have to re-start the application after
deploying a new plug-in? For that, just throw in a couple more Win32 APIs: as the application
starts, create a low-priority thread that calls FindFirstChangeNotification /
FindNextChangeNotification / WaitForSingleObject or WaitForMultipleObjects and this thread
can notify every time a valid plug-in DLL is deployed at the known location – so that the
application can act (start using the plug-in / ask user whether to enable the plug-in, etc.).

Other enhancements
        As mentioned above, in most cases if you want the application to dynamically sense
         when plug-ins are deployed and run them without having to re-start, you will need a disk
         monitoring thread like the one described above.
        You will probably want to implement versioning on your program’s contract with the
         plug-ins. After all, it’s very likely that your plug-ins interface will evolve over time, and
         you want the program to be able to ignore / reject plug-ins that were not written for its
         version of the contract (e.g. user deploys plug-ins written for a more recent version of the
         program on an older version of the program and vice versa). Such a versioning protocol
         can be implemented in the abstract class that represents your program’s contract with the
         plug-ins, so that the program can decline using plug-ins that don’t conform to its
         versioning requirements.
        Sometimes the plug-ins may need to add to the application’s help (CHM) files. I will not
         get into the details, but it can be done quite easily if the main program’s help file is
         properly written for file merging. If the plug-in is deployed together with its help file, the
         Windows help engine can automatically merge the main program’s CHM help file with
         each of the plug-ins CHM help files, resulting in a seamless user experience. Maybe I
       will address this in another article – until then, Google “Merging Help Files at Run
       Time”. I have done this and I know it works fine without any pain.



I have implemented this pattern since 1998 with great results. One of the free programs available
from this site (daVinci) uses this architecture to implement parsers for different file formats. As a
user needs a parses for another file format, we implement it and make the parser available for
download on our site. The user downloads the parser and deploys it under the “parsers”
subdirectory of the application (without even closing the application), and all of a sudden the
application can handle the new file format.




   Part 2: DynObj - C++ Cross Platform
    Plugin Objects [2]
By Arne Steinarson

Introduction
DynObj is an open source library that provides a C++ application with run-time class loading
facilities (AKA plugins). It's written in standard C++, and can be used with any C++ compiler
with good template support, so it is cross-platform from the outset. It uses a minimal platform
specific layer to handle OS specifics (Win32, Linux + various Unix supported now).

The project started out with me needing a way to support plugins in a cross platform application.
The approaches I found were either too heavy weight (Modzilla XPCOM) or were
platform/compiler specific. An article by Chad Austin provided a good starting point for the
DynObj library.

This article will cover some ground, so it is split in a number of sections:

      Introduction - This page. General description of the problem area.
      Background - Explores C++ classes and the linking process.
      Solution - Describes the DynObj library.
      Sample - A sample plugin together with a main application.
      Library documentation - Documentation for the DynObj library.
      Building - Documentation for building the DynObj library.
Background - Problem Area

C++ is a very feature rich language and within the same link time module (all sources and
libraries) functionality can be exposed and shared without much difficulty. You may need to get
some link flags right, but it can be done, with templates and the whole C++ machinery operating.

This can be extended to run-time with shared libraries (DLLs on Windows, shared dynamic
object files (SO) on various Unices).

However, it quickly becomes rather difficult, since there is an intricate linking process going on
between the application and the loaded modules. With C++ name mangling and automatically
generated templates and a rich set of compiler/linker options, dependencies become complicated.
With a code base under heavy development, linking with a a DLL compiled on a different
system a month ago is not likely to work.

This approach usually assumes that the same compiler is used for host and library. To link the
binaries, often the compiler version must be similar. You could not expect to link together a
template or class library compiled with G++ with an application compiled with a Microsoft
based compiler. You'd have problems doing it with a version of the same compiler from a year
ago.

The approach described above can be termed a 'tight' or a 'full' linking scheme. C++ has never
had it easy to make its internal features available to the outside world in a standard way. Old
style:

 Collapse

extern "C" int FunctionToExport(...)

has often been the way (and yes, this works reliably but can only expose global functions and
variables).

A Plugin Approach

A plugin is a more decoupled run-time library, supposedly not dependent on the main application
being a certain version, and preferably the requirements on the plugin should be a bit more loose
(they shouldn't need to know about every header file and #define in the application, and they
need not mutually resolve massive amounts of global symbols).

C++ doesn't provide any language or standard library support for this, but there are some good
starting points in the language.

What can be investigated is what parts of the language can be used without creating link-time
complexity, and what parts to be avoided for a plugin. Maybe one is not confined to only extern
"C" ....
On the Windows platform, COM is a way forward. To be language neutral, it limits (severely)
what features of C++ can be exposed and then re-used inside another run-time module (including
expressing inheritence relationships). It is based on a particular way of sharing the VTables for
run-time binary objects. However, it works, but has not evolved in a cross-platform direction.

Middle Ground

What we're looking at is if there is some larger middle ground that can be defined, between (A)
the compile time process when the full C++ feature set (and lots of header files) are be shared,
and (B) the run-time DLL loading where only type-less symbols can be looked up in a loaded
module (extern "C"...")

The next section will define this middle ground and later introduce an object framework
(DynObj) around it.

A C++ Class
We have this class definition:

 Collapse

class AnExample {
public:
    AnExmaple( int i ) : m_i(i) { }
    virtual int Add( int i );
    int Sub( int i );
    int Get( ){ return m_i; }
    bool operator < (int i){ return m_i
    static int StaticFunc( );
protected:
    int m_i;
};

For linking (which is the key issue in dynamic loading), this applies to the members of
AnExample:

      The constructor AnExample(int i) is inline so it does not generate any linking
      The virtual function int Add(int i) is an entry in the VTable of the class. To the compiler
       and linker, it is an index in this table.
      The function Sub(int i) does generates linking. It refers to a function implemented
       somewhere else.
      Get() is an inline function and does not generate linking.
      operator < is also an inline function (no linking).
      The StaticFunc() member requires linking. This is true for any static member.
      The member variable m_i is a type and offset into the binary object. It does not generate
       linking.
So, it seems it's only static members and functions that are both non-virtual and non-inline
member that generate any linking!

VTables

To clarify things here a bit we should remember what virtual functions are:

      Each class (that has one or more virtual functions) has a unique virtual function table (VTable).
      The VTable is per type, not per instance.
      A pointer to the VTable (if the object has one) is always stored first in the object. It is often
       referred to as the VPTR.
      A virtual member function is at run-time an index into this VTable where the address of the
       function to use is stored.
      A derived class has a copy of the base class VTable first in its own VTable. The new functions it
       introduces are stored at the end of that copy.

The VTable in itself is in itself a symbol located inside a DLL. However, if an object is
instantiated from inside the DLL/SO that owns the VTable, this is not part of the link process
either.

So it seems we have found some middle ground.

C++ Inheritance

We have a class that uses inheritance:

 Collapse

class SideBaseClass {
public:
    virtual const char *GetPath( );
    virtual bool SetPath( const char *path );
};

class MultiBaseClass : public AnExample, public SideBaseClass {
public:
    virtual int Add( int i );
    virtual bool SetPath( const char *path );
protected:
    int m_j;
};

MultiBaseClass    has two base classes and overrides a function from each base class. From a
run-time perspective, the inheritance boils down to the following object layout for an instance of
MultiBaseClass:


word offset 0 VPTR for MultiBaseClass
word offset 1 m_i (base class data)

word offset 2 m_j (derived class data)

word offset 3 VPTR for SideBaseClass


The VTable for MultiBaseClass will be:

slot 0 MultiBaseClass::Add(int i)

slot 1 SideBaseClass::GetPath( )

slot 2 SideBaseClass::SetPath( const char *path)


This object data layout can vary with compiler and data type (compilers have settings for
padding and alignment). However, this is constant also between compilers:

    1. The VPTR (if any) is always stored first in the object.
    2. The first virtual function in a base class always occupies slot 0 in the VTable.
    3. Subsequently declared virtual functions stored in order of declaration.

For derived classes, the following applies:

    1. A derived class inherits its default VTable from its first base class that has a VTable (call it main
       base class).
    2. New virtual functions are appended (in order of declaration) at the end of the default VTable.
    3. For other base classes with VTables (side bases), a full copy of the object is stored at an offset
       into the binary object, including a copy of side base class VTable.
    4. The VTable of side bases can be modified (functions are overridden) but it cannot be extended.

This is the main picture, there are a couple of exceptions to above rules. They are:

    1. The ordering of overloaded functions in the VTable does not correspond with declaration order
       for some compilers (MSVC among others, for historical reasons).
    2. Virtual destructors are handled in different ways by different compilers. They cannot be directly
       invoked across a compiler and DLL/SO boundary.

A central part of the DynObj framework is to account for offsets between multiple bases
generated by potentially different compilers.

What is important to recognize here is that: "Inheritance (neither single nor multiple) does not
generate any linking."
Common Ground Defined

A class definition:

      with one or more (non-template, non-virtual) base classes
      with any number of virtual functions
      with operators that are either virtual or inline
      and any inline function
      with any non-static data members

can be reused and implemented by both a host application and a plugin. Furthermore, they can
use each other's implementations of these classes. Code made by one compiler may use such a
class compiled from another.

For function members, this works all the time, also in the cross-compiler case. For data
members, it works as long as the compilers on each side have used the same sizes and padding
for the data members.

When moving (casting) between base classes, the address offset must at all times be calculated
based on offsets generated by the source (plugin) compiler.

This common ground is of course in addition to the old extern "C" MyFunction(...) style.
That is, however, an important part that we will make use of below.

The Run-Time Link

We know now that instances of a class fulfilling above spec can be used by both the host
application and the plugin, without requiring a linking process. But how do we handle creation
and destruction of object instances?

Since a call across the plugin boundary is essentially typeless, we cannot communicate the type
directly to the plugin as a C++ type.

Plugin Object Creation

Say that from the host, we want the plugin to create an instance of AnExample:

 Collapse

AnExample *pe = new AnExample; // This doesn't work

This would just make the host instantiate it.

Since we don't have the compile time type machinery here, (remember, we're communicating
with a binary module possibly from another compiler), we have to communicate the type to the
plugin in some other way.
To solve this problem, we use a factory function in the plugin that takes the type encoded as a
string and an integer:

 Collapse

extern "C" void* CreatePluginObject( const char *type_name, int type_id )
    if( !strcmp(type_name,"AnExample") &&
        type_id == ANEXAMPLE_TYPE_ID )
        return new AnExample;
    else return NULL;
}

Then on the host side we can do:

 Collapse

typedef void* (*PluginFactoryFn)(const char *type, int id);

// First locate CreatePluginObject inside a loaded DLL

PluginFactoryFn create_fn = /* Locate it */;

// Now create object

AnExample *pae = (AnExample*)create_fn( "AnExample",
        ANEXAMPLE_TYPE_ID );

The DynObj framework automates this conversion step so that we can use the expression:

 Collapse

AnExample *pe = do_new<AnExample>; // This works

to instantiate objects from inside plugins. The conversion:

      C++ type => (type string,type ID)

is taken care of by templates classes available in the host application.

Plugin Object Destruction

There are some points to consider here to keep cross-compiler compatibility:

      We cannot be sure that the host and the plugin share the same memory allocator (on Windows
       this is often not the case). So using C++ delete on plugin objects is not a good idea.
      Virtual destructors are used in different ways by different compilers.

Essentially, the host must make sure a plugin object is 'recycled' by the same plugin that created
it. To handle this, the DynObj framework has used a solution where each object that is created
has a virtual member function doDestroy():
 Collapse

DynObj *pdo = /* Create object and use it */;
pdo->doDestroy(); // End of object

We see here that we have used DynObj as a base class for objects that are created by a plugin.

The DynObj framework works with any classes, but objects that can be created and destroyed by
plugins must derive from DynObj.

Linking Revisited

The solution with factory functions gives the responsability of setting up the VTable to the
plugin, and so, all the functions we need from the plugin are contained in these pre-linked
VTables. Each instantiated object comes back with a VPTR as its first binary member.

The only run-time linking we have to do is to lookup these factory functions inside the plugin
DLL (and possibly some other init/exit functions). This keeps the host and the plugin in a loosely
coupled relationship, defined by the plugin interface. Next comes the description of the DynObj
solution using this approach.

Solution
This describes the properties of the DynObj library solution to the plugin/linking problem.

Cross-Platform

The library is written in C++, and a decent C++ compiler should build it (tested with MSVC 8
and G++ [4.1.2 and 3.4.5]). It relies on a minimalistic cross-platform layer for dynamic linking
and a spartan threading interface.

Cross Compiler

The library/plugin compiler can be a different one than the main application compiler. All
casting between types is allways done based on offsets from the source (library) compiler.

C++ Classes used Across DLL/SO Boundary

DynObj supports ordinary C++ classes across the plugin boundary. Any class that consists of:

      Zero, one or more base classes/interfaces
      Virtual functions (argument overloading not supported)
      Inline functions
      Operators
      Data members (keep track of member alignment!)
So a fairly large subset of the C++ class concept can be used over the boundary. This is what
cannot be used:

      Non-virtual member functions implemented in a separate source file
      Static members (functions,data)

Object Model

The object from a plugin represents a full C++ object, including the possibility of having
multiple nested base classes. At the source code level, a tagging scheme is used to decide which
bases to expose. The whole (exposed part) of the inheritance tree is communicated to users of the
object.

The object is usually accessed using a single inheritance interface/class. Using a cast operation
(query type) one can move between the different interfaces/sub-objects that are implemented.

C++ Type Query

An object can implement a number of interfaces and/or classes. To query an object for another
type, the C++ template:

 Collapse

template<class U, class T> U do_cast(T t)

is used. It operates the same way as C++ dynamic_cast<> and provides typed safe casts across
the plugin boundary. do_cast (and related functions) provides similar functionality to
QueryInterface in COM.

An example:

 Collapse

DynI pdi = /* Basic DynI pointer from somewhere */;
DynSharedI pdsi = do_cast<DynSharedI*>(pdi)

Arbitrary Types and DynI Derived Types

The library introduces a small interface and class collection, based on DynI (a class which knows
its own type and can be queried for other types). Both classes based on DynI and arbitrary classes
with virtual methods may be used across the plugin boundary.

When using classes derived from DynI, a separate registration step may be skipped, since a DynI
object always knows its own type.
The provided classes derived from DynI also provides for a certain way of instantiating and
destroying objects (DynObj), for handling objects with shared ownership (DynSharedI), and also
for weak references.

When using arbitrary classes, they must have at least one virtual member function. The library
provides templates that safely detect if an object has a vtable or not. To use such objects across a
plugin boundary, one instance of the type must be registered first.

Simple Type Identifiers

Types are identified based on the pair:

      Type string
      Type identifier (32-bit integer)

This is a simple scheme that does not guarantee global (world-wide) type uniqueness. It can,
however, guarantee that the types used inside the application are unique. It is always simple to
find the string name for types. In cast operations, usually only the type integer is carried around
(no 128 bit ID structures).

Most times we don't need to know these, we just use the C++ types (which in their turn use the
type strings/IDs when needed).

Plugin Role

Plugins can use types from the main application (as long as it has headers for it) and also from
other loaded plugins. It can also instantiate plugin objects (from itself, other plugins, or the main
app).

Light-Weight

The library is self-contained and relatively small, including the cross-platform layer. A
compressed archive of the source is around 200 KB. It does not rely on STL, Boost or any other
big component library. It is not tied to a single platform API.

Facilities

The library includes a collection of practical classes, to handle libraries, object
instantiation/destruction, smart pointers and more.

Optionally (and recommended) one can use the class DoRunTimeI, which provides shared
resources to the application and the plugins. Among other things it makes sure that the various
libraries access the same type information, it provides for a pool of named 'published' objects,
per-object and per-thread error handling.
A run-time string class, DynStr (in itself a plugin object) is provided, giving plugins a way to
deal with Unicode strings.

Source Code Preprocessor

To setup a C++ class as a plugin type, some registration needs to be done and a library file must
be created. To help with this, a tool pdoh (Parse DynObj Header) is used. It reads C++ source
file and triggers on // %%DYNOBJ tags in the source code.

The pdoh tool outputs most of the glue code that is needed, including generating type ID:s.

With other Languages

The library relies on the default way of using vtables in C++ together with a binary type
description structure. This is a simple binary scheme. So, plugin classes could be used from any
language that can use these. A C implementation is straight forward (an object would be a
structure with the first member being a pointer to an array of functions). Also, a plugin class
could be implemented in another language and used from C++.

Inline functions cannot be shared with another language (they are really compiled on both host
and plugin side).

Requirements

The library relies on these features from the C++ compiler:

      It uses vtables in the default way (one pointer per function, first function at index 0, new
       functions are stored in declaration order)
      Support for extern "C" style of exposing non-mangled function names
      Support for __cdecl function calling convention

When a library is compiled, this information is stored and made available at load time, so an
incompatible library can be detected. Virtual destructors are not used across plugin boundaries,
since compilers implement them in slightly different ways. Some earlier versions of g++ (prior to
version 2.8) used two slots per function in the VTable, that would not have been compatible.

When exposing data members in a class across a plugin boundary, the best is to make each
member fill up one word (32/64-bit) in the structure. That avoids any possibility of unaligned
data access. The size of an exposed type (using sizeof from the plugin compiler) is stored in the
type information. The user of a plugin class could detect if data members are aligned differently.

The calling convention can be configured when the library is compiled, some other convention
could be used as long as the main and plugin compiler agree on it. On Linux, the default
(implicit) calling convention is __cedcl. Next: A sample using the DynObj library.
A Sample: Plugin + Application
Here we will create a couple of plugin libraries and use them from a simple main application. It
wil demonstrate how to instantiate plugin objects, how to use plugin objects as ordinary C++
classes, how to query for supported types.

Creating an Interface File

We start out with defining a simple interface file that manages data about a person (PersonI.h):

 Collapse

#include <string.h> // We use strcmp below

class DynStr;
// %%DYNOBJ class(DynI) <---Directive to pdoh preprocessor

class PersonI : public DynObj {
public:
    // DynI methods <---Implement GetType and Destroy - for all DynObj:s

     virtual DynObjType* docall doGetType( ) const;
     virtual void docall doDestroy( ) { delete this; }

     // PersonI methods <---Add our new methods

     virtual const char* docall GetName( ) const = 0;
     virtual int docall GetAge() const = 0;

     virtual bool docall SetName( const char *name ) = 0;
     virtual bool docall SetAge(int age) = 0;

     // ---Simple default inline implementation of operator

     virtual bool docall operator<( const PersonI& other ) const {
         return strcmp(GetName(),other.GetName()) < 0;
     }

     // ---Non-virtual, inline convenience function

     // Derived cannot override.

     PersonI& operator=( const PersonI& other ) {
          SetAge( other.GetAge() );
          SetName( other.GetName() );
          return *this;
     } };

Then, from a command prompt/shell, we run the pdoh preprocessor on this file (the -o option
tells the parser to write generated code directly into the file instead of to stdout):

 Collapse
$ ./pdoh PersonI.h -o
$

Looking at the header file, we see that a section at the beginning of the file has been added:

 Collapse

// %%DYNOBJ section general

// This section is auto-generated and manual changes will

// be lost when regenerated!!



#ifdef DO_IMPLEMENT_PERSONI
#define DO_IMPLEMENTING // If app has not defined it already

#endif
#include "DynObj/DynObj.h"


// These are general definitions & declarations used

// on both the user side [main program]

// and the implementor [library] side.


// --- Integer type ids for defined interfaces/classes ---

#define PERSONI_TYPE_ID 0x519C8A00

// --- Forward class declarations ---

class PersonI;

// --- For each declared class, doTypeInfo template specializations ---

// This allows creating objects from a C++ types and in run-time casts

DO_DECL_TYPE_INFO(PersonI,PERSONI_TYPE_ID);

// %%DYNOBJ section end

This section provides the glue needed to convert from a C++ PersonI type to the type strings
and type ID:s that are used across the plugin boundary.

If we would like to move this section we're free to do that. The next time the preprocessor is run
on the same file, it will keep the section where we put it.

We also see that code has been inserted at the end of the file:

 Collapse
// %%DYNOBJ section implement

// This section is auto-generated and manual changes

// will be lost when regenerated!!

// ... comments


// Define the symbol below from -only one place- in the project implementing

// these interfaces/classes [the library/module].

#ifdef DO_IMPLEMENT_PERSONI

// Generate type information that auto-registers on module load

DynObjType
g_do_vtype_PersonI("PersonI:DynObj",PERSONI_TYPE_ID,1,sizeof(PersonI));
// DynI::doGetType implementation for: PersonI

DynObjType* PersonI::doGetType() const {
    return &g_do_vtype_PersonI;
}
#endif // DO_IMPLEMENT_...

The preprocessor has inserted code to do two things:

      Declare a DynObjType structure for our type.
      It provides a default implementation of doGetType() for our class

When we define the symbol DO_IMPLEMENT_PERSONI from a C++ source file, the code above
ends up in that file.

Creating an Implementation File

Next we create a source file that implements the interface (PersonImpl1.cpp):

 Collapse

// This will cause PersonI class registration info to come in our file.

#define DO_IMPLEMENT_PERSONI
#include "PersonI.h"


// We're also implementing our class

#define DO_IMPLEMENT_PERSONIMPL1
The defines above puts the class registration code into this source file. Each interface/type that is
handled must be declared as a global registration structure once. The defines DO_IMPLEMENT_...
correspond to a class we're implementing in this file.

 Collapse

// Declare the class to the pre-processor.

// %%DYNOBJ class(dyni,usertype)

class PersonImpl1 : public PersonI {
public:

Here we tell the preprocessor that a plugin class is being defined. The flag usertype informs it
that this class can be instantiated by the host. Therefore it must generate a factory function for
this type in the library section.

 Collapse

// DynObj methods

virtual DynObjType* docall doGetType( ) const;
virtual void docall doDestroy( ) { delete this; }

// PersonI methods

virtual const char* docall GetName( ) const {
    return m_name;
}
...

The above implementats functions in DynObj and PersonI. Since we are inside a class
PersonImpl1 which definintion is never is exposed, we can generate the function bodies inside
the class definition.

 Collapse

// Constructor, Init from a string: "Bart,45"

PersonImpl1( const DynI* pdi_init ) : m_age(0) {
    // We will do this setup slightly awkward now, and improve in

     // the following examples.

     *m_name = 0; // NUL terminated

     ...

The constructor for a DynObj always take a simple argument of type const DynI*. Since DynI
can implement any interface, we can pass pretty much any type of data to the constructor. To
pass simple data like int/double/const char* and friends, one can use an instance of template
class DynData<T>.

 Collapse

protected:
   // Need not really be protected since user of PersonI cannot look here
anyway.

   char m_name[NAME_MAX_LENGTH];
   int m_age;
};
// %%DYNOBJ library

The last comment tells the preprocessor that we want library code inserted at this location. In this
library section it will put the factory functions and any glue needed to instantiate plugin objects
to the host.

Next we run the parser on this source file (the -p option tells pdoh where it can find template
code):

 Collapse

$ ./pdoh PersonImpl1.cpp -o -p../
Found library insertion point
$

The parser has now inserted code that generates glue for library functions. The glue code can be
included/excluded using #define DO_MODULE:

 Collapse

// %%DYNOBJ section library

...
// Only include below when compiling as a separate library

#ifdef DO_MODULE
...
// The object creation function for this module

extern "C" SYM_EXPORT DynObj* CreateDynObj(
    const char *type, int type_id,
    const DynI *pdi_init, int object_size )
{
    ...
        if( ((!strcmp(type,"PersonImpl1") || type_id==PERSONIMPL1_TYPE_ID))
||
            ((!strcmp(type,"PersonI") && type_id==PERSONI_TYPE_ID)) ){
                return new PersonImpl1(pdi_init);
            }
            DO_LOG_ERR1( DOERR_UNKNOWN_TYPE, ... );
               return 0;
}

After compiling, we can connect this as a plugin to a host application, the preprocessor has
generated the bits and pieces that are required, both for the host and the plugin side.

A Main Application

Finally we create the main application (main1.cpp) that uses the plugin:

    Collapse

#include <stdio.h>

#include <stdlib.h>


// DynObj support

#include "DynObj/DynObj.h"

#include "DynObj/DynObjLib.h"

#include "DynObj/DoBase.hpp"


// Interfaces we're using

#include "PersonI.h"

#include "ProfessionI.h"


int main( ) {
    // Check that things have started OK

       if( !doVerifyInit() ){
           printf("Failed DoVerifyInit\n");
           exit(-1);
       }

Now we want to start using the plugin. For this, we use DynObjLib which wraps a cross-
platform run-time (DLL/SO) library loader (initial code for this came from Boby Thomas).

It is worth noting that the main application and the library are loosely linked, making it easy to
implement on any platform that supports explicit run-time loading of binary libraries.

    Collapse

// Load library

DynObjLib lpimpl1( "pimpl1", true );
if( lpimpl1.GetState()!=DOLIB_INIT ){
    printf( "Could not load library (pimpl1): status (%d)\n",
        lpimpl1.GetState() );
    exit( -1 );
}
// Create object

PersonI *pp = (PersonI*)lpimpl1.Create("PersonI",PERSONI_TYPE_ID);
if( pp ) {
    pp->SetName( "George" );
    pp->SetAge( 34 );
    ...

We have instantiated the object the 'raw' way here, giving type name and ID to DynObjLib. After
that, it is just to start using the object as any standard C++ object.

We next query the object for an interface using do_cast:

 Collapse

ProfessionI *pi = do_cast<ProfessionI*>(pp);
if( pi )
    ;// Use the interface


pp->doDestroy();
return 0;
}

The template do_cast<T> takes care of the details of checking if ProfessionI is supported. It
transforms the C++ type to type name and ID. Using type information from the plugin, it can
walk the class layout and return an adjusted interface pointer.

It is important that any address offsets applied inside objects are always based on information
from the plugin compiler. When using an interface pointer returned in this way, we can only
assume it is valid for the duration of the current function. We do not need to release it in any
way. Finally, we delete the object using DynObj::doDestroy (which will recycle it in the
memory manager of the plugin that created it).

Further documentation here.

The DynObj Library
DynObj is an open-source cross-platform library that uses the run-time plugin approach just
decribed. Although the mechanisms used are generic and fairly simple, the library fills many
gaps and make it straight-forward to use plugins inside a C++ application.

The library provides:
       A small class hierarchy (VObj, DynI, DynObj, DynSharedI) establishing some common
        ground between a host and a plugin. The DynObj library also works with classes that are not
        rooted in this hierachy.
       A type description facility that allows types to be defined and shared by both host and plugins
        (DynObjType).
       A way to convert C++ types to a plugin library (doTypeInfo, DO_DECL_TYPE_INFO).
       Cast functions to query an object about the types it implements. This is similar to
        dynamic_cast<T> in C++ or QueryInterface in COM (do_cast<T>, doGetObj,...).
       Instantiating C++ objects from plugins (do_new<T>).
       A plugin library loading/unloading mechanism(DynObjLib).
       C-level functions to handle objects.
       Other practical C++ classes and templates for objects.

In addition to these library facilities, it includes a tool (pdoh) that parses C++ header files and
generates source code for type registration.

VObj, DynI, DynObj and Friends

All classes discussed below are defined in DynObj.h. The library is based on some properties of
objects with VTables:

       The VPTR is always stored first in a binary object
       VTables are shared by all instances of a class, but not with instances of any other class (so it
        provides a type identifier).

In C++ there is not a built-in way to denote these classes. However, we define the class VObj to
represent an object with a VTable with unknown size and methods. VObj in that sense becomes
the 'stipulated' root class for all classes that contain one or more virtual functions.

The VObj Class
Base:       (no base class)

Methods: (no methods)


We see that VObj does not introduce any methods (it cannot since that would interfere with
derived classes which use the first VTable slot).

However, VObj has a number of convenient inline functions to query for types (VObj::IsA,
VObj::CanBeA, VObj::GetObj,...), asking about errors (VObj::Get/Set/ClearError) and
more.

To determine if a class is a VObj or not, these templates can be used:

 Collapse
bool has_vtable = IsVObj<SomeType>::v;

template<class T>
VObj* to_vobj( T* pt );

This provides type safety so that we cannot try to convert say a char* to an interface pointer (the
compiler would give an error).

The DynI Class
Base:            VObj

Returns          Methods:         Arguments

DynObjType* doGetType

void*            doGetObj         const char* type_name

const char* doGetError            int *perr_code

void             doClearError



The DynI class provides a way to know its type (doGetType) and for asking about other types it
supports (doGetObj). To ask if a DynI supports the DynStr interface:

 Collapse

DynI *pdi = /* Wherever pointer comes from */;
DynStr *pds = pdi->doGetObj("DynStr");

This is equivalent to:

 Collapse

DynStr *pds = do_cast<DynStr*>(pdi);

The DynI class has an advantage over VObj:

         It knows its own derived type

In contrast, to find the type of a VObj, a lookup into a global table, using the VPTR, has to be
made (and works only after the types have been registered).

Since DynI is used across DLL (and possibly compiler) boundaries, we cannot use C++
exceptions. To provide error handling, the methods doGetError and doClearError are
introduced. They allow for an object specific error state, without burdening the class with
member variables for this. SetError is not a member, since object errors are usually are not set
from 'outside'.

We see also that the DynI interface has no support for creation or destrcution. The same applies
to VObj. The lifespan that can be assumed is that of the current method.

If a reference to the object is to be kept, these are different ways to go about it:

       Ask for a DynSharedI interface (ref counted ownership)
       Create a weak reference (if object supports NotifierI))
       The object may be a known global or singleton which explicitely allows for references to be
        stored

The DynObj Class
Base:   DynI

Returns Methods:     Arguments

void    doDestroy


void    doDtor



The DynObj interface represents an object that can be created and destroyed. It represents an
object owned from a single point. Usually the functions doDestroy and doDtor would be
implemented like:

 Collapse

void MyClass::doDestroy(){ ::delete this; }
void MyClass::doDtor(){ this->~MyClass(); }

An object is destroyed through doDestroy. doDtor provides access to the destructor of the class
(library internal use).

Objects can be created in some different ways:

       Using do_new<T>
       Using DynObjLib::Create(...)
       Temporary objects can be created and released using DynObjHolder<T>

The DynSharedI Class
Base:   DynObj
Returns Methods:         Arguments

int       doAddRef


int       doRelease



The DynSharedI interface represents an object with shared ownership. doAddRef and doRelease
increases and decreases the ownership counter.

DynSharedI derives from DynObj since it depends on a way of          destroying itself
(DynObj::doDestroy) when the lifetime counter reaches 0.

To protect the object from being deleted before its actual end-of-life, a doDestroy method can
check that the counter is actually zero:

 Collapse

virtual void docall doDestroy( ) {
    if( !m_ref_cnt )
         ::delete this;
    else
         SetError(DOERR_DESTROY_ON_NON_ZERO_REF,
         "DynSharedI - Destroy on non-zero ref");
}

Documenation on building the DynObj library.

Building and Using the Library
Directory layout of the library:

         doc
                o   doxygen - docs generated from doxygen
         src
                o   DynObj - main source of DynObj library
                          samples - samples of using library here
                          tools - source for the pdoh tool
                          msdev - Visual C++ project for DynObj and samples here
                o   pi - sources for platform independence layer
                o   utils - utility like C++ classes

A description of the pdoh tool is available here.
Build Model

Plugins are usually opened while the application is running, so there is no build-time link
between the main application and the plugins. When a plugin module is loaded, by default, the
linker is instructed not to backlink into the application (UNIX). On Windows, backlinking is not
possible.

An application can expose functionality to plugins through interfaces. DoRunTimeI provides a
way to make named instances of objects known globally.

Compiler Defines

A number of compilers define controls how libraries and main applications are built. The defines
are described in detail in src/DynObj/DoSetup.h (and under DynObj defines in doxygen doc).

When compiling a library DO_MODULE should be defined. Also, the name of the library should be
stored in DO_LIB_NAME (i.e. #define DO_LIB_NAME "DynStr").

The main application should define DO_MAIN.

The default settings in DoSetup.h are OK when compiling the samples. In general the defines
are used like this (example DO_USE_DYNSHARED):

      #define DO_USE_DYNSHARED - The option is not used
      #define DO_USE_DYNSHARED 0 - The option is not used
      #define DO_USE_DYNSHARED 1 - The option is activated

This allows having sensible defaults and for overriding them reliably from outside in a build
environment.

Building the DynObj library

There are two build methods provided with the library:

      Cross-platform GNU makefile - This works for the g++ compiler on Unices and Windows
       (mingw).
      Visual C++ - Solution and project files for for Visual C++ on Windows.

Building a Plugin (Module)

The compiler define DO_MODULE should be set.

A plugin module requires compiling with:

      DynObj.cpp
         vt_test.cpp

Building a Main Application (Using Plugins/Modules)

The compiler-defined DO_MAIN should be set. The main application links against one of two
static libraries:

         dolibdort - Enables using DynObj:s and DoRunTime
         doliblt - A minimalistic library without support for DoRunTime

The libraries are projects in the Visual C++ solution file. To build the libraries using the
makefile:

    Collapse

$ cd src/DynObj
$ make dolibdort
$ make doliblt

The DynStr Library

A run-time plugin class for strings is provided: DynStr. This enables plugins to use a C++ string
class in a safe way, internally and in function calls. The DynStr library is built with:

    Collapse

$ cd src/DynObj
$ make ds

Building the Samples

The samples defines a PersonI and ProfessionI interfaces. Then three slightly different
implementations are provided in three plugins: pimpl1, pimpl2, pimpl3. Three different main
applications are provided as well: main1, main2, main3. There are sub-projects for each of
them in the VC++ solution file.

From the command prompt:

    Collapse

$    cd samples
$    make pimpl1
$    make pimpl2
$    make pimpl3
$    make bmain1
$    make bmain2
$    make bmain3
The pdoh Tool

This tool takes a C++ header or source file and outputs a modified version of the file, provided it
finds // %%DYNOBJ tags in it. It basically scans for class and struct declarations and collects
inheritance information.

The pdoh tool can generate these sections:

       A general section (in a header file). This part is used by both the plugin and the main
        application. It contains class declarations, type IDs and the bridge from C++ types to type IDs.
       An implement section (by default in a header file). This part is used only by the plugin. To keep
        things simple, the code is generated inside the header file (that keeps things in one place) and
        the plugin must trigger inclusion of this section with a #define DO_IMPLEMENT_NAMEOFLIB.
       A library section. This goes into a source file and makes up the part that the user of the plugin
        communicates with directly. The most important function is the one that receives a type ID/
        type string and instantiates this object to the caller.

By default pdoh sends its output to stdout. Use option -o to overwrite the input file, or -
oNewFile.h to write to another file. To generate less comments in the output -b can be used.

pdoh  can be integrated into a build environment, it is a simple file scanner so it is fast. If it does
not find any //%%DYNOBJ tags it will not generate any output.

If the tags have not been modified since the previous run (on the same file), it will also not
generate any output (so it does not affect file time stamps when there is no need for it).


    Part 3: Send data to/from C++ and C#
     Windows processes [3]
Introduction
This demo demonstrates how to send / receive WM_COPYDATA messages between a collection of
C++ and C# programs.

Why use WM_COPYDATA?
Traditionally WM_COPYDATA was used to send a limited amount of data between processes on the
same machine. It is still desirable to interoperate this way even with the rise of .NET. .NET
Remoting is out of the scope of this demo. This demo is intended for people like myself that are
dealing with legacy environments.
The demo consists of four programs and one DLL. Two programs are written in C++ (MS
Version 5.0) and two are written in C# (using .NET 2003). A C++ exe and C# exe will send the
structure shown below to a C++ and C# receiving program.

 Collapse
struct sTagData
{//this is the structure that will be sent (copied) by the WM_COPYDATA

private:
    char szTag[ciMaxTag];
    char szData[ciMaxData];
}

The DLL was written in VC 5.0 and it wraps the WM_COPYDATA message.

Additionally, the DLL provides a uniform way of :-

   a. Protecting the receiving application from third party processes that may be unwise
      enough to HWND_BROADCAST the WM_COPYMESSAGE.
   b. Allows incoming messages to be limited to specific senders either by handle or sender id.
   c. Optional use a crude bitwise encryption to discourage hackers from spying on messages.

Unzip using the full paths to your C drive. The result should be the shown directory structure.

      C:\WmCpyDta
      C:\WMCpyDat\Build - the folder where all build output is placed for all projects.
      C:\WMCpyDat\From_C - a C++ program that sends the structure
      C:\WMCpyDat\TO_C - a C++ program that receives the structure
      C:\WMCpyDat\From_C_Sharp – a C# program that sends the structure
      C:\WMCpyDat\\TO_C_Sharp - a C# program that receives the structure.

To use the demo, open folder build and click on all 4 executables. Click on the send buttons in
the FROM apps and the message will appear in the TO apps. Both FROM programs are
document to show how to use the DLL’s interface. Likewise, both TO programs are documented
in their WindowProc()’s to show how to receive the data.

Platforms supported
These programs have only been tested on Windows XP.

Other facts
Philip McGahan is the sole creator of this demo. No part of it has been plagiarized from other
sources. Any party may use all or part of this demo with out credit or reference to Philip
McGahan.
Lastly, it is only fair to tell my reader that I am very new at C#. I did try to use C# to build the
DLL. But beyond never getting the data to come across with out junk at the end of the string. I
was not comfortable with having to setup unsafe blocks. The whole thing just seemed clumsy.
When or if I get better at C#, I will revisit this issue.




Reference
   [1] http://www.abstraction.net/ViewArticle.aspx?articleID=67

   [2] http://www.codeproject.com/KB/library/dynobj.aspx

   [3] http://www.codeproject.com/KB/cs/wm_copydata_use.aspx

								
To top