Technical White Paper Definiens Cellenger Architecture: A Technical Review April 2004 Technical White Paper Definiens Cellenger 2 Contents Product Overview............................................................................................................................................... 3 Product Positioning ........................................................................................................................................... 3 Product Design Concept ................................................................................................................................... 3 Applications........................................................................................................................................................ 5 Histopathology ................................................................................................................................................... 5 High-Content Screening.................................................................................................................................... 6 Basic Research................................................................................................................................................... 6 Functional Description ...................................................................................................................................... 7 Cellenger Developer Studio ............................................................................................................................... 7 Cognition Networks -the Basis of the Cellenger Meta Language..................................................................... 7 Input Data......................................................................................................................................................... 8 Image Object Hierarchy.................................................................................................................................... 8 Features ........................................................................................................................................................... 8 Classes and Classification ............................................................................................................................... 9 Processes......................................................................................................................................................... 9 Integrated Development Environment............................................................................................................. 10 Cellenger Enterprise ........................................................................................................................................ 10 Modular Client Server Architecture ................................................................................................................. 10 Open Architecture for Flexible Workflow Integration....................................................................................... 11 System Requirements...................................................................................................................................... 11 About Definiens................................................................................................................................................ 12 Contact .............................................................................................................................................................. 12 Technical White Paper Definiens Cellenger 3 Product Overview Industrial and academic life science research relies more and more on detailed and consistent analysis and quantification of image data. Modern optical hardware and supporting systems can quickly generate more data than can be interpreted manually, resulting in a considerable bottleneck. This is especially true for images used in pharmaceutical and clinical environments. Cellenger provides quantitattiv results even in high-throughput environments, automating previously manual work that produced only qualitative results. Product Positioning Biomedical imaging covers a large variety of different images and applications. Structures at different scales of genes, proteins, cells and tissues are imaged using very different technologies. Technologies for image acquisition have made large progress during the last years. IHC stains, GFP, fluorescence and confocal microscopes deliver new insights into the secrets of life. Although image acquisition, as well as image storage and management can be automated to a high degree, this is not true for the automated image analysis. Automated image analysis is available only for a small number of applications that tend to be very specific and rely on high image quality. Therefore many approaches for automating image analysis workflow chains aim to increase image quality and reduce complexity of the image analysis itself. The large success of fluorescence readers are a good example for this approoach On the other hand the challenge of automating biomedical image in all it’s variety and in-depth understanding that human beinng can gain from images is a yet unsolved problem. Fluorescence Cell Images H&E stained tissue (liver & kidney) EM images from the liver Cellenger is the first software that offers the flexibility and the ease-of-use necessary to develop image analysis solutions even for complex tasks including noisy, low-contrast data and questions that require semantic understanding of the image data. Cellenger is designed to meet the needs of pharmaceutical, clinical and academic research. Definiens provides customer tailored solutions that are specifically suited to common problems in the industry. Cellenger solutions cover the application fields of cell biology, histopathology and diagnostics. Product Design Concept Definiens’ core technology was designed to model and simulate the human cognition process. Based on this technology, called Definiiens Cognition Network Technology, object-oriented image analysis has been successfully used in the field of remote sensing for many years. Cellenger is the newest generation product developed from this technology, and addresses specific image analysis needs for life sciences. Technical White Paper Definiens Cellenger 4 The Cellenger Product Suite offers an integrated development environment for building fast and intuitive customer-tailored image analysis solutions using an interactive interface. A separate running environment provides all features required to fully automate image analysis solutions and report generation. Definiens Cellenger enables customers to automate complex image analysis tasks based on Definiens’ unique technology. Cellenger Developer Studio provides the Cellenger Meta language with an interactive graphical user interface. It allows users to easily build automated image analysis solutions without programming knowledge. The development is carried out in direct interaction with the image data. The developed solutions, called Cellenger rule sets, can be saved for automated image analysis in a distributed running environment. Definiens also provides ready-made solutions for some problem domains. Cellenger Enterprise enables ready-to-use image data processing. It allows quick access and management of stored images, result data and rule sets. Whole image series can be automatically analyzed using a single mouse click. After automatic image analysis statisttica reports can be generated and selected properties of individual images and entire image series can be exported. Furthermore the user can navigate through the regions of interest in each image, allowing detailed examination of the results. Database handling, image processing and data exchange is managed by different server modules of Cellenger Enterprise. All server components, as well as the comprehensive GUI client are designed to form a scalable and distributed high performance image analysis environment. Based on the power of the unique Cellenger Meta Language, the Enterprise System can be upgraded with any solution created using the Cellenger Developer System. Technical White Paper Definiens Cellenger 5 Applications Histopathology With Cellenger, Definiens has revolutionized image analysis in maximizing interactive software support in such a way that has not been seen before. The unique features of Cellenger’s analysis are of paramount importance in executing the histopathologist’s most primary yet unsolved task: the extraction of distinct tissue structures and the computation of staining properties (or other parameters of interest) within these structures. Cellenger applications have been developed to automate the computation of the proliferation indices using PCNA staining in liver and in jejunum tissue as shown below. A lot of error-prone manual work was previously required in performing large-scale toxicology studiees Cellenger is able to automatically analyze thousands of images and at the same time is able to account for variations in image quality as well as in structural variations. To this end, Cellenger distinguishes proliferating and resting endothelial cells in rat jejunum based on morphometric and neighborrellate characteristics, rather than on color information. Histopathology: Proliferation index using PCNA stained rodent jejunum tissue. Crypts with best sectioning were detected and the proliferation index as well as morphometric properties of the crypts were extracted. The Cellenger PCNA application responds to the above requirements and calculates the PCNA index for the individual crypts within the jejunum section. As stated previously, Cellenger imitates basic processes of human visual image perception and operates by following a step-by-step calculation in which the results are prioritized according to importance and relevance. As such, Cellenger identifies those crypts with a median cut along the lumen -the “best” crypts according to standardization requirements-and ignores crypts sliced in the periphery. Cellenger automatically delivers succinct, comprehensive and objective final results. This Cellenger application: · finds and extracts the best crypts reliably · measures morph metrical features of the “best” crypts (area, length, width) · finds and extracts PCNA stained and unstained nuclei within the crypt area · automatically exports statistics for all best crypts per scene Technical White Paper Definiens Cellenger 6 High-Content Screening The early phases of drug discovery necessitate cell screening assays which are run in automated high-throughput environments. Until recently this highly advanced processing chain lacked an image analysis tool which was both reliable and adjustable. A: Wild type B: Compound 1 C: Compound 2 Now with Cellenger customized application packages Definiens is able to deliver solutions for both high-throughput and high-content screening approaches in one solution. Cellenger applications fulfill the needs of modern screening approaches by analyzing and quantiffyin multiple sub-cellular events in one single assay. Furthermore, the analysis is performed in a fully automated mode. This provides our customers with a detailed but fast multi-parameter analysis and can be smoothly integrated into the R&D automation chain. As in this Cell Cycle Application, the analysis can include as many parameters as are provided by sample preparation and microscopic imaging equipment: Here, apoptotic and proliferating cells caught in naturally different cell cycle stages are simultaneously detected and quantified. Cellenger separates the individual cells independent from absolute intensity values and predefines cell masks. It combines and interprret morph metric information (cell shape, area, borders) with the specific staining signals originating from the applied cytosceletal and nuclei antibody as well as apoptotic and proliferative markers. Thus, a detailed result in terms of proliferative and apoptotic events is given for each single cell as well as for specific cell types. A: Wildtype B: Compound 1 C: Compound 2 Technical White Paper Definiens Cellenger 7 Basic Research In contrast to traditional image analysis approaches, Cellenger works reliably and highly independently of image quality (signal-to-noise ratio, texture etc.), image resolution and overall spectral information. These are vital demands for the analysis of low contrast TEMmicroograp which are easily fulfilled by Cellenger. Cellenger automatically identifies ultra-structural features such as different cytoplasmmi organelles in the TEM-micrograph, for example mitochondria, nucleoli or the endoplasmic reticulum. Cellenger calculates morphommetri and relational attributes for each single structure. Cellenger carries out a step-by-step extraction of coarse features within the images and within these broad regions of interest; the desired ultra structural objects are classified iteratively. The results, from coarse to fine resolution: · original image (Figure 1A) · the whole sinusoid (Figure 1B) · the sinusoidal lumen, lining endothelial cells, red and white blood cells, thrombocytes (Figure 1C) · different cytoplasmatic organells of the hepatocytes (Figure 1D) Functional Description Cellenger Developer Studio The core of the Cellenger Developer Studio is formed by Image Analysis Engine in conjunction with the Cellenger Meta Language. The Cellenger Meta Language is a concrete realization of a Cognition Network with access to the Image Analysis Engine. It was perfecctl adapted to fit the needs of image analysis and it provides a new and revolutionary approach to this familiar problem. In contrast to pixel based approaches it adds several new aspects to image analysis, like the representation of the image data by image objects, the provision of intuitive support for local adaptive processing and the incorporation of image semantics and domain specific expert knowledge. Cognition Networks -the Basis of the Cellenger Meta Language The basis of the Cellenger Meta Language forms the Definiens core technology called Cognition Network (CN). A Cognition Network is able to extract, represent and store knowledge from a complex input like images or texts. The knowledge stored in a Cognition Network is represented by the networked structure of linked objects. All objects of a CN may carry various kinds of data and may be linked by link objects. Since links are objects themselves, any link may carry data and may be linked itself to other objects. Technical White Paper Definiens Cellenger 8 There are two basic types of objects: objects representing the concrete data input, and model objects representing the knowledge about the given task. The sub-network of model objects is called model network. In the beginning of the cognition process the model network creates a separate hierarchical input object structure (instance network) from the input data (like pixels or words). This initial instance network is linked during the cognition process into the model network by a classification procedure. The state of both, the model network and the instance network linked into it, is called a network situation. The situation is modified by the procedural objects until a final state is reached. This final network situation represents the knowledge extracted from the input data. Input Data Cellenger uses a very generic model for representing image data. Images may have an arbitrary number of layers. Each layer may have a different data type (RGB, 8Bit unsigned, 16Bit signed/unsigned, 32Bit float, etc). Images and image layers are treated as objeect within the instance network. Image Object Hierarchy The image data in Cellenger is represented as image objects. Image objects represent connected regions of the image. The pixels of the associated region are linked to the image object with an “is-part-of” link object. Two image objects are neighbored to each other, if their associated regions are neighbored to each other. The neighborhood relation between two image objects is represented by a speciia neighbor link object. The image is partitioned by image objects; all image objects of such a partition are called an image object level. The output of any segmentation algorithm can be interpreted as a valid image object level. Each segment of this segmentation result defines the associatte region of an image object. Two trivial image object levels are the partition of the image into pixels (the pixel level) and the level with only one object covering the entire image (the scene level). Image object levels are structured in an image object hierarchy. The image object levels of the hierarchy are ordered according to inclussion The image objects of any level are restricted to be completely included (according to their associated image regions) in some image object on any “higher order” image object level. The image object hierarchy together with the image forms the instance cognition network that is generated from the input data. Features Features are arbitrary numbers, which can be computed by a well defined algorithm from the current network situation. (Reading a data entry is also considered as a computation). There are two major types of features: object features, which are linked to an object in the Cognition Network, and global features, which can be any kind of other information. Image object features measure properties of the individual image objects. Since regions in the image provide much more information than single pixels, there is a large number of different image object features for measuring color, shape and texture of the associated regions. Even more information may be extracted by taking the network structure and the classification of the image objects into account. Important examples of this type of features are the “relative border to neighboring objects of a given class” and “the number of sub objects of a given class”. Global features describe the current network situation in general. Examples are the “mean value of a given image channel”, the “number of levels in the image object hierarchy” or the “number of objects classified as a given class”. Global features may also represent metadata as an additional part of the input data. Technical White Paper Definiens Cellenger 9 The sun stand in an aerial photo or the type of tissue in a toxic screen might be expressed via metadata objects and thus incorporated into the analysis. Process Variables are stored data values. Each variable is linked to an according feature. This feature provides “read access” to the variable value throughout the system. Variable values may be used and modified by procedural objects and may be linked to any other object in Cellenger. Classes and Classification Class objects describe the semantic meaning of other objects in the cognition network. Classes can be linked by inheritance links to inherit class descriptions and group links to group different classes together to a group class. Class descriptions are created via a fuzzy logic based system. The classes form a structured sub-network of the CN called the class hierarchy [3,4]. Image objects are linked to class objects by classification link objects. Each classification link stores the fuzzy membership value of the image object to the linked class. An image object may have an arbitrary number of classification links. The class with the highest membership value for the image object is called the current class of the image object. The image objects of class C is the set of all image objects with current class C. Classification can be performed with any classification algorithm as long as the results may be translated into fuzzy membership values. The current implementation uses fuzzy membership functions on image object features and a Nearest Neighbor Classifier. Since process variables may be used for class descriptions, these can be modified by the cognition network itself during the image analysis. Processes The network situation is modified by processes. Processes are linked by flow-control link objects to describe the order of process execution in time. During process execution each process holds a temporary execution context object that stores all information related to the process execution state. A process is the combination of an algorithm and an image object domain. Processes may have an arbitrary number of sub-processes. The algorithm describes what the process will do. Examples for algorithms are classification, creating image objects (segmentation) or image objects modifications like merging, splitting of an image object or rearranging sub-objects of an image object. Other important algorithms are computing and modifying attributes (see 2.6) and exporting results. The image object domain describes where the algorithm and the subproccesse of the process will be executed in the image object hierarchy. They are defined by a structural description of the according subset. Examples for domains are an image object level or all image objects of a given class. By applying the usual set operators to the basic domains many different domains can be generated. Since during process execution image objects of a domain are treated one after the other, image objects domains may be defined relative to the current image object of the parent process (PPO); e.g. the sub objects or the neighboring objects of the PPO. Technical White Paper Definiens Cellenger 10 Integrated Development Environment The Developer Studio enables the user to create Cellenger Rule Sets in an interactive click-and-drop Interface. The full power of the Cellenger Meta Language is accessible without the need to learn a complicated programming language. The result of each step of the rule-set development is visualized immediately in the comprehensive image display area. The interactive behavior of the Cellenger Developer Studio enables the users to create powerful solutions with speed. Cellenger Enterprise Cellenger Enterprise automates image analysis and report generation using the rule sets designed with the Cellenger Developer Studio. It is designed as modular client server system, which offers API’s at various levels for smooth integration into existing workflow chains. Modular Client Server Architecture The Cellenger Enterprise is composed of several server components: Cellenger Image Analysis Server is a server version of the Cellenger Image Analysis Engine. It performs automated image analysis on user request and delivers results for further processing. Several Image Analysis Servers may be included into an Enterprise Setup to increase analysis power. Cellenger Data Control Server stores and manages image and result data in the database. Different database connectors are supporrte for database access. In the current version database connectors are available for Oracle 9i. and MS Access. Messaging Server is based on the JMS standard and connects all Enterprise modules. Different modules communicate using the messaging service via XML messages. A special client API allows third-party software to connect to the Cellenger messaging server to get access to all services of the Cellenger Enterprise System. Technical White Paper Definiens Cellenger 11 Enterprise Client provides access to all functionality of the Enterprise System. The user may view and mange the image data, analyze sets of images and browse through the analysis results. Detailed analysis results are available for each individual image as well as reports over entire analysis runs on image sets with a single-mouse-click. Cellenger Developer Studio Cellenger Developer Studio included Analysis Engine included Enterprise Client included Cellenger Enterprise Enterprise Client scalable, 1 included Analysis Engine Server scalable, 1 included DataControl Server included Messaging Server included MS Access Database Connector included Oracle 9.x Database Connector included Open Architecture for Flexible Workflow Integration The open architecture allows smooth integration of the Cellenger System in existing customer workflow chains. APIs at different levels provide the required level of access to functionality of Cellenger. Definiens Professional Services offers Consulting and Support coveriin all details of System Integration. System Requirements Cellenger Developer Studio minimum recommended Processor Pentium III 800 Pentium IV 3GHz RAM 512 MB 2 GB HD (free disk space) 2 GB 10 GB Cellenger Enterprise Analysis Server Processor Pentium III 800 Pentium IV 3GHz RAM 512 MB 2 GB HD (free disk space) 2 GB 10 GB Remarks Fast I/O System Bus for RAM and HD recommended Cellenger Enterprise Client Processor Pentium III 800 Pentium III 2 GHz RAM 512 MB 1 GB HD (free disk space) 2 GB 5 GB Cellenger DataControl Server Cellenger Messaging Server Cellenger Database Connector Processor Pentium III 800 or better RAM 256 MB 512 MB Technical White Paper Definiens Cellenger 12 About Definiens Definiens is a leading provider of high-content image analysis software and services for life sciences. Based on Definiens Cognition Network Technology®, a fundamental technology which models human thought, Definiens products enable the extraction of objects of interest from biomedical images. Main applications are in drug research and other biopharmaceutical processes. Definiens has more than 100 customers in the field of life sciences including companies such as Academia, Altana, Aventis, Intervet, Merck, NGFN, NIH, Novartis, and Novo Nordisk. Founded in 1994 by Nobel Prize Laureate Prof. Dr. Gerd Binnig, Definiens today has 40 employees and is based in Munich, Germany. For further information regarding the company and its products please visit www.definiens.com. Contact For more information about Definiens and our products visit www.definiens.com or contact sales.cellenger@definiens.com or support.cellenger@definiens.com Definiens AG Trappentreustr. 1 D-80339 München Phone +49 89 23 11 80 -0 Fax +49-89-23 11 80-90 www.definiens.com Technical White Paper Definiens Cellenger 13 © 2004 Definiens AG. All rights reserved. Definiens and the corporate logo are registered trademarks of Definiens AG in Germany and throughout the world. All other company and product names are used for identification purposes only and may be trademarks of their respective owners. Definiens cannot guarantee completion of any future products or product features mentioned in this document, and no reliance should be placed on their availability.