Center for Biomedical Imaging Informatics

Center for Biomedical Imaging & Informatics University of Medicine & Dentistry of New Jersey Anthony E. Grygotis, M.D. for David J. Foran, Ph.D. Problem and Motivation  Subtle visible cues exhibited by some malignant lymphomas and leukemia give rise to a significant number of false negatives during routine microscopic evaluation. Mantle Cell Lymphoma (MCL) is particularly problematic in this regard: – recently identified (1992), often misdiagnosed as  Chronic Lymphocytic Leukemia (CLL)  Folicular Center Cell Lymphoma (FCC) – much more aggressive clinical course than CLL or FCC – standard therapy for CLL and FCC is ineffective with MCL  David J. Foran, Ph.D. Hypothesis Hypothesis: By exploiting a set of non-traditional image metrics, statistical pattern recognition and data fusion an image guided screening system can be developed which reduces the frequency of false negatives while improving the accuracy of differential diagnoses. David J. Foran, Ph.D. System Architecture Logical Blocks & Data Flow David J. Foran, Ph.D. Distributed Telemicroscopy Client/Server Functions:   Server (C++/ JAVA)  Image acquisition rates & resolution  Coordinates communications among users  Coordinates communications w/ robotics  Serializes digitized pathology  Controls client/server, CPP/JAVA socket communications  Performs entropy-based auto-focusing operations Client (JAVA)  Image distribution and update requests: – Primary client / secondary clients functionality – Dynamic slider for variable image resolution  Dynamic text messaging  Shared graphical pointers David J. Foran, Ph.D. Distributed Control of Robotics Implemented with Software Tokens David J. Foran, Ph.D. Distributed Telemicroscopy & Intelligent Image Archival Module Animated Diagram Features: Distributed Telemicroscopy Remote Control of Robotics Controls Passed as Token Platform-Independent Shared Graphical Pointers White-boarding Text-based Messaging Auto-focusing Distributed Image Archiving Auto Feature Extraction Auto Database Management David J. Foran, Ph.D. Distributed Telemicroscopy Client Interface Telemicroscopy Demo Video David J. Foran, Ph.D. Image Guided Decision Support Module  Supports submission of queries from local and remote sites. Client-server communications supports multi-threaded, multiuser environments. Real-time analysis of query images at client side.   Decisions based on Ground truth database of cases for which diagnosis has been independently confirmed w/ immunophenotyping and/or molecular studies. Statistical pattern recognition and data fusion determine similarity between unknown and ground truth cases. Automatically retrieve images and correlated clinical data of statistically best matches.     Spectral and spatial signatures used to formulate multivariate query vector.  Speech recognition & voice feedback support. David J. Foran, Ph.D. Multivariate Feature Extraction Client side computation and generation of query vector Spectral & Spatial Signatures:  Chromaticity (L*U*V* Color Space)   Pixel Area of Cellular Components Shape (Elliptic Fourier Descriptors) (Multi-scale Simultaneous Auto-regression Model)  Texture David J. Foran, Ph.D. Robust Color Segmentation  Based on non-parametric analysis of feature spaces: Algorithm detects significant modes in L*u*v* space – based on the gradient ascent mean-shift procedure – randomly tessellates the space with search windows – moves windows till convergence and prune mode candidates Commaniciu and Meer, 1998 David J. Foran, Ph.D. Elliptic Fourier Descriptors Elliptic Fourier descriptors of a closed curve are given by: an  T 2n  2 2  i 1 m x i   2n  ti   2n  ti  1  cos  cos   T   T  t   i cn  T 2n  2 2  m m i 1 y i  2 n  t i   2n  ti  1  cos  cos  T  t   T    i bn  T 2n  2 2  i 1 m x i   2 n ti   2n  ti  1  sin  sin   T   T   t   i dn  T 2n  2 2  i 1 y i   2n t i   2n  ti  1  sin  sin  T  ti   T    i m where  x i   x i  x i  1 , y i  y i  y i  1  ,  ti   x i  2  y i  2 , 1 ti   t j j 1 , T   t i 1 i The phase shift from the major axis is given by:  I n  a n  bn  c n  d n 2 2 2 2  1  2 a 1b1  c1 d1   2 a 2  b 2  c 2  d 2    1 1 1 1 2 2 2 2 J n  a n d n  bn c n K1 ,n  a1  b1 a n  b n   c1  d 1 cn  d n   2 a1 d1  b1c1 a n cn  bn d n  2 2 2 2 David J. Foran, Ph.D. Elliptic Fourier Descriptors To make the Fourier descriptors independent of starting point, can be limited to the interval  n a* c  * n 1      ,  2 2  by using the transformation given by bn*  d   * n  an c  n bn  cos n  1 d n sin n  1   sin n  1  cos n  1   . Further, to make the Fourier descriptors a n , bn , c n , 2 2 2 independent of scaling, each of factor D  a 1  b1  c1  d 1 2 2 2 2 and d are divided by a scaling 2 n 2 2 2 . he EFD’s are then computed by: T I n  a n  bn  c n  d n 2 J n  a n d n  bn c n K1 ,n  a1  b1 a n  b n   c1  d 1 cn  d n   2 a1 d1  b1c1 a n cn  bn d n  2 2 2 2 2 2 2 2 David J. Foran, Ph.D. Elliptic Fourier Descriptors Invariant to start point, rotation, scale A B C Caption: Representation of a closed contour by Elliptic Fourier descriptors. A. Input B. Series truncated at 16 harmonics C. Series truncated to 4 harmonics David J. Foran, Ph.D. Similarity Invariant Boundary Representation  Elliptic Fourier Descriptors of the chain coded contour: – Kuhl and Giardina, 1982. Resolution of the boundary representation should not exceed segmentation uncertainty. Superimposed contours for 25 segmentations (the darker pixel the higher certainty of delineation) Experiment: – 5 images – segment cells using 25 different ROI’s – compute the normalized variance for the first 16 harmonics (64 coeffs) – typical result – first ten harmonics are reliable.   David J. Foran, Ph.D. Multiscale Texture Representation    Multiscale Simultaneous Autoregressive Model (MRSAR): Symmetric MRSAR applied to the lightness L* component : – the pixel at any location depends linearly on its neighbors – 15-dimensional feature vector (3 resolutions) and its covariance matrix derived for each cell in the database. Distances between cells is computed taking into account within and across class variations. Examples of nuclear textures. (dynamic range enlarged to improve reproduction) David J. Foran, Ph.D. Multivariate Fusion     Combine feature measures for area, shape, texture and color of cellular components Optimize the sum of conditional probabilities of correct decision across entire data set Ameboid search for global max on objective surface Weighting factors computed on a dual-PIII w/ 2GB J = 3.4207 Shape 0 .1 1 4 0 T e x tu re 0 .5 7 7 1 A re a 0 .3 0 8 9 J 3 .4 2 0 7 David J. Foran, Ph.D. Man-Machine Performance Studies CLL CLL FCC MCL NRM .8 3 8 9 .0 2 5 0 .1 3 5 7 .1 3 3 3 FCC .0 2 0 0 .9 0 0 0 .0 1 4 3 .1 2 0 0 MCL .0 7 1 1 .0 0 0 0 .8 3 3 3 .0 0 0 0 NRM .0 7 0 0 .0 5 0 0 .0 0 0 0 .7 3 0 0 No Dec .0 0 0 0 .0 2 5 0 .0 1 6 7 .0 1 6 7 IGDS Confusion Matrix CLL CLL FCC MCL NRM .5 6 4 7 .0 2 8 5 .1 5 3 8 .1 2 2 8 FCC .0 3 5 2 .9 4 2 8 .0 7 9 6 .0 0 0 0 MCL .2 1 1 7 .0 0 0 0 .5 5 3 8 .1 0 5 3 NRM .1 7 6 4 .0 2 8 5 .1 6 9 2 .7 5 4 3 No Dec .0 1 1 7 .0 0 0 0 .0 4 6 1 .0 1 7 5 Human Observer 1 False Negatives (averaged): IGDS – 4.3%; Human Observers – 14.6% David J. Foran, Ph.D. IGDS Client Interface IGDS Demo Video David J. Foran, Ph.D. Unsupervised Specimen Analysis Video demonstration of unsupervised analysis & data management David J. Foran, Ph.D. Regional, Network-Based Laboratory for Research in Biomedical Informatics Through HUBS, we are establishing a high speed virtual private network link among strategic sites at UMDNJ, the University of Pittsburgh Medical School, Johns Hopkins University, the University of Pennsylvania School of Medicine, and the Pittsburgh Super Computer Center. This network-based laboratory will be utilized to expand our research base in collaborative telemedicine, interactive medical education and computerbased decision support in diagnostic pathology and radiology. David J. Foran, Ph.D. International Telemedicine Program  Deployed portable telemedicine systems to RWJUH and to sister hospital at Zhong Shan Hospital, Shanghai, China. The systems are used for conferencing, clinical training, and remote digital consultation in pathology and radiology. David J. Foran, Ph.D. Future Directions  Conduct comprehensive, multi-site statistical assessment of IGDS using recall and precision metrics to evaluate retrieval effectiveness of the query algorithms (HUBS).  Optimize algorithms based on recall, precision & man-machine studies.  Complete the modifications which enable submission of queries from Virtual Microscopes (OSU, JHMI, UMD). Explore the potential of multi-resolution, CBIR (UPitt).  Evaluate the use of the IGDS in a broader spectrum of hematopathology and cytopathology applications (UPenn).  Expand performance studies used to test the DT/IGDS core system in imaging, analyzing, and archiving tissue microarrays (CINJ). David J. Foran, Ph.D. Center for Biomedical Imaging & Informatics Video with closing credits David J. Foran, Ph.D.