maps, mash‐ups and metadata
medical imaging and mapping
medical imaging and mapping
compared and contrasted
topics
• … just what is a map anyway?
• 21st Century mapping
• Mashups = meta‐analysis?
• When it goes wrong
• Particle Radiotherapy
• Pathology and ‐omics
• Comparison of map‐based workflows
• Concluding remarks
what is a map (OED)
what is a map (OED)
• 1 a. A drawing or other representation of the earth's surface
or a part of it made on a flat surface, showing the distribution
of physical or geographical features (and often also including
of physical or geographical features (and often also including
socio‐economic, political, agricultural, meteorological, etc.,
• …
• 3 a. A diagram or collection of data showing the spatial
distribution of something or the relative positions of its
g p
components. Freq. with distinguishing word.
more functionally…
more functionally
• A diagram derived from an image or a survey, with
– components labelled, coloured or otherwise annotated according to
one or more legends
one or more legends
– (optionally) associated with a coordinate system
– further metadata for scale, provenance, projection…
• A legend links a conceptual model appropriate to the diagram
with features in the diagram through the use of symbols or
colour
• A coordinate system allows registration of the diagram with
lit th fd t d t d t
reality or other sources of data and metadata
mash ups
‘mash ups’
• Dataset joining from unconnected data sources facilitated by
common keys, labels or coordinates unanticipated by the
originators of the data
originators of the data
• Most commonly map based
Coordinate systems ubiquitous, easily identifiable within text, often
– Coordinate systems ubiquitous easily identifiable within text often
accurate
• Latitude and longitude
• Postal code
– Well documented services and APIs
Compelling: collision of mobile devices, high bandwidth networks,
– Compelling: collision of mobile devices high‐bandwidth networks
GPS, satellite imagery
• A user defined legend, often with ‘crowd sourced’ data
• A form of ‘meta‐analysis’
It commonly happens that there is no one conclusive study
but 'meta‐analysis' gives you a combined result from every
randomised trial that has been done around the world. For
example, the drug Tamoxifen ‐ an oestrogen blocker that may
prevent breast cancer cells growing ‐ was the object of forty
prevent breast cancer cells growing was the object of forty‐
two studies world‐wide ‐ of which only four or five had shown
significant benefits. But this did not mean that Tamoxifen did
t t t i tb t 'Wh t ll th
not protect against breast cancer. 'When we put all the
studies together it was blindingly obvious that it does ‐ you
don't have to be a medical statistician to see that. Nor do you
y
need to be an economist to see the advantages of saving tens
of thousands of lives with this inexpensive drug’
Prof. Richard Gray, University of Birmingham
early breast cancer trialists’
collaborative group
• Initiated in 1983
• Hundreds of institutions worldwide
• Consensus on 30 variables, a data model and submission
format
Analyse data every 5 years
• A l d 5
• Computable data and follow‐up for 200,000 cases in the 2000
review
• Rock‐solid evidence base for the treatment of early breast
cancer
polluter zip code ‐> Google map
polluter zip code > Google map
• The US EPA took the zip code of their polluter register and
plotted it on Google Maps
Anomalous heavy polluter appeared in downtown San
• A l h ll t di d t S
Francisco…
the address for a noted polluter was actually that of its legal
• … the address for a noted polluter was actually that of its legal
representatives
caveats
• Metadata is expensive, so how good is free metadata?
• Often rely on implicit data standards – no information on
li
compliance or accuracy
• Often lexical matching – unreliable
Crowd sourced information – no provenance
• C d di f i
• Originator of data may be biased
h h ll f h l d df h
• Might not have all of the contextual metadata required for the
proper interpretation of the data
PARTICLE RADIOTHERAPY
tumour
100
80
SOBP
Dose (%)
60
D
40
P i ti k
Pristine peak
20
50 100 150
Depth (mm)
combining fields
combining fields
80 50
60 150 0
150
40 0
y
X‐Rays Protons/Ions
Paravertebral Epithelioid Sarcoma
Intensity Modulated Protons (IMPT) vs.
Intensity Modulated Photons (IMRT) 7 (field)
y ( ) ( )
IMPT IMRT
greater precision and control means…
greater precision and control means
• We can:
– irradiate tissue that is close to important organs
boost the tumour dose without increasing the dose to the surrounding
– boost the tumour dose without increasing the dose to the surrounding
tissues
– reduce immediate side effects of radiotherapy
• Ocular; CNS; sarcomas esp. where resection is difficult or
incomplete; paediatric cancers; prostate; breast tumours…
MAPPING IN RADIOTHERAPY
the problem
the problem
• Based on the consensus of opinion, 32% of radically treated
patients should receive inverse‐planned IMRT and 22%
forward‐planned IMRT, making a total of 55%. In fact, 2%
forward planned IMRT making a total of 55% In fact 2%
receive inverse‐planned IMRT and 11% the less complex
p , ,
forward‐planned IMRT. Thus, with an estimated 75,948 radical
treatments being carried out with megavoltage radiotherapy,
the professional opinion is that 41,421 of patients would
benefit from treatment with IMRT. In fact, only 9775 were so
b f f h f l
treated in 2008; a shortfall of 32,497 patients treated instead
with conventional radiotherapy.
with conventional radiotherapy
– Survey of the availability and use of advanced radiotherapy technology
in the UK. Mayles et al, Clin Oncol (R Coll Radiol). 2010 Oct;22(8):636‐
42. Epub 2010 Jul 27.
radiotherapy underutilised
radiotherapy underutilised
The machines are in the trust, but have insufficient resources
• Th hi i h b h i ffi i
to plan advanced treatments – even for IMRT
Compounded by need to plan in four dimensions (time)
• Compounded by need to plan in four dimensions (time)
– tumours shrink (we hope)
– tumours move (lung cancer)
• Need automation
– organ recognition
– delineation of target and treatment margins
delineation of target and treatment margins
– better calculations of dose
– comparison of multiple plans and multiple modalities
– calculation of risk factors such as malignant inductance probabilities
– through complex calculations and Monte Carlo simulations
ICRP Publication 110 Phantoms
125 119 119 119 109 109 109 109 109 109 109 109 109 109 109 109 109 109 109
109 109 109 109 28 28 28 28 28 28 28 28 28 109 109 109 109 109 ...
Elemental
Composition
N H Na S
4.2 3.6 0.3 0.3 Mg
0.2
P
9.4
O
44.8
C
15.9
Ca
21.3
Tissue №
voxel Mineral bone Blood/bone
marrow content
Organ № 1% blood, 0% red
28 ‐ Femora, upper bone marrow, 0%
half, cortical yellow bone marrow,
100% bone
Mass Density
1.92gcm‐3
ICRP Publication 110 Phantoms
ICRP Adult Male Phantom ‐ from left to right: anterior surface, coronal slice showing mass density, mass density
integrated over coronal slices, calcium content integrated over coronal slices
Monte Carlo simulations
• Using the Monte Carlo code MCNPX Remainder Tissues
1.2
(transports photons, protons and
( p p ,p 1.0
)
efficient (Gy/Gy)
heavy ions) 0.8
0.6
• Implemented phantom geometry 0.4
Coe
0.2 JJ
• Benchmarked by comparing x‐ray 0.0 DW
organ dose conversion coefficients 0.01 0.1
Photon Energy (MeV)
1
with Jan Jansen of Health Protection
Agency
Can perform simulations and extract
• C f i l ti d t t
3D dose data – useful for comparisons
with treatment planning
with treatment planning
conversion to DICOM
• Can replace each voxel with an
estimated Hounsfield Units (HU)
( )
value, then export to DICOM
using CERR toolkit for Matlab
• Allows comparison of treatment
planning systems with Monte ICRP 110 Organs
(Soft Tissue Region)
( g )
Carlo simulations, investigation of
C l i l ti i ti ti f
pping Power
1.2
HU vs electron density/proton 1.15
stopping power
stopping power
at 100MeVV
Relative Proton Stop
1.1
– Potentially requires more care for 1.05
protons given sharp dose gradients
1
e
0.95
-100 0 100 200 300
CT Number (Hounsfield Units)
25
system diagram
XiO TPS MATLAB MCNPX
Dan s
Dan’s
CERR
code
Clinical
DICOM planC
l C Ph t
Phantom
Monte Carlo
Monte Carlo
Treatment
Simulations
Plans
Dan’s
CERR
code
l l ki difi i h h
• Also looking at modifications to the phantom – i i
inserting a tumour,
adding finer scale structure to the lungs and bone
MIP Calculations
MIP Calculations
• Heatmaps for the probability of malignant induction for
different treatments (with the Mayo Clinic)
atypical meningioma
– Subtotally resected atypical meningioma
– Compare plans for different types of radiotherapy
• 3D Conformal RT (standard treatment)
• IMRT
• RapidArc (helical RT)
• Spot scanning protons
Spot scanning protons
– Account for fractionation
Dose (GyE) ‐log10(Surviving Fraction)
protons
p
IMRT
Cells Transformed per cm3
x10‐3
protons
p
x10‐3
IMRT
• (show the video…)
multimodality image workspace for
contouring
Commercial
Commercial
TPS
CT
CT
CT
CT Push data back
Push data back
CT
CT into commercial
TPS vis DICOM‐RT
PET PET
CT
CT
CT
CT
CT
MRI Fast image Automatic segmentation
registration of normal tissue structures
P it f t t
Persistence of structures
CT
CT across image modalities
CT
CT
CT
PET
AccelRT
/
• a Cluster/Grid‐enabled treatment planning workbench for
DICOM images
automatic draft delineation of tumour and adjacent tissues in four
– automatic draft delineation of tumour and adjacent tissues in four
dimensions
– ‘find‐one‐like‐it’ searches across image repositories
– declarative extension for new services
• led by clinicians at University of Cambridge/Addenbrookes
• segmentation, registration services: Cambridge eScience
• workbench, service declaration, metadata management:
PTCRi
PATHOLOGY IMAGE PROCESSING
automation of scoring
automation of scoring
• Pathologist annotation is a bottleneck and expensive
• Use astronomy image analysis to provide metadata to the
tissue microarray maps
ti i
• Publish the image set, together with legend and metadata for
further meta‐analysis and DAN studies
further meta analysis and DAN studies
• Requires automatically generated maps and annotations of
the microarrays
the microarrays
TMA definitions in a metadata registry
TMA definitions in a metadata registry
4
Construct TMA and
Create EXCEL map
ARIOL server
TMA database
1. Transform to XML SQL Server
DEEP ZOOM 2. Import slidemap
slidemap
i
images ARIOL WEB
stored on SERVICE
file system (AWS)
7. Export 4. Export images+ 3. Accept requests for
scores metadata images
6. Manual on‐line scoring
JPEG images
5. Rename images
Batch Deep Zoom
Converter program
Statistical Pathgrid
Image analysis image
User
Scorer analysis
Manual score: Intensity 3, Proportion 4 = 7
ER positive
ER negative Manual score: Intensity 1, Proportion 1 = 2
10
Pathgrid automated score for ER in highly correlated with the
Allred manual score.
Allred manual score.
‐ summary of 1571 core images (1183 ER positive, 388 ER negative)
‐ 92.4% sensitivity, 93.7% specificity
8
7
6
5
4
ER+
3
2
ER‐
0
11
Pathgrid automated score for HER2 in highly correlated with the
manual score
Herceptest manual score
‐ summary of 1775 core images (1573 HER2‐, 115 HER2 borderline, 87 HER2+)
‐ 96.5% sensitivity 97.6% specificity
3 HER2+
2 borderline
1
HER2‐
HER2
0
unscored
15
CONCLUDING REMARKS
getting from A to B in the 21st Century
getting from A to B in the 21
Workflow Route planning
• Image acquisition • Satellite
• Map registration • Automatic
g
• Feature recognition y
• Aided by dataset
• Dataset joining • Features of interest, traffic
• Human annotation / p
• Crowd sourced/subscription
• Making the decision • Automated algorithms
• Execution g
• GPS guidance
the workflow remains the same
… the workflow remains the same
Workflow Treatment planning
• Image acquisition • CT, PET, Ultrasound
• Map registration • Tattooed reference point
g
• Feature recognition • Want to be automatic
• Dataset joining • Elemental composition
• Human annotation p
• Expert sourced
• Making the decision • Manual treatment planning
• Execution • Servo actuated relative to
reference point
the workflow remains the same
… the workflow remains the same
Workflow Tissue sub study ‐> evidence
• Image acquisition • Photograph of microarray
• Map registration • Spot coordinates
g
• Feature recognition g g g
• Astrogrid image recognition
• Dataset joining • Clinical data about subject
• Human annotation p (p g )
• Expert sourced (pathologist)
• Making the decision • Statistical analysis
• Execution p p
• Paper publication
common functions and workflow
common functions and workflow
• Many process are the same at some level of abstraction from
a software perspective
Facilitates the development and reuse of common software
– Facilitates the development and reuse of common software
• Allows us to define the workflow and transformations at a
higher level from user perspective
higher level from user perspective
– more automation of the analysis process
– but science will always want to be cutting edge
• Explains why we are all here already sharing and reusing
software
acknowledgements
• Raj Jena (Addenbrookes)
Lorna Morris (CRI, Cambridge)
• Lorna Morris (CRI, Cambridge)
• Bleddyn Jones, Claire Timlin, Dan Warren
(PTCRi)
thank you
Steve Harris
James Martin Research Fellow
James Martin Research Fellow
Oxford University Computing Laboratory
Parks Road, Oxford. OX1 3QD
Tel: 01865 610653
Fax: 01865 283531
F 01865 283531
steve.harris@comlab.ox.ac.uk
steve.harris@ptcri.ox.ac.uk
http://www.comlab.ox.ac.uk/people/Steve.Harris/
htt // l b k/ l /St H i/
http:/www.ptcri.ox.ac.uk/people/#PicSH