Illustration, quantiﬁcation and a solution to the problems in imaging
moving and deformable regions of interest
Olivier Noterdaemea∗ and Michael Bradya
aDepartment of Engineering Science, Wolfson Medical Vision Laboratory, Oxford, UK
Abstract. To avoid image blurring, many Magnetic Resonance Image (MRI) sequences of the liver are acquired
in short breath holds, each covering a portion of the liver. Multiple breath hold images are designed to cover the
entire region of interest contiguously. However, there is no automated feedback conﬁrming that the set of breath
hold images satisfy these properties. Indeed, in practice they may not be contiguous, and we present an example of
liver imaging where non-contiguous MR images resulted in an inaccurate assessment of the number of lesions. A
geometrical model of the liver is used in Monte Carlo simulations to assess the impact of variations in breath hold
imaging. We ﬁnd that lesions are incorrectly staged in 24% of imaging studies. This motivates the presentation of
a two-stage method where a reference image covering the whole region of interest (ROI) is ﬁrst acquired. Second,
individual breath hold images, each covering an unknown part of the ROI are registered to the reference image and a
representation is built up to identify those areas of the ROI that have been covered, perhaps multiple times. Finally,
we illustrate and highlight the need for such a system with two examples.
The tissue properties fundamental to Magnetic Resonance Imaging (MRI) are the T1 and T2 relaxation times and
images can be T1 - or T2 -weighted. To fully characterise liver lesions, different types of MR images are necessary
and T2 -weighted images are particularly helpful. Benign cysts are of very high signal, metastatic disease of less high
signal, and often benign disease is of intermediate signal. MRI of moving organs, such as the liver, is subject to several
deteriorating factors, mainly due to breathing, which may cause ghost artifacts and blurring. To reduce such effects, it
is common practice to partition long sequences into several shorter breath hold acquisitions , during which the tissue
is approximately static. The multiple breath hold images, each of which covers only a portion of the region of interest
(ROI), are designed to be contiguous and to cover the entire ROI, in our case the liver. Breath hold acquisitions of the
liver are done on exhalation as it is the longest pause in the breathing cycle  and most reliably recreates the position
of the liver on subsequent occasions . However, the internal organs do not always return to the same position for the
same lung volume or at the same point in the breathing cycle . Techniques which rely on the monitoring of the lung
volume, either through externally applied bellows or navigator echoes  can thus not fully compensate for variations
in the organ position. In addition, free breathing or respiratory triggered techniques suffer from lower signal to noise
ratio (SNR), inferior image quality and reduced lesion clarity while signiﬁcantly increasing the scan time [1,5]. Breath
hold imaging is thus the preferred acquisition method for abdominal imaging. Though the set of multiple breath hold
images are designed to be contiguous and cover the entire liver, no automated feedback is currently provided to the
clinician that the set of breath hold images satisfy these two properties. Variations in organ position due to the variable,
unpredictable, and unmeasured levels of inspiration and expiration may result in areas of the liver that are imaged
repeatedly, and others that are not imaged at all. This can result in under- or overstaging of cancer, for example by
suggesting an erroneous estimate of tumour size and lesion count.
This paper begins by illustrating the problem with a clinical dataset in which three radiologists identiﬁed different
number of lesions. Section 3 describes a geometrical model of the liver used in Monte Carlo simulations to quantify the
impact of variations in breath hold on the staging of lesions. Section 4 presents a two-stage method to identify ﬂawed
regions and correct for these deﬁciencies. We illustrate the technique using examples from clinical liver imaging.
However, the idea is more widely applicable to the imaging of moving and deformable organs. The article does not
present a novel image registration algorithm (much of the registration work presented here was to demonstrate the
usefulness of the concept and has been manually), but a frequently occurring problem in clinical imaging. We quantify
the impact of such datasets on decision making and propose a solution to circumvent this problem.
2 The clinical problem
Figure 1 shows a selection from four consecutive slices of a T1 -weighted MR dataset. Images 1(a) and (c) were acquired
in the ﬁrst breath hold, while images (b) and (d) were acquired in the second breath hold. Inspection of the full stack
of 2D slices reveals that there is a shift of the internal organs between the two image sets acquired in separate breath
∗ email: email@example.com
(a) (b) (c) (d)
Figure 1. Four “consecutive” axial FSPGR images showing two distinct lesions (circled). The dotted lesion in (d) was
incorrectly counted as a third lesion by two radiologists.
(a) (b) (c) (d)
Figure 2. Coronal view of reformatted axial images from Figure 1 before ((a), (b)) and after ((c), (d)) correction.
The corrected reconstruction shows that each circled lesion is one continuous large lesion, rather than multiple distinct
holds, but the degree of this shift cannot be determined by simply looking at the 2D image series. Three radiologists,
with together over 25 years of experience in abdominal MRI, were asked to highlight every major lesion a single time
on the original image sequence. This patient has ﬁve hypointense lesions, two of which are circled in Figure 1(a).
However, several of these lesions, such as the one marked with a dashed circles in 1(d) were incorrectly taken for
additional lesions by radiologists. The correct number of lesions was determined by the radiologists after presentation
of a correctly ordered dataset. To explain why radiologists miscounted some of the lesions, it is helpful to reformat
the axial images into the coronal plane. Figure 2(a) and (b) shows that the shift between the interleaved image sets
causes discontinuities. After re-sorting the images the visual appearance of the reformatted images, shown in Figure
2(c) and (d), is improved with smoother boundaries for the liver and tumours (hypointense areas inside the circles). In
the original dataset each lesion appears to consist of multiple distinct parts. If a lesion seems to start and ﬁnish, it may
be perceived as two distinct lesions. However, if the images are correctly ordered, ie. they are actually in anatomical
order, then what previously seemed to be distinct lesions now appears as a single lesion in the reformatted images.
It is worth noting that all of the medical image viewers we tried (GE Advantage Workstation, ITK-Snap1, VolView1 ,
AMIDE1 ) failed to deal correctly with inconsistently spaced image slices. They either assume constant slice spacing,
or, if the images are acquired with a known overlap, fail to reconstruct reformatted images because they lack the ability
to deal with overlapping data.
Variations in breath holds pose a real clinical problem and can result in misdiagnosis through missing lesions, double
counting them or the erroneous estimate of their size. Radiologists may not be aware either of the fact, or the extent,
of missing or overlapped data. Section 3 presents a geometrical model to compute the errors in estimated liver tumour
volumes resulting from variations in the liver position on subsequent breath holds.
3 Monte Carlo simulation
3.1 Geometrical model
Figure 3 shows that an ellipse is a reasonable approximation for the shape of the liver when viewed in the sagittal plane.
We assume that the liver is ideally covered by two contiguous rectangular breath hold acquisitions (see Figure 3(b)),
each consisting of multiple 2D slices. The tumour is marked by a circle placed centrally between the two breath hold
acquisitions. Solitary spherical liver lesions enable the simulations in 2D while retaining full validity of the model in
3D. Consistent with the variations in liver position we have observed in clinical datasets, the allowed cranio-caudal
translation are ±10mm, with rotations about the centre of the image of ±3.4 deg (both normally distributed). For
comparison, a review article on liver motion  gives cranio-caudal translations of up to 75mm in deep inspiration,
1 http://www.itksnap.org/; http://www.kitware.com/products/volview.html; http://amide.sourceforge.net/
0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500
(a) (b) (c)
Figure 3. (a) Sagitally reformatted T1 W 3D FSPGR image and (b) the corresponding geometrical model showing liver,
tumour and two rectangular breath hold acquisitions. (c) One of 3 million simulations where the liver returned to a
different position on the second breath hold, which is equivalent to a rotation of the red breath hold acquisition.
180 missed lesions
0.8 "responding" lesions
160 "stable" lesions
visual lesion coverage [%]
3 4 5 6 7 8 9 10 11 12 13 14 15 3 4 5 6 7 8 9 10 11 12 13 14 15
lesion radius [mm] lesions radius [mm]
Figure 4. (a) Percentage of tumour volume visually covered as a function of true tumour radius and (b) the resulting
staging classiﬁcation according to .
and anterior-posterior and left-right motions of up to 12mm and 9mm, respectively. Figure 3(c) shows one of 3 million
image acquisitions for which the tumour radius ranged from 3mm to 15mm. For each simulation the covered tumour
volume was computed. The real impact of inconsistent breath holds on the computed tumour volume can only be
assessed with a full clinical trial, which is outside the scope of this paper. However, because we use clinically observed
data in our simulations, it is possible obtain an indication of the severity of the problem.
3.2 Staging of lesions according to visual lesion coverage
Accurate tumour volume quantiﬁcation requires measurements on a 3D image set. In the simulations, the exact tumour
volume is known and the visually covered tumour volume is determined. In a clinical setting, the “true” tumour volume
would correspond to the baseline volume estimated from a previous study using semi- or fully automatic segmentation
techniques. The patient is re-scanned (without change in lesion size) and the lesion is measured again. Lesions are
classiﬁed as responding if the volume decreases by 65% and progressing if the volume increases by 73% . As the
simulations are based on noiseless images, a lower bound on the accuracy of segmentation techniques can be estimated.
Figure 4(a) shows a boxplot of the variation in volumetric lesion coverage as a function of tumour radius. As expected,
the median is centred at 100% lesion coverage. The interquartile range, ie. deviation from the true tumour volume,
decreases with increasing lesion size. Overall, 2.7% of all lesions would not be imaged at all, and using the above
staging criteria, 10.4% of lesions would be classed as “responding” while 10.6% would be classed as “progressing”
despite not having changed in size (Figure 4(b)). Using the visually covered tumour volumes, without compensating for
variations in breath hold, can result in only 76% of lesions being correctly staged. For volume estimation techniques
to be clinically useful, it is vital to reduce the uncertainties in estimated tumour volumes. This can be achieved by
enhancing the precision of the underlying data, predominantly by developing a clear understanding of missing and
overlapping data. The next section will present a method to do this.
4 Quantiﬁcation of missing and overlapping data in abdominal imaging
In the proposed method a single reference image covering the whole region is ﬁrst acquired. The primary criterion
being that the image acquisition period of the reference image should be sufﬁciently short that the tissue imaged can
be considered to be essentially static. In the case of liver imaging, we use a T1 -weighted 3D FSPGR sequence, as the
reference image to manually segment the liver. This type of MR sequence can acquire a full 3D image of the liver in
Figure 5. (a) 2D Accumulator representation in which the ROI is depicted schematically as an ellipse. Two breath
hold images, depicted as rectangles, have been acquired. The multiply imaged region corresponds to those voxels with
value 2; those not yet imaged to voxels with value zero. (b) The ﬂowchart describes the complete image acquisition
process, ensuring full coverage of the ROI.
(a) (b) (c) (d)
Figure 6. Fat-saturated sequence for slices 9/10 taken at the same anatomical location, showing the alteration in liver
position due to differing breath holds. Registration of fat-saturated sequence to the liver shows uncovered areas.
less than 20 seconds, ie. within a single breath hold, during which the liver will not move. Next, we consider the partial
breath hold images, in this case a T2 -weighted fat-saturated sequence, consisting of three blocks of nine slices each.
Each image block takes approximately 20 seconds to acquire. The intention of the acquisition protocol is that the last
slice of one image block is identical to the ﬁrst slice of the following image block. Accordingly, Figure 6(a) and (b) as
well as Figure 7(a) and (b) should be identical. However, they are not, and these examples have been chosen as they
are also obvious to non-radiologists. There is no method to determine the extent of this displacement, the amount of
missing or overlapping information directly from 2D slices.
Our proposed method registers each of the acquired breath hold images to the T1 -weighted reference volume. Con-
current with this sequence of registrations, a representation is built up to identify those areas of the ROI that have
been covered, perhaps multiple times. In particular, areas that have not been covered can easily be determined and
acquired in additional scans. In the pilot implementation, we use a simple accumulator representation (Figure 5(a)) in
which the counter for a voxel is incremented each time that the voxel is determined (following registration) to belong
to the current breath hold image. The ﬁgure depicts, schematically, an accumulator representation in which the ROI is
represented as an ellipse. Two breath hold images, depicted as rectangles, have been acquired. The regions acquired
multiple times corresponds to voxels with value 2; those not yet imaged to voxels with value zero. The accumulator
representation provides immediate visual feedback to the operator. If the system were implemented at the scanner
console, then areas that have not been covered could be (automatically) acquired in additional scans (Figure 5(b)).
After the acquisition and alignment of all partial images, there can be regions where the accumulator has got a value
of 2. The combination of independent acquisitions of nominally the same piece of tissue enables the improvement of
the SNR, eg. by averaging. Indeed, we have found improvements in the SNR of 39%, corresponding closely to the
theoretical improvement of 41% ( 2).
For the ﬁrst case presented, we see that alignment of each group of T2 -weighted images to the hand-segmented liver
(Figure 6(c) and (d)) highlights a gap and relative rotation between the two breath hold image sets . The third
acquired block is not shown because it did not contain any part of the liver. For the second case, Figure 7(c) and (d),
we observe that there is a larger than intended degree of overlap between two of the breath hold acquisitions (arrow).
For the interface between the other two breath hold acquisitions the overlap is as intended (triangle).
(a) (b) (c) (d)
Figure 7. Fat-saturated sequence for slices 18/19 taken at the same anatomical location, showing the alteration in
liver position due to differing breath holds. This resulted in a larger than intended overlap between two breath hold
acquisitions (arrow). The overlap between the other two breath hold acquisitions (triangle) is of the desired, smaller,
Through alignment of individual breath hold images to a reference volume it is apparent that what were a priori
believed to be contiguous blocks (with a single slice in common) is in fact not true. In this case, signiﬁcant portions of
the liver remain unimaged or were imaged multiple times, complicating clinical reporting. Our technique enables the
accurate determination and correction of such areas.
We have presented an application of image registration to a clinical problem that represents a fundamental challenge in
liver MRI. Section 2 illustrates that variations in the level of breath hold can result in non contiguous datasets, which
can cause errors in the accurate staging of a disease. It was also noted that medical image viewers can not correctly
deal with the representation of inconsistently spaced datasets. Based on a geometrical model of the liver, Section 3 uses
Monte Carlo simulations to quantify the extent of this problem and we ﬁnd that up to 24% of scans could be incorrectly
staged. To account for the variations in breath holds, we acquire a reference image and delineate the region of interest.
Subsequent images are automatically aligned to this reference image. An accumulator representation is constructed
during the succession of registrations, and highlights any parts of the ROI that are not imaged. Radiographers are for
the ﬁrst time able to evaluate their work plan. The identiﬁcation of unimaged areas directs further acquisitions to yield
complete coverage of the ROI. We believe that liver MRI would greatly beneﬁt from aligning individual images to a
reference volume and describing the covered region with an accumulator representation.
This work is supported by General Electric Healthcare. The authors would like to thank Anthony McIntyre for acquir-
ing the images.
1. J. Gaa, H. Hatabu, R. L. Jenkins et al. “Liver masses: replacement of conventional T2-weighted spin-echo MR imaging with
breath-hold MR imaging.” Radiology 200(2), pp. 459–64, 1996.
2. M. A. Clifford, F. Banovac, E. Levy et al. “Assessment of hepatic motion secondary to respiration for computer assisted inter-
ventions.” Comput Aided Surg 7(5), pp. 291–9, 2002.
3. T. Rohlﬁng, J. Maurer, C. R., W. G. O’Dell et al. “Modeling liver motion and deformation during the respiratory cycle using
intensity-based nonrigid registration of gated MR images.” Med Phys 31(3), pp. 427–32, 2004.
4. C. J. Zech, K. A. Herrmann, A. Huber et al. “High-resolution MR-imaging of the liver with T2-weighted sequences using
integrated parallel imaging: comparison of prospective motion correction and respiratory triggering.” J Magn Reson Imaging
20(3), pp. 443–50, 2004.
5. J. Augui, O. Vignaux, C. Argaud et al. “Liver: T2-weighted MR imaging with breath-hold fast-recovery optimized fast spin-echo
compared with breath-hold half-fourier and non-breath-hold respiratory-triggered fast spin-echo pulse sequences.” Radiology
223(3), pp. 853–9, 2002.
6. A. R. Padhani & L. Ollivier. “The RECIST (Response Evaluation Criteria in Solid Tumors) criteria: implications for diagnostic
radiologists.” Br J Radiol 74(887), pp. 983–6, 2001.
7. O. Noterdaeme, R. Phillips, F. Gleeson et al. “Quantiﬁcation of missing and overlapping data in multiple breath hold abdominal
imaging.” Eur J Radiol 64(2), pp. 273–278, 2007.