Document Sample

Color Quantization of Images †‡ Michael Orchard Charles Bouman Department of Electrical Engineering School of Electrical Engineering Princeton University Purdue University Princeton, NJ 08544 West Lafayette, IN 47907-0501 (609) 258-4643 (317) 494-0340 Abstract 1 Many image display devices allow only a limited number of colors to be simultaneously displayed. Usually, this set of available colors, called a color palette, may be selected by a user from a wide variety of available colors. Such device restrictions make it particularly diﬃcult to display natural color images since these images usually contain a wide range of colors which must then be quantized by a palette with limited size. This color quantization problem is considered in two parts: the selection of an optimal color palette and the optimal mapping of each pixel of the image to a color from the palette. This paper develops algorithms for the design of hierarchical tree structured color palettes incor- porating performance criteria which reﬂect subjective evaluations of image quality. Tree structured color palettes greatly reduce the computational requirements of the palette design and pixel map- ping tasks, while allowing colors to be properly allocated to densely populated areas of the color space. The algorithms produce higher quality displayed images and require less computations than previously proposed methods. Error diﬀusion techniques are commonly used for displaying images which have been quantized to very few levels. This paper studies problems related to the application of error diﬀusion tech- niques to the display of color images. A modiﬁed error diﬀusion technique is proposed for resolving these problems. The new error diﬀusion techinque is shown to be easily implemented using the tree structured color palettes developed earlier. EDICS #6.2.6 Image Transformations for Display and Printing † This work was supported by the Oﬃce of Naval Research and the Naval Research Laboratory under Grant N00014-87-0189, by an IBM Graduate Fellowship, and by an AT&T Fellowship. ‡ IEEE Trans. on Sig. Proc., vol. 39, no. 12, pp. 2677-2690, Dec. 1991. 1 Permission to publish this abstract separately is granted. 1 1 Introduction Video monitors display color images by modulating the intensity of the three primary colors (red, green and blue) at each pixel of the image. In a digitized color image each primary color is usually quantized with 8 bits of resolution in order to eliminate distinguishable quantization steps in luminance, color hue and color saturation. Thus, full color digital display systems use 24 bits to specify the color of each pixel on the screen. The cost of high-speed memory needed to support such a full-color display on a high- resolution monitor makes many applications impractical. An alternative approach, which is used by many currently available displays, is to provide a limited number of bits, such as 8, for specifying the color at each pixel. Each of these 28 = 256 values is then used as an index into a user deﬁned table of colors. Each entry into the table contains a 24 bit value which speciﬁes the color’s red, green and blue components. In this way, the user is allowed to select a small subset of colors, called a palette, from the full range of 224 = 16, 777, 216 colors. The drawback of this scheme is that it restricts the number of colors which may be simultaneously displayed. This paper addresses the problem of displaying color images using a restricted palette. Since natural images typically contain a large number of distinguishable colors, displaying such images with a limited palette is diﬃcult. The approach we take is structured around algorithms for performing two tasks. The ﬁrst task, called color palette design, selects the best possible set of colors for a particular image. The second task, called pixel mapping, associates each pixel of the image with a color from this palette to yield the highest quality image. A number of approaches [1, 2] have been suggested for the design of color palettes which involve iterative reﬁnement of some initial palette. However, these algorithms suﬀer from the disadvantage of being computationally intensive. In many applications, the eﬃciency of algorithms for performing color palette design and pixel mapping is very important. This is particularly true in applications involving large image data bases. In these applications, 2 images are in some common format which must be displayed on monitors with diﬀerent limitations on color palette size. In such an environment, the color palette design and the pixel mapping functions must be performed at the time an image is displayed. This makes computational eﬃciency of critical importance. Even if the color palette is transmitted with each image, standard image coding techniques require that the pixel mapping be performed locally at the time each image is displayed. This is because standard image coding techniques make use of the high correlation between the values of the primary colors of neighboring pixels to reduce transmission bit-rate. The color palette indices of neighboring pixels are not highly correlated and, therefore, cannot be coded with high eﬃciency. Thus, even if a color palette of correct size is provided with each stored image, the pixel mapping function must be performed after the image is retrieved from the data base and decoded into a full color format. The ﬁrst problem we address is the problem of designing color palettes. In general, a color palette will accurately display a particular image if the true color of each pixel in the image can be well approximated by some color from the palette. Heckbert [2] proposed a method for constructing an initial color palette by assigning to each color from the palette an approximately equal number of pixels in the image. This method is based on the construction of a binary tree in which each leaf contains an approximately equal number of similar pixel colors which can be described by a single quantization color. The tree is constructed by recursively determining the leaf node with the largest number of pixels and splitting it into two new nodes with equal numbers of pixels. To improve this initial color palette, Heckbert proposed an iterative procedure, derived from the Linde, Buzo, Gray (LBG) algorithm [3] for quantizer design, which attempts to minimize the total squared error (TSE) between the original pixels and colors assigned from the palette. Later, Braudaway [1] proposed constructing an initial color palette by centering colors at peaks in the color-value histogram. To avoid concentrating too many colors about one peak, the histogram values are reduced in a region surrounding each assigned color after the color is selected. Beginning with this 3 initial palette, Braudaway also used the iterative LGB algorithm as suggested by Heckbert. An approach similar to that of Braudaway’s has also been used by Gentile, Allebach and Walowit [4] to study the eﬀects of color transformations on the quality of quantized images. In both Heckbert’s and Braudaway’s methods, the initial condition is chosen to allocate more quantization levels to regions of the color space with larger numbers of pixels. However, neither initial condition attempts to minimize the TSE or any other objective measure of error. Therefore, the LBG algorithm is used to improve these initial color palettes by search- ing for a local minimum of the TSE. While the LBG algorithm does reduce the initial TSE, it is greatly inﬂuenced by these initial palettes. Consequently, the local minimum which is reached is unlikely to be close to the global minimum. In addition, the LBG algorithm is very computationally intensive since, at each iteration, it requires a search of the full color palette for each pixel. Also, the LBG algorithm destroys the tree structure of Heckbert’s initial palette. This is important since the tree structure could otherwise be used to reduce the computation required for palette searches. This paper describes a hierarchical approach to the design of color palettes for natural images. The proposed algorithms attempt to produce color palettes which minimized an objective error criteria. Therefore, they result in high quality displayed images with minimum artifacts [7, 8]. We use vector quantization (VQ) [9] as a basis for addressing the design of good color palettes. In this framework, the colors in the palette can be considered as codes in a code book, and the pixel mapping as the image pixel coding. In addition, our approach emphasizes the use of color palettes with a binary tree structure. This tree structure has the advantage of substantially reducing the computation required both to design a palette and to quantize colors using a palette. We develop a basic binary splitting approach which creates tree structured color palettes using a minimum TSE criteria. This allows for very eﬃcient pixel mapping by making search times proportional to the tree depth. Empirical evidence indicates that the best color palettes are not necessarily ones which minimize TSE. This is because the subjective error resulting from quantizing a pixel’s color 4 value can not be treated independently of the quantization of spatially adjacent pixels. Standard VQ based algorithms do not address many of these subjective qualities because they generate a codebook based on the distribution of pixel colors without consideration of their spatial relationships. We develop two approaches to modifying the basic binary splitting algorithm to account for these subjective qualities. The ﬁrst approach computes a weighted TSE (WTSE) by applying a subjective weighting to the squared error at each pixel and uses an extension of the basic binary splitting VQ algorithm to minimize this WTSE. The subjective weighting reﬂects variations in our ability to perceive small intensity changes as a function of the spatial intensity gradient at each pixel. The second approach modiﬁes the order in which the binary splitting algorithm splits nodes so as to reduce the correlation of quantization errors between adjacent pixels. This approach is based on reducing the sizes of interiors of constant color areas in the displayed image. The second part of this paper addresses the problem of optimum pixel mapping once a color palette has been selected. Pixel mapping involves assigning a color from the palette to each pixel in the image in a way which produces the highest quality display. Although the TSE can be minimized by assigning each pixel to the color that is closest to its original color, higher quality displays are often produced by using dithering methods to assign colors. These methods attempt to match the local color averages of the displayed image to the local color averages of the original image. A variety of dithering techniques have been investigated for displaying bi-level or half-toned images (i.e. a color palette containing only the colors black and white). Both Heckbert and Braudaway suggest the use of a dithering technique known as error diﬀusion [11] which was originally proposed for the display of bi-level images but is easily extended to multilevel quantization. Error diﬀusion can signiﬁcantly improve the quality of many displayed images when the palette size is small. However, when used in conjunction with an unstructured color palette, error diﬀusion requires a search of the full color palette for each pixel of the image and is therefore quite computationally intensive. We investigate a number of approaches for improving display quality given a ﬁxed color 5 palette. These methods produce more accurate average color at the cost of added noise and increased computation. By using error diﬀusion in conjunction with color palettes having a binary tree structure, we substantially reduce the computation required to quantize an image. In addition, we propose a modiﬁed algorithm for performing the error diﬀusion. This algorithm avoids nonlinear oscillations which can occur due to the nonuniform quantization of the color space. This results in better image reproduction with fewer artifacts. The paper is organized as follows. Section 2 starts by deﬁning the color space coordi- nate system in which the palette design algorithms operate. Section 2.1 describes the basic binary splitting approach to the design of color palettes using a minimum TSE criteria. Sec- tions 2.2 and 2.3 develop extensions to the basic binary splitting algorithm which account for subjective measures of image quality. Section 3 describes both ordered dither methods and methods for eﬃciently implementing error diﬀusion techniques using color palettes hav- ing a binary tree structure. Finally, section 4 reports on simulation results of the various techniques. 2 Color Palette Design We ﬁrst deﬁne some notation. The image is assumed to be on a rectangular grid of N pixels. The set of all grid points is denoted by S, and its members s ∈ S may be explicitly written as s = (i, j) where i is the row index and j is the column index. The color value of the pixel at grid point s is denoted xs = [rs , gs , bs ]t where the components are the red, green and blue tristimulus values for the pixel [13] and superscript t means transpose. For a typical monitor, there is a nonlinear relationship between the input value of the primary and its displayed intensity. For a particular primary such as red, this relationship may be approximated by Ir = rγ where Ir is the intensity, r is the input to the display and γ ≈ 2.3. More accurate models for this relationship generally include constant, linear and logarithmic terms [4]. In order 6 to compensate for this nonlinear relationship between input and luminance, it is common for video cameras and other image capture devices to predistort their output by the inverse relationship. (1/γ) r = Ir We shall refer to the predistorted primary color values as the gamma corrected colors. In eﬀect, this is the coordinate system being used if one manipulates color image data and directly displays it without correction. It is important to emphasize that the gamma corrected colors are not linear in intensity, and that the gamma correction, and therefore the resulting coordinate system, varies with changes in a display’s parameters. However, the gamma corrected colors have the advantage that they may be directly displayed without transformation, and that they are less susceptible to noise than the corresponding linear primary intensities [13]. Alternatively, standard coordinate systems have been deﬁned for the representation of colors. Two such coordinate systems are the L∗ a∗ b∗ and the L∗ u∗ v ∗ color spaces [12]. These coordinate systems are nonlinearly related to the primary intensities, and are chosen so that a ﬁxed euclidean distance represents a ﬁxed perceptual distance independent of position in the space. Gentile, Allebach and Walowit have studied the application of the L∗ u∗v ∗ coordi- nate system to the problems of image quantization [4], halftoning [5] and gamut mismatch [6]. This work indicates that these coordinate systems can substantially improve quantized image quality relative to standard coordinate systems which are linearly related to primary luminances. Therefore, it is reasonable to expect that perceptually based coordinate systems such as L∗ u∗ v ∗ could improve the quality of displayed images relative to gamma corrected coordinates. However, it will be assumed throughout this paper that the original image is speciﬁed in terms of gamma corrected coordinates, since this avoids the computational requirements of coordinate transformation methods. All, the techniques developed can of course be applied using perceptually based color spaces. 7 2.1 Binary Tree Palette Design The objective of the palette design algorithm is to partition S into M disjoint sets or clus- ters where M is the predetermined palette size determined by hardware constraints. The algorithm proposed here constrains the partitioning of S to have the structure of a binary tree. Each node of the tree represents a subset of S, and the children of any node partition the members of the parent node into two sets. The set of image pixels corresponding to node n is denoted Cn. For purposes of illustrating the operations of the algorithm, we will number the nodes so that the root is 1 and the children of node n are 2n and 2n + 1. Since the leaves of the tree form a partition of S, the pixels in S may be coded by representing all pixels in each leaf node with a single color in the palette. Only the members of the tree’s leaf nodes need be stored when the algorithm is implemented. Each of the leaf node sets can be stored as a linked list. When a node is split, the linked list may be reorganized into two lists without relocating any of the image color data. The method for generating the binary tree is speciﬁed by the number of leaves, M, the method of splitting a node into its two children, and the order in which the nodes are split. The methods used for splitting nodes and determining which nodes to split both attempt to minimize the total squared diﬀerence between the actual and quantized image. The TSE is deﬁned by T SE = ||xs − qn ||2 all leaves s∈Cn n where qn is the quantization value used to represent the colors in the set Cn . In order to restrict the complexity of the algorithm, second order statistical properties will be used to determine the order and manner in which nodes will be split. The three required cluster statistics are Rn = xs xt s (1) s∈Cn mn = xs s∈Cn Nn = |Cn | 8 where Rn is a 3x3 matrix, and mn is a 3 component vector. Since the mean value of a cluster is the point from which there is minimum squared deviation, the quantization value of a cluster, qn , is assumed equal to the cluster mean. mn qn = . Nn We also deﬁne the cluster covariance as ˜ 1 Rn = Rn − mn mt . n Nn The splitting of a node into two nodes is equivalent to choosing two new quantization levels, q2n q2n+1 , and associating each member of the cluster with the closer quantization level. This, in turn, is equivalent to selecting a plane which best splits the cluster’s colors. In the proposed algorithm, we determine the direction in which the cluster variation is greatest, and then split the cluster with a plane which is perpendicular to that direction and passes through the cluster mean. For a large cluster with Gaussian distribution it can be shown that this strategy is optimal. More speciﬁcally, we determine the unit vector e which maximizes the expression ˜ ((xs − qn )t e)2 = et Rn e . s∈Cn ˜ Since R is symmetric, the solution is the eigenvector, en , corresponding to the largest or ˜ principal eigenvalue λn of R. The total squared variation in the direction en is therefore ((xs − qn )t en )2 = λn . (2) s∈Cn Once the principal eigenvector has been determined, points in Cn can be sorted into two sets C2n and C2n+1 in the following way. C2n = {s ∈ Cn : et xs ≤ et qn } n n (3) C2n+1 = {s ∈ Cn : et xs > et qn } . n n 9 Finally, the new statistics for each node may be calculated by ﬁrst calculating R2n , m2n and N2n for node 2n, and then applying the relations R2n+1 = Rn − R2n (4) m2n+1 = mn − m2n N2n+1 = Nn − N2n . The order in which nodes are split is chosen with the objective of producing the greatest reduction in TSE at each stage of the algorithm. Because this strategy does not look ahead to the results of further splits, it is not necessarily optimal; however one would expect it to perform well in most practical situations. When a node is split its variance is mainly reduced along the direction of the principal eigenvector. Therefore, it is reasonable to assume that the reduction in TSE should be proportional to the total squared variation along the direction ˜ of the principal eigenvector, en , of Rn . For this reason, the principal eigenvalue, λn is used as a measure of the expected reduction in TSE if node n is split. Given this approximation, the best allocation of a single quantization level is to split the leaf with the largest principal eigenvector. The basic binary quantization algorithm may now be described. 1. Set C1 = S. 2. Calculate R1 , m1 and N1 . 3. Do the following M − 1 times 3.1 Find the leaf, n, such that λn is largest. 3.2 Use (3) to form the new nodes 2n and 2n + 1. 3.3 Calculate R, m and N for the new nodes using (4). For typical image sizes (e.g. 512x512 pixels) the computation is dominated by steps which must be performed at each pixel of the image. These steps consist of the splitting of clusters 10 using (3) and the computation of statistics using (4) and (1). Assuming an approximately balanced tree, each pixel of the image is involved in log2 M splits, each requiring 3 multi- plications to compute (3). The initial computation of the matrix R1 requires 6 multiplies per pixel due to the matrix’s symmetry. Assuming approximately balanced splitting of each node, 3N log2 M multiplies are required to compute the remaining values of Rn . This results is a total of 6N(log2 M + 1) multiplications and slightly fewer additions for the design of the color palette. Therefore, for M > 2, fewer multiplications are required for the design of the complete color palette than a single iteration of the LBG procedure described in section 2.4. 2.2 Subjectively Weighted TSE Criteria In this section, we develop a framework for incorporating subjective measures of quality through the use of a weighting function which accounts for local properties of the image. This weighting can be applied using the same framework of the basic splitting algorithm of section 2.1, with the substitution of a weighted total square error (WTSE) in place of the true mean square error. The WTSE is deﬁned as W T SE = Ws (x)||xs − qn ||2 all leaves s∈Cn n where Ws is a function of x in some neighborhood of s. Regions of the image which are subjectively more sensitive to errors in quantization are given a larger weight. By applying the same arguments used in deriving the basic splitting algorithm, we calculate the following forms for the new cluster statistics. Rn = Ws xs xt s (5) s∈Cn mn = Ws xs s∈Cn Nn = Ws s∈Cn ˜ 1 Rn = Rn − mn mt n Nn mn qn = Nn 11 We ﬁrst consider some properties that the weight Ws should possess. For simplicity, Ws will only depend on the gamma corrected luminance component, ys , of the image. In the NTSC system, ys is given by ys = [0.300, 0.586, 0.115]xs . (6) 1/γ If Is is the true luminance at position s, then ys is approximately given by ys ≈ Is . Weber’s law [14] suggests that any measure of subjective image quality should be invariant to linear scaling of the color component intensities. However, this is not true of the TSE based on gamma corrected color components. Scaling Is by the constant cγ results in y being scaled by c and the TSE being scaled by c2 . Therefore, to ensure the invariance of the WTSE to scaling of Is , Ws should have the property that 1 Ws (cx) = Ws (x) . (7) c2 This suggests that Ws should be inversely proportional to a quadratic function of a local linear function. The most prominent artifact of a limited palette is false contouring. False contours result from mapping smoothly varying regions of an image to a small number of colors from the color map. Instead of displaying slow variations of color across the region, smaller regions of constant color are displayed with abrupt color changes across region boundaries. These abrupt color changes are perceived as contours in the displayed image. A reasonable measure of the visibility of a false contour is the width of uniformly quan- tized region. Since the luminance component has the greatest variation, we will use it to estimate the likely size of false contours. Figure 1 is a schematic representation of a one- dimensional slice through the luminance component of an image along the direction of the gradient. The two curves are the true luminance and the quantized luminance. Notice that the width of the uniformly quantized region, ∆, is equal to the quantization step size, E, 12 ∆ Figure 1: This ﬁgure illustrates the relationship among the gradient of the image luminance (▽y), the size of quantization steps (E) and the width of regions with ﬁxed quantization value (∆). divided by the gradient size. This suggests that for a region of the image with uniform quantization ||xs − qn ||2 E2 ∆2 ≈ (area) ≈ (area) . s∈Cn || ▽ ys ||2 12|| ▽ ys ||2 12 In practice, the gradient of ys is not constant, so a more accurate estimate may be obtained by convolving the magnitude of the gradient with a smoothing kernel hs . The form of Ws which is used is 2 1 Ws = (8) h ∗ (min{|| ▽ y||, 16} + 2) where y takes on values between 0 and 255, and h is a kernel which ﬁlters the gradient estimates. The eﬀect of ▽y is limited to the range between 2 and 16 since values below 2 are visually indistinguishable, and values greater than 16 correspond primarily to edges. Also, these limits allow ﬁxed-point implementation of the algorithm by limiting dynamic range. The increase in computation due to the use of the WTSE criteria comes from the addi- tional multiplications required by the appearance of Ws in (5) and the computation of the weightings in (8). (5) requires an additional 1.5N(log2 M + 1) multiplications, increasing the complexity of the basic algorithm by 25%. The major computational cost of calculating 13 the weights is the computation of the gradient, the convolution with h and the squaring of the weights. The division can be avoided by using power of two quantization. The compu- tation required for ﬁltering depends on the ﬁlter size, and weights. The gradient calculation together with the implementation of a 5x5 ﬁlter of 1’s and the squaring of the result requires 13N arithmetic operations including 1N multiplications. In practice, the calculation of these weights adds substantially to the total computation time. 2.3 Erosion-Based Weighting Another method for improving the quality of the quantization is to develop an error criteria which depends directly on the qualities of the quantized image. Such a method can be used to dynamically focus quantization values in regions of the quantized image which are problematic. More speciﬁcally, if regions of the quantized image that may result in false contouring can be eﬃciently identiﬁed while the binary tree is being created, then the relevant clusters can be split to avoid artifacts. The visual eﬀect of false contouring is caused by the highly correlated nature of the error which results from large regions being quantized with a single value. Even if the TSE (or WTSE) is small in such regions the errors will be visually signiﬁcant because they will form well deﬁned patterns. However, regions of the image which are quantized with single values may be identiﬁed by searching the two dimensional ﬁeld of quantization code words for regions with large interiors. The interiors of each region formed by a ﬁxed code word value may be measured using the technique of erosion from morphological ﬁltering [15]. This is done by selecting a ﬁxed pattern of lattice points and counting the number of times the pattern may be uniquely positioned completely within a region consisting of a single code word value. This operation is shown graphically in Figure 2. If a square block of nine points is used as the eroding set, then this is equivalent to counting the number of pixels in a cluster for which all eight neighbors of that pixel belong to the same cluster. For this reason, the interior size ωn of cluster n is a measure of the number of interior pixels in the cluster. 14 Figure 2: Sample erosion with nine point block. After an initial allocation of M0 code words, the weights ωn may be used to bias the order in which clusters are split by more heavily weighting principal eigenvalues of clusters with large interiors. Instead of choosing the cluster with the largest eigenvalue, λn , the weighted eigenvalue, ωnλn , is used. All experimental results use M0 = (2/3)M. We have found that for palettes with 256 colors this value of M0 allocates a suﬃcient number of colors to eliminate regions with large interiors. The binary quantization algorithm with erosion-based weighting may now be described. 1 Set C1 = S. 2 Calculate R1 , m1 and N1 . 3 Do the following M0 − 1 times. 3.1 Find the leaf, n, such that λn is largest. 3.2 Use (3) to form the new nodes 2n and 2n + 1. 3.3 Calculate R, m and N for the new nodes using (4). 4 Calculate the interior weights, ωn , for all leaf nodes, n. 5 Do the following M-M0 times. 5.1 Find the leaf, n, such that ωn λn is largest. 15 5.2 Use (3) to form the new nodes 2n and 2n + 1. 5.3 Calculate R, m and N for the new nodes using (4). 5.4 Compute ω2n and ω2n+1 for the two new leaves. The calculation of the weights ωn represent only a modest overhead in computation because they can be eﬃciently computed using only compare and logical operations. Step 4 requires that an interior weight be computed for each region of the image. For a K point eroding set, the worst case number of comparisons required for step 4 is N(K − 1). However, this is a conservative estimate since the search for diﬀering quantization levels can be performed sequentially, and can be stopped when unlike quantization levels are found. Tests using a 9 point eroding set for quantizing a large sample of images have found that an average of 1.4N comparisons are needed for computing the interior weights. The total computation required for all applications of step 5.4 is generally less than is required for step 4. This is because the splitting of a cluster only eﬀects the interior points of the cluster being split. Therefore, time required to recompute these two weights is proportional to the number of pixels in the cluster being split. Since the number of pixels contained in the last M/3 of clusters split is generally much less then N, this represents a small amount of computation. 2.4 The LBG Algorithm The LBG Algorithm [3, 9] is a method used in VQ for iteratively reﬁning a code book. The algorithm is quite general, and has also been applied to wide variety of problems in pattern recognition and clustering [10]. Since the problem of designing an optimal code book is equivalent to ﬁnding a set of palette colors, {qn : 1 ≤ n ≤ M}, which best approximate the set of image pixel colors, {xn : 1 ≤ n ≤ N}, the LBG algorithm is directly applicable to the color quantization problem. In the case of minimum TSE palette design, the algorithm is deﬁned as follows: 1. Choose an initial color palette, {qn : 1 ≤ n ≤ M}. 16 2. Form M pixel clusters, Cm = {xn : ||xn − qm || ≤ ||xn − qk ||, 1 ≤ k ≤ M} 1 3. Recompute the color palette using, qm = xs |Cm | s∈Cn 4. Repeat 2 and 3 until no more changes occur or the decrease in TSE is less than a predetermined amount. Each step of the LBG algorithm causes the TSE to either decrease or remain the same. Therefore, the algorithm is guaranteed to converge to a minimum of the TSE. However, this minimum is generally not the global minimum of the TSE. Therefore, the initial palette plays a key role in determining the local minimum to which the algorithm converges. In fact, the experimental results of Section 4 indicate that the LBG algorithm will improve initial color palettes of poorly quantized images, but it will not signiﬁcantly change palettes of well quantized images. Consequently, we take the view that the LBG algorithm is best used as a post-processing algorithm for improving suboptimal initial palettes. A signiﬁcant disadvantage of the LBG algorithm is its computational cost. The forming of clusters in step 2 generally requires NM distance calculations, and this step must be performed in each iteration. Therefore, one iteration of the LBG algorithm requires 3NM multiplications. Finally, we noted that binary tree structure of a palette is destroyed by the LBG algorithm. Without the tree structure, pixel quantization requires a full search of the palette, or NM distance calculations. However, pixel quantization with the tree structure requires only N log2 M inner products and compares. This is particularly important in the error diﬀusion applications discussed below. 3 Pixel Mapping Once a color palette has been selected, the TSE of the displayed image is minimized by map- ping each pixel to the color from the palette which is closest to its original color. The pixel mapping techniques described in this section attempt to improve the quality of the displayed image by accepting a larger TSE so that the local average color of the displayed and original 17 images may be matched. These techniques reduce error power at low spatial frequencies and increase error power at high spatial frequencies where error is less noticeable. This section discusses two types of pixel mapping techniques, ordered dither and error diﬀusion, which have been used widely for the halftoned display of grey-scale images. The application of these techniques to the display of color images with limited color palettes raise diﬃculties which do not exist in the display of bi-level images. 3.1 Ordered dither Ordered dithering is a method for improving the appearance of a displayed image by adding a pseudorandom noise pattern, d(k,l) , to blocks of pixels, xs , before quantizing. Therefore, ˜ the image before quantization, x, is given by ˜ x(i,j) = d(i mod n,j mod n) + x(i,j) , where d(k,l) is a n by n matrix of dithering values. d(k,l) is chosen to have an average value of zero and an energy spectrum with minimum energy at low spatial frequencies. The amplitude of the noise pattern is chosen so that any area of constant color value is quantized into a variety of nearby color values, thereby breaking up regions of false contouring. This reduces the correlated error patterns that are characteristic of areas of false contouring. The application of ordered dithering to the display of color images is complicated by the nonuniform spread of colors in the palette. In image-speciﬁc color palettes, the distance between nearby colors varies signiﬁcantly across the color space. Therefore, it may be im- possible to determine an amplitude which will suﬃciently dither all areas of constant color value in the image without adding noticeable noise to other areas. One method of addressing this problem is to adjust the dithering to the distance and orientation between nearby colors in the palette. This can be done by ﬁrst determining two candidate quantization colors, q1 and q2 , which are nearest to the actual pixel color. The noise pattern value for the pixel can then be scaled by the distance between quantization colors and rotated in color space by the 18 ns H Figure 3: Block diagram of error diﬀusion ﬁlter. The quantization error is propagated forward to reduce the low frequency component of the error. orientation of the colors to form the dithered signal ˜ x(i,j) = (q2 − q1 )d(i mod n,j mod n) , +x(i,j) (9) where d takes on scalar values on the interval [−1/2, 1/2]. The dithered value can then be quantized to one of the two levels. This approach adjusts the dithering to the non-uniform spread of quantization colors; however, it has a number of disadvantages. In regions which change signiﬁcantly enough to cause q1 and q2 to change, the dither noise will be modulated by the changing quantization colors. This will cause the spectrum of the dither noise to spread to low spatial frequencies. Because the dither is only added along the direction of the diﬀering quantization values, the average error in directions orthogonal to that component will not be eﬀected. The compu- tational cost of searching for candidate quantization colors can be reduced by restricting the search to colors which appear in a limited neighborhood of the pixel; however, this search still possesses a substantial computational overhead. Finally, no ordered dithering method guarantees that after quantization the error spectrum will continue to be concentrated at high spatial frequencies. 19 3.2 Error diﬀusion The object of error diﬀusion is to quantize the image in such a way that the average value of the quantized image is the same as the average value of the true image [11, 16]. Figure 3 illustrates the method. The pixels are chosen in some ordering (raster ordering is commonly used), and the residual quantization error is propagated forward to unquantized pixels. The ˜ error is propagated by computing an adjusted value of the pixel color, xs , formed by summing the actual color with a weighted sum of previous errors. The equations for error diﬀusion are as follows ˜ xs = xs + hs−k nk (10) k<s x qs = Q(˜s ) ˜ ns = xs − qs , where s − k = (s1 − k1 , s2 − k2 ) and s < k means (s1 = k1 & s2 < k2 ) or s1 < k1 . We note that while these equations have a somewhat diﬀerent form then those originally presented by Floyd and Steinberg, they are equivalent. Since xs is a vector, (10) describes a system of vector equations where hs is a matrix for each value of s. For simplicity, the h is usually chosen to be diagonal. The two-dimensional Fourier transform of h is given by H(ω) = hs e−jω·s , s>0 where ω = (ω1 , ω2 ) and · is the notation for inner product. Notice that the variable, ns , is the error generated by the quantizer, but it is not the diﬀerence between the actual and quantized image. The actual displayed image error and its relationship to the quantizer error is given by εs = xs − qs x = xs − (˜s − ns ) = ns − hs−k nk . k<s 20 The two-dimensional Fourier transform then yields E(ω) = (I − H(ω)) N(ω) . Therefore, we see that the spectral components of the displayed error may be shaped by choosing the ﬁlter I − H(ω) correctly. In general, this ﬁlter should have a high-pass char- acteristic since it is desirable to suppress the low-frequency error components. Billotet- Hoﬀmann and Bryngdahl have suggested a simple ﬁlter kernel which we have used in all experimentation [16]. The methods of color palette generation described in the previous sections all generate a color palette and a mapping from the image pixels to that palette. However, in general, palette design and pixel quantization can be performed separately. In the case of minimum TSE coding using an unstructured color palette, quantization is performed by Q(x) = arg min{||x − qn ||2 } . q n This is very computationally intensive since quantizing an image requires NM distance calculations, each of which requires three multiplies, three subtractions and a comparison. In the case of a palette with binary tree structure, an approximate minimum TSE coding, Qb (x), can be performed with order N log M computation. This is done by storing the principal eigenvectors and cluster means of all the internal tree nodes during the tree construction. These values can then be used to uniquely descend through the tree using the rule from (3). Notice, that both types of quantization can be performed for colors outside the original image. This is important since error diﬀusion will require the quantization of colors not in the original image. However, this leads to a unique problem associated with the application of error diﬀusion to image speciﬁc palettes. The advantage of an image-dependent palette is that quantization values are densely spaced only in regions of the color space with signiﬁcant numbers of pixels. This advantage can be quite signiﬁcant because for typical images a large amount of the color space is empty. The situation may be visualized by considering a region of an image in which the 21 color varies smoothly (such a region is more likely to exhibit the problems typical of a limited palette.) The image is then a smooth mapping from a two-dimensional surface into a three- dimensional space. The range of that mapping is, in general, a manifold with dimension ≤ 2. In particular, the local dimension of the manifold is given by the rank of the matrix ∂r ∂g ∂b ∂i ∂i ∂i ∂r ∂g ∂b ∂j ∂j ∂j where i and j are temporarily considered to take values on the real line. Most often this matrix will have rank 2 since the color components are likely to vary with some degree of independence. However, the manifold will have dimension 1 if two of the components are a function of the third. This can occur if the region has ﬁxed hue and saturation and varies only in intensity. The most degenerate case is when the color of an region is ﬁxed and the manifold is zero-dimensional. When the color palette is designed, the majority of colors are allocated around these surfaces in the color space. Unfortunately, the sparse nature of these surfaces can cause problems when colors outside the set are quantized. Error diﬀusion accumulates errors made in the pixel coding until those errors are negated by errors of opposite polarity. However, if all the quantization levels fall to one side of the manifold then there will be no local colors to oﬀset accumulated error. This situation can occur if the manifold is a convex surface since the minimum TSE criteria will tend to allocate color on the concave side of the manifold. Ultimately, the error will become so great that either the color will become saturated to some extreme value, or more likely, a color from another manifold (region of the image) will be chosen. In this situation the assumption of independence between the input to the quantizer and the quantization error becomes invalid, and the nonlinear feedback structure of error diﬀusion can lead to a unstable condition in which large oscillations in quantization error occur between colors far from the original color. This, of course, creates noticeable artifacts in the resulting image. We would like to constrain the ﬁlter, H in ﬁgure 3, to operate in a region in which the 22 λ ns ns H Figure 4: Block diagram of modiﬁed error diﬀusion ﬁlter. The feedback is clipped if mag- nitude of the error is signiﬁcantly larger than the longest axis of the corresponding cluster formed in the palette design process. quantizer error, ns , is of the order of the distance between quantization colors within the manifold. A reasonable method for detecting when the ﬁlter is oscillating is to compare the error magnitude to the diameter of the cluster associated with the quantization color. This can be done by storing along with the quantization color the associated principle eigenvalue of the cluster. Since the principle eigenvalue of the cluster is proportional to the square of the cluster’s diameter along its longest axis, we can determine when oscillations occur √ by comparing the error magnitude to λα where α is a constant. When oscillations are detected, the ﬁlter can then be damped by opening the feedback loop, and the unadjusted ˜ pixel intensities, xs , can be quantized instead of the highly biased values, xs . This combined √ operation is simply performed by “clipping” the input of the ﬁlter to a magnitude of λα. ns if |ns |2 < λα2 ˜ ns = 0 otherwise A diagram of the modiﬁed error diﬀusion process is shown in Figure 4. 23 4 Results The previous sections have proposed a number of algorithms, each of which attempts to maximize performance by minimizing a measure of displayed image distortion (e.g. TSE, WTSE and region interior size). This section compares these algorithms by measuring their performance with respect to all of the quality measures which have been discussed. Besides providing a means for comparing algorithms, the results of this section oﬀer insights into the validity and signiﬁcance of the various performance criteria which have been proposed. The algorithms were tested on a variety of images available on public image databases, and four images were selected for comparisons. Figure 5 shows monochrome versions of the four color images used. Images 1 and 2, known as Lenna and Tiﬀany respectively, are characterized by large smooth regions of light skin tones which are particularly diﬃcult to quantize transparently. This is probably due to the combined eﬀects of a sensitive region of luminance/hue together with strong prior expectations on the part of the viewer. Image 1 contains colors from a larger area of the color space than image 2, and it suﬀers from the most noticeable false contouring of the four images used for comparisons. Image 3, known as F16, is relatively easy to quantize transparently because its large saturated white background contains colors concentrated in a small region of the color space. Image 4, known as Peppers, contains colors from many disjoint regions of the color space and produces signiﬁcantly higher TSE than the other images. Table 1 compares ﬁve color quantization algorithms with respect to three performance criteria. The comparisons are shown for images 1 and 4 which are the most challenging images to quantize. The ﬁrst algorithm, labeled (8,8,4), uses a image independent separable color map with 8 red, 8 green, and 4 blue quantization levels. The second algorithm, labeled HIST, is a generic histogram-based method similar to that used for the initialization step in [1]. The ﬁnal three algorithms, labeled BS, WBS, and EBBS, are the basic binary splitting algorithm of Section 2.1, the weighted binary splitting algorithm of Section 2.2, and the erosion-based binary splitting algorithm of Section 2.3. The WBS method uses a 5x5 moving 24 Lenna; Tiﬀany F16; Peppers a b c d Figure 5: Monochrome versions of color images used in comparisons of algorithms. The images are referred to as a) Lenna; b) Tiﬀany; c) F16; d) Peppers. average ﬁlter to smooth the gradient components, and the EBBS uses the nine point block of Figure 2. Three criteria are used to judge algorithm performance: the RMS error given by T SE W T SE RMSE = N , the WRMS error given by W RMSE = N , and the average codeword interior size (ACIS) computed with the nine point eroding block. Image 1 Image 4 Algorithm RMSE WRMSE ACIS RMSE WRMSE ACIS (8,8,4) 23.66 23.35 187 22.30 21.92 189 HIST 7.87 7.36 28 11.39 10.22 31 BS 5.96 5.28 5.7 8.26 7.28 6.0 WBS 6.31 5.08 3.4 8.73 7.06 3.2 EBBS 6.10 5.27 2.3 8.43 7.27 1.5 Table 1: Color quantization algorithms compared according to RMSE, WRMSE, and average codeword interior size (ACIS). Several observations can be made from Table 1. It is clear that the algorithms generating image dependent color maps yield signiﬁcantly better performance than the ﬁxed (8,8,4) 25 color map according to all performance criteria. The generic histogram algorithm, which is not developed from maximizing any performance criterion, is outperformed by the three tree structured algorithms according to all criteria. It should be noted that the displayed images produced by the ﬁrst two algorithms show noticeable false contouring and are clearly of lower subjective quality than those produced by the ﬁnal three algorithms. The diﬀerences between the tree-structured algorithms are less obvious and, in many cases, can be identiﬁed only under close examination. It is not surprising that, for each of the three criteria, the algorithm giving the best performance is the one that was developed to maximize performance relative to that criteria. However, some insight into the signiﬁcance of the three criteria can be gain by comparing the results of the three algorithms. The BS and EBBS algorithms diﬀer only in the selection of codewords which are split by the last 25% of the node splitting operations. Although the EBBS gives a 2% increase in the RMSE by using the erosion-based criterion for selecting codewords to split, it achieves greater than 50% reduction in ACIS, thus suggesting that the RMSE criterion is a poor measure of false contouring in the displayed image. The EBBS algorithm performs roughly equal to the BS algorithm relative to the WRMSE criterion, reﬂecting the fact that codewords with large interiors are often found in regions which are given high weights by equation (8). The relationship between the weighting function of Section 2.2 and the erosion-based weighting of Section 2.3 is also reﬂected in the performance of the WBS algorithm. Although it does not directly attempt to reduce interior size, Table 1 shows that the WBS algorithm produces signiﬁcantly smaller ACIS compared with the basic BS algorithm. Both the WBS and EBBS algorithms produce noticeably less false contouring than the BS algorithm. The LBG algorithm described in Section 2.4 can be applied to improve the color maps generated by each of the algorithms considered in Table 1, and Figure 6 shows the RMSE that results at each iteration. Since the ﬁrst two algorithms were not designed to minimize any error function, the early LBG iterations oﬀer signiﬁcant reduction in RMSE for these 26 BS 15 WBS RMS EBBS Error 10 8 6 0 5 10 Iterations Figure 6: RMS Error versus the number of iterations of the LBG algorithm. Each plot corresponds to an initial color palette generated using one of the ﬁve quantization algorithms. algorithms and then quickly reach a local minimum in RMSE. Although the LBG iterations oﬀer signiﬁcant improvements to the subjective quality of these algorithms, the displayed im- ages produced by the resulting local minima still exhibit very objectionable false contouring in the case of the ﬁxed (8,8,4) map and noticeable false contouring in the case of the palette generated by the HIST algorithm. The last three algorithms start near local minima and realize small improvements from the LBG iterations. For these algorithms, the LBG itera- tions produce no noticeable change in the subjective quality of the displayed images. Tests on all four images show that, although the LBG algorithm does reposition palette colors at each iteration, it reaches a local minimum before it can perform any major reallocation of colors in the color space. Because the general allocation of palette colors to regions in the color space remains the same before and after the LBG iterations, the quality of displayed images produced by the LBG algorithm is very dependent on the initial palette selected. Table 2 compares the computational complexity of the algorithms by showing their ex- ecution times on a Sun Sparcstation and a rough measure of required multiplications. The 27 execution times are averaged over the four images which were quantized. Since the three tree- structured algorithms perform pixel mapping during the palette design process, the times shown for the ﬁrst two algorithms also include both palette design time and pixel mapping time. The time listed for the LBG algorithm reﬂects the execution time of a single iteration utilizing a recently proposed fast method for computing nearest neighbors [17]. The second column of Table 2 lists the number of multiplications used by each algorithm in terms of the number of pixels, N, and the number of palette colors, M. This computation count relates the complexity of the last three algorithms and the LBG algorithm by comparing the multiplications required for computing distances and autocorrelation matrix elements. However, this count does not measure the complexity of the ﬁrst two algorithms since their computational complexity is dominated by operations other than multiplication. Since the multiplicative complexity of the fast algorithm in [17] varies signiﬁcantly as a function of the map, the multiplication counts given for the HIST and LBG algorithms reﬂect the com- plexity of the full search algorithm for computing nearest neighbors instead of the algorithm used in the timing comparisons. Algorithm Execution Time (sec.) Mult. Count (8,8,4) 4.0 – HIST 305.5 3NM BS 56.3 6N(log2 M + 1) WBS 127.6 7.5N(log2 M + 1) + N EBBS 59.6 6N(log2 M + 1) 1 iteration of LBG 32.1 3NM Table 2: Computational complexity of color quantization algorithms. Column 1 lists run times for each algorithm on a Sun Sparcstation, and column 2 lists the required number of multiplies for a image with N pixels and a palette with M colors. The error diﬀusion methods for pixel mapping attempt to shape the error spectrum in order to minimize low-frequency error at the expense of higher RMS error. Figure 7 shows error spectra of quantized versions of Image 1 with and without error diﬀusion. Both plots use a 64 color palette generated by the EBBS algorithm. The algorithm of Figure 7a maps each color to the nearest neighboring palette color while the algorithm of Figure 7b uses the 28 modiﬁed error diﬀusion method with (α = 6) as described in Section 3.2. The plots show that, while nearest neighbor pixel mapping minimizes overall error power, error diﬀusion produces a shaped error spectrum with lower error energy at low frequencies. Table 3 shows this same result numerically by comparing these same two methods of pixel mapping using the the RMSE and a ﬁltered RMSE (FRMSE) performance criteria. The FRMSE is computed by low pass ﬁltering the error before computing the RMS deviation. This reﬂects the frequency sensitivity of the human visual system together with the spatial bandwidth of the display device. Algorithm RMSE FRMSE Nearest Neighbor 8.87 5.49 Error Diﬀusion 10.53 4.70 Table 3: RMSE and ﬁltered RMSE (FRMSE) produced by the nearest neighbor and error diﬀusion methods of pixel mapping. Section 3.2 discussed how the nonuniform distribution of colors in the palette can lead to quantization error oscillations and proposed the modiﬁed error diﬀusion method to eliminate these oscillations. One method of detecting oscillations is to measure the output error of the quantizer in the error diﬀusion loop. When this error is large it suggests that the input values to the quantizer are far from any available palette colors. In practice large errors result in very poor quality images often containing high frequency patterns of spots. Table 4 uses the image Peppers (the worst case of the four images) to show the dependence of the quantizer output RMSE on the value of the clipping parameter α and the method of quantization. It is useful to note the conventional error diﬀusion may be thought of as modiﬁed error diﬀusion with α = ∞. The two types of quantization methods used are tree-based quantization, and a full search quantizer which ﬁnds the minimum distance palette color. In addition to the quantization RMSE, the percentage of pixels at which clipping occurs is also listed. This, together with the FRMSE, gives an indication as to when an excessive amount of clipping is occurring due to a small value of α. It is interesting to note that oscillation occurred both with and without the full search quantizer, and that when clipping was used the two 29 Min distance Power Spectral Density (a) MED Power Spectral Density (b) Figure 7: Spatial spectra of the quantization error in two displayed images. Each plot shows the RMS spectrum of the luminance component of the error. The plots correspond to the error spectrum from a) minimum distance pixel mapping and b) the modiﬁed error diﬀusion. 30 quantization methods had comparable performance. This table also indicates that α = 6 yields a reasonable compromise between oscillation damping and restricted limiting. Algorithm Quantizer RMSE % of pixels clipped FRMSE ED (tree search; α = ∞) 426.9 0 8.24 ED (full search; α = ∞) 242.4 0 6.68 MED (tree search; α = 6) 26.4 8.1 4.89 MED (full search; α = 6) 25.6 7.7 4.70 MED (tree search; α = 3) 16.5 22.2 4.99 MED (full search; α = 3) 16.0 21.7 4.83 Table 4: Performance comparisons of error diﬀusion and modiﬁed error diﬀusion methods for pixel mapping. The parameter α controls the amount of quantization error which can occur before the feedback signal is clipped. In order to illustrate the artifacts present in the quantized images, we have included monochrome versions of some of the quantized images. These images were generated by using the luminance component of the quantized colors as deﬁned in (6). While distortions in color are not visible, false contouring can still be seen. Each ﬁgure shows the center region of the image Lenna enlarged by a factor of two. This emphasizes smooth regions in her shoulder and face which are diﬃcult to quantize. Figure 8 contains the results of the ﬁve color quantization algorithms, ﬁxed (8,8,4), HIST, BS, WBS and EBBS. The ﬁxed color map shows very severe degradation and contouring. The result of the histogram-based method is signiﬁcantly better than the ﬁxed map, but it still contains severe artifacts in areas of smooth color variation. The third image, generated using the basic binary splitting algorithm, has relatively good but not transparent quality. The areas above “Lenna’s” lips and on her forehead and along her shoulder show false contours. The contours along her shoulders and face are signiﬁcant when viewed on a high quality display. Both the WBS and EBBS algorithms are signiﬁcant improvements over the basic algorithm, but the EBBS exhibited slightly more reliable performance with substantially less computation than the WBS algorithm. For 256 color palettes, we have found that both methods yield transparent quality for a variety of images. The advantage of the erosion method is that the actual quantization quality is measured during the palette design. In contrast, the weighting method 31 predicts problematic areas of the image. The results of two error diﬀusion algorithms using a 64 color EBBS palette are also shown in Figure 9. Both images used tree-based quantizers, but the ﬁrst image used conventional error diﬀusion and the second used the modiﬁed error diﬀusion of Section 3.2 with α = 6. The conventional error diﬀusion method resulted in saturated regions and spotty patterns caused by the large oscillations of the error diﬀusion ﬁlter. The modiﬁed error diﬀusion eliminated these artifacts and improved the display quality relative to minimum TSE pixel mapping. However, the image quality is still some what degraded relative to the original. We have found that error diﬀusion can dramatically improve the quality of an image generated from a excessively small palette, but it generally does not produce transparent results since in the process it adds some distinguishable amount of noise. 5 Conclusion A variety of methods where explored for designing and using hierarchical image speciﬁc color palettes. The advantage of the hierarchical design is that nearly optimal color palettes can be designed in O(N log M) time where N is the number of image pixels and M is the number of colors in the palette. In addition, the hierarchical structure allows colors outside the image to be quantized in O(N log M) time. This computational savings is important in dithering applications. The color palette was improved by modifying the minimum total square error criteria to incorporate more subjective measures of image quality. This was done using two methods: subjective weighting and erosion. Erosion was found to require less computation, but both methods were found to yield very high quality results for palettes of size 256. The techniques of ordered dithering and error diﬀusion were extended for use with image speciﬁc color palettes. Basic error diﬀusion exhibited instability due to the sparse nature of the color palette, but a modiﬁed error diﬀusion technique was proposed to limit this instability when it occurred. Modiﬁed error diﬀusion substantially reduced artifacts caused by excessively small palettes of size ranging from 32 to 128 colors. 32 References [1] G. Braudaway, “A Procedure for Optimum Choice of a Small Number of Colors from a Large Color Palette for Color Imaging,” Electronic Imaging ‘87, San Francisco, CA, 1987. [2] P Heckbert, “Color Image Quantization for Frame Buﬀer Display,” Computer Graphics, vol. 16, no. 3, pp. 297-307, July 1982. [3] Y. Linde, A. Buzo and R. Gray, “An Algorithm for Vector Quantizer Design,” IEEE Trans. Commun., vol. COM-28, no. 1, pp. 84-95, Jan. 1980. [4] R. Gentile, J. Allebach and E. Walowit, “Quantization of Color Images Based on Uni- form Color Spaces,” Journal of Imaging Technology, vol. 16, no. 1, pp. 11-21, Feb. 1990. [5] R. Gentile, E. Walowit and J. Allebach, “Quantization and Multilevel Halftoning of Color Images for Near-Original Image Quality,” J. Opt. Soc. Am. A, vol. 7, no. 6, pp. 1019-1026, June 1990. [6] R. Gentile, E. Walowit and J. Allebach, “A Comparison of Techniques for Color Gamut Mismatch Compensation,” to appear in J. Imaging Technol. [7] C. Bouman, and M. Orchard, “Color Image Display with a Limited Palette Size,” Proc. of SPIE Conf. on Visual Comm. and Image Proc., pp. 522-533, Philadelphia, PA, Nov. 8-10, 1989. [8] C. Bouman, Hierarchical Modeling and Processing of Images. Ph.D. thesis, Princeton University, October 1989, Department of Electrical Engineering. [9] R. Gray, “Vector Quantization,” IEEE ASSP Mag., vol. 1, pp. 4-29, April 1984. [10] R. Duda and P. Hart, Pattern Classiﬁcation and Scene Analysis, Wiley, New York, NY, 1973. 33 [11] R. Floyd and L. Steinberg, “An Adaptive Algorithm for Spatial Gray Scale,” Proc. of the SID, vol. 17, no. 2, pp. 75-77, 1976. [12] A. Robertson, “The CIE 1976 Color-Diﬀerence Formulae,” Color Research and Appli- cation, vol. 2, no. 1, pp. 7-11, Spring 1977. [13] A. Netravali and B. Haskell, Digital Pictures. Plenum, New York, 1988. [14] B. Rogowitz, “The Human Visual System: A Guide for the Display Technologist,” Proc. of the SID, vol. 24, no. 3, pp. 235-252, 1983. [15] J. Serra, Image Analysis and Mathematical Morphology. Academic Press Inc., Orlando, 1982. [16] C. Billotet-Hoﬀmann and O. Bryngdahl, “On the Error Diﬀusion Technique for Elec- tronic Halftoning,” Proc. of the SID, vol. 24, no. 3, pp. 253-258, 1983. [17] M. Orchard, “A Fast Nearest Neighbor Searching Algorithm with Applications in Vector Quantization,” in preparation. List of Figures 1 This ﬁgure illustrates the relationship among the gradient of the image lumi- nance (▽y), the size of quantization steps (E) and the width of regions with ﬁxed quantization value (∆). . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2 Sample erosion with nine point block. . . . . . . . . . . . . . . . . . . . . . 16 3 Block diagram of error diﬀusion ﬁlter. The quantization error is propagated forward to reduce the low frequency component of the error. . . . . . . . . . 20 4 Block diagram of modiﬁed error diﬀusion ﬁlter. The feedback is clipped if magnitude of the error is signiﬁcantly larger than the longest axis of the corresponding cluster formed in the palette design process. . . . . . . . . . . 24 34 5 Monochrome versions of color images used in comparisons of algorithms. The images are referred to as a) Lenna; b) Tiﬀany; c) F16; d) Peppers. . . . . . . 26 6 RMS Error versus the number of iterations of the LBG algorithm. Each plot corresponds to an initial color palette generated using one of the ﬁve quantization algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 7 Spatial spectra of the quantization error in two displayed images. Each plot shows the RMS spectrum of the luminance component of the error. The plots correspond to the error spectrum from a) minimum distance pixel mapping and b) the modiﬁed error diﬀusion. . . . . . . . . . . . . . . . . . . . . . . . 31 8 A comparison of color palettes using enlarged monochrome versions of color images. Each palette contains 256 colors. a) Original image is shown together with quantized versions using b) an image independent color map of 8 red, 8 green and 4 blue; c) histogram based method (HIST); d) binary splitting method (BS); (continued on next page) . . . . . . . . . . . . . . . . . . . . . 38 8 e) weighted binary splitting (WBS); f) erosion based binary splitting (EBBS). 39 9 Monochrome versions of quantized color images using error diﬀusion methods. Each image uses a 64 color palette generated with the EBBS method. a) Result of conventional error diﬀusion, and b) modiﬁed error diﬀusion using α = 6 and a tree search quantizer. . . . . . . . . . . . . . . . . . . . . . . . . 39 List of Tables 1 Color quantization algorithms compared according to RMSE, WRMSE, and average codeword interior size (ACIS). . . . . . . . . . . . . . . . . . . . . . 26 2 Computational complexity of color quantization algorithms. Column 1 lists run times for each algorithm on a Sun Sparcstation, and column 2 lists the required number of multiplies for a image with N pixels and a palette with M colors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 35 3 RMSE and ﬁltered RMSE (FRMSE) produced by the nearest neighbor and error diﬀusion methods of pixel mapping. . . . . . . . . . . . . . . . . . . . . 30 4 Performance comparisons of error diﬀusion and modiﬁed error diﬀusion meth- ods for pixel mapping. The parameter α controls the amount of quantization error which can occur before the feedback signal is clipped. . . . . . . . . . . 32 36 Original Fixedmap HIST BS a b c d Figure 8: A comparison of color palettes using enlarged monochrome versions of color images. Each palette contains 256 colors. a) Original image is shown together with quantized versions using b) an image independent color map of 8 red, 8 green and 4 blue; c) histogram based method (HIST); d) binary splitting method (BS); (continued on next page) 37 WBS; EBBS e f Figure 8: e) weighted binary splitting (WBS); f) erosion based binary splitting (EBBS). ED; MED a b Figure 9: Monochrome versions of quantized color images using error diﬀusion methods. Each image uses a 64 color palette generated with the EBBS method. a) Result of con- ventional error diﬀusion, and b) modiﬁed error diﬀusion using α = 6 and a tree search quantizer. 38

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 3 |

posted: | 4/22/2012 |

language: | |

pages: | 38 |

OTHER DOCS BY bolonsafro

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.