Application of the ILO International Classification of Radiographs of Pneumoconioses to Digital Chest Radiographic Images

A NIOSH Scientific Workshop

DISCLAIMER: The findings and conclusions in these proceedings are those of the authors and do not necessarily represent the official position of the National Institute for Occupational Safety and Health (NIOSH). Mention of any company or product does not constitute endorsement by NIOSH. In addition, citations to Web sites external to NIOSH do not constitute NIOSH endorsement of the sponsoring organizations or their programs or products. Furthermore, NIOSH is not responsible for the content of these Web sites.

Assuring image quality for classification of digital chest radiographs

Ehsan Samei, PhD
Carl E Ravin, MD

Duke Advanced Imaging Laboratories (DAI Labs)
Department of Radiology
Duke University Medical Center, Durham, NC 27705
Phone: 919-684-1440
Fax: 919-684-1491
Email: samei@duke.edu
 

Introduction

Digital radiography (DR) offers notable advantages when compared to its film-screen counterpart. As noted in an earlier paper in these proceedings, attributes of DR include a wide dynamic range during image acquisition, the ability to post-process the images, electronic archival and distribution, and the potential for automated analysis and quantification of data. These characteristics provide unique benefits for the identification and classification of pneumoconiosis. However, they may only be realized with proper implementation and utilization of the technology. As such, quality assurance is an integral part of a digital radiography operation.

Quality assurance (QA) and quality control (QC) are not new concepts in medical imaging. However, when utilizing digital radiography, and particularly when there is interest in the extraction of quantitative information from images, QA and QC become essential. Two key attributes of digital radiography are the “fluidity” of image quality, and the ability to quantify image information. The potential to maximize the advantages of these two apparently contradictory attributes makes approaches to the quality of digital radiography unique. For example, a digital image can take on any number of appearances depending on the post-processing technique applied. However, the classification of disease in quantitative terms using digital images makes it essential that the images are processed in a predictable standardized fashion. Furthermore, if the classification of disease relies, at least in part, on automated analysis, the format, exposure dependency, and attributes of the image must be consistent, so that quantification can be performed with accuracy and precision. A rigorous quality control program is needed to enable optimum implementation of digital radiography.

In this paper, we outline the quantitative metrics of image quality, the elements of quality control for DR, and finally suggest requirements for classification of pneumoconiosis using either visual or automated approaches.

Quantitative Metrics of Image Quality

Imaging performance using digital radiography systems is based on attention to three fundamental aspects of image quality: resolution, noise, and signal-to-noise ratio. Quality control methods generally correspond to these three aspects.

Resolution: The resolution of a medical imaging system refers to the ability of the system to represent distinct anatomical features within the object being imaged. The resolution of an imaging system is best characterized in terms of its modulation transfer function (MTF), a measure of the ability of the system to reproduce image contrast from subject contrast at various spatial frequencies, or levels of detail (Figure 1) (1). Most radiographic systems are able to render lower frequencies (i.e., coarser detail) better than the higher frequencies (i.e., finer detail), leading to a loss of image sharpness. The MTF is a plot of the ratio of the output-to-input modulations as a function of their spatial frequency. The higher the MTF, the better the sharpness and resolution of an image, as illustrated in Figure 2.

Figure 1, Schematic of the MTF representing the resolution attributes of a digital radiographic system.

The resolution properties of digital radiographic systems can be ascertained by measuring the blurriness of images obtained from sharp objects. Extensive experimental methods have been developed for the assessment of the MTF of digital radiographic systems from such test objects (2-4).

Noise: Noise, in the context of quality control, refers to superfluous variations within an image that do not originate within the imaged subject, and that interfere with the visualization of an anatomic abnormality of interest, and thus with the interpretation of the image. While often quantified in terms of variance or standard deviation, radiographic noise is best characterized by the noise power spectrum (NPS) (Figures 3-4). The NPS defines the magnitude of noise within an image associated with specific spatial frequencies (i.e., levels of coarseness) of the noise (5, 6). The integral of the NPS is equal to the noise variance.

Inherent fluctuations associated with acquisition of a digital radiograph are best revealed when viewing a uniform image with no object in the field of view. Broad, large-scale variation in such an image is conventionally characterized as non-uniformity, while finer-scale fluctuations are characterized as noise. Similar to MTF, extensive experimental methods have been developed to measure the NPS of digital radiographic systems from such uniform images (7-9).

Figure 2, High MTF (left) and low MTF (right) reflecting the resolution properties of a magnified image.

Figure 3, Schematic of the NPS (one-dimensionally) representing the noise attributes of a digital radiographic system.

Figure 4, Correlated NPS (a) and uncorrelated NPS (b) reflecting the noise texture properties of a magnified image.

Signal-to-Noise Ratio: Resolution, described in terms of the MTF, reflects the ability of the imaging system to represent signal (i.e., contrast) within the image. Noise or the NPS, on the other hand, reflects the noise aspect of system performance. Image quality, in terms of the ability to see pathology of interest within an image, depends on a combination of these attributes in the form of the signal-to-noise ratio (SNR). Pioneering work by Albert Rose has demonstrated that (SNR)2 is inversely proportional to the image contrast, determines and the diameter of objects that can be reliably detected in radiographic images (10). Images with higher SNR can render objects of lower contrast and smaller diameter.

Due to detector inefficiencies, non-x-ray-quanta sources of noise, and added blur in image formation, the magnitude of the SNR within a radiographic image is always less than that dictated by the incident exposure, even if all the x-ray quanta were to be most efficiently used to form the image. The ratio of actual SNR2 to ideal SNR2, known as the detective quantum efficiency (DQE), is a metric commonly used to characterize the intrinsic SNR performance of a digital radiographic detector (3, 11). Its value is always less than ideal unity, i.e., 100%.

The formulation and measurement of the DQE does not take into account the influence of the focal spot blur, magnification, scattered radiation, and anti-scatter grid on the SNR obtainable from a digital radiographic system. A recent extension of the concept of the detector DQE to system DQE, known as effective DQE (eDQE), has further included those factors so as to quantify the actual SNR obtainable from a digital radiographic system (12-14). The higher the DQE or eDQE values, the better the SNR characteristics of the detector or the system, respectively.

Quality Control of Digital Radiography Systems

To assure reliable performance and reproducible results from a digital radiographic system, the system needs to be properly installed, maintained, and monitored through a quality control program. A proper QC program consists of a number of key components.

Acceptance testing: Upon installation and prior to clinical use, a digital radiographic system needs to undergo an acceptance testing procedure. Such an undertaking insures that the device is capable of delivering the basic expected safety and performance requirements, which ideally are outlined in the purchase contract. It provides the basic performance attributes of the system in terms of resolution, noise, and SNR, necessary to enable the extraction of quantitative image features from images. Acceptance testing also establishes the baseline performance characteristics as a starting point for subsequent periodic quality control tests.

Key aspects of the system performance to be included in acceptance testing are the MTF, the NPS, the DQE, scatter fraction, and, ideally, the eDQE at exposure levels representing those the system is designed to utilize. The knowledge of these inherent quantitative metrics is required to assure optimum appearance and accurate classification of the image. Other aspects of acceptance testing include the assessment of image artifacts, image non-uniformities, system linearity, noise in the absence of image signal (i.e., dark noise), visual high- and low-contrast imaging performance, accuracy of exposure indicator, and throughput (15) (Table 1).

System calibration: Digital radiographic systems are susceptible to systematic image non-uniformities due to inherent non-linearities of sensors. Such artifacts are generally corrected by a calibration procedure. Depending on the system specification, for some systems, this calibration needs to be performed on a daily basis at the outset of the clinical use for the day, while for others it needs to be done every few months.

Preventative maintenance: Any imaging device used clinically needs to undergo routine preventative maintenance to reduce the likelihood of down-time and performance degradation over time. This function is usually performance by service engineers contracted by the manufacturer’s service providers.

Periodic assessments: The performance of a digital radiographic system is prone to degradation over time. As such, it is important to track the system performance over time to ensure patient dose is within acceptable limits, and image quality is maintained. This objective is best achieved by initiating a periodic assessment program through which the basic performance aspects of the system are regularly tested and benchmarked against the results of acceptance testing and prior system QC tests. Testing should include resolution, noise, and artifact aspects of the system performance as listed above. The QC program needs to include established quantitative acceptance criteria to determine whether a given result meets expectations. Failures should prompt corrective actions before the device is put back into service.

While periodic assessment is an important aspect of a quality digital imaging operation, it is equally important that it is executed in an efficient manner. In that regard, it is important for the program to focus less on aspects of the performance that are proven to be stable over time. Furthermore, the results should be placed in a database that can be readily queried and conveniently interrogated by the responsible parties for assessing performance trends over time.

Requirements for Classification of Pneumoconiosis

Digital radiography provides an unprecedented opportunity to provide a standardized classification of pneumoconiosis. It can do so through its quantitative nature and its tractable performance characteristics. However, this is only possible if those attributes are properly utilized. As such, a robust classification of pneumoconiosis would have the following prerequisites:

1. The performance of the digital imaging system should be maintained and monitored through robust preventative maintenance and quality control programs.
2. A standardized image acquisition protocol is necessary. The protocol should specify the kVp and filtration settings, and exposure levels to achieve certain target SNR levels within the image. The latter can be prescribed based on the measured eDQE performance of the system.
3. An index of the exposure level used to form the image (i.e., an exposure indicator) should be provided with values reported in a consistent fashion across systems from different manufacturers.
4. The image data from the system needs to be available in a raw, “For Processing” format. In this manner, the data can be processed to permit consistent visualization, or analyzed for automated quantification of pneumoconiosis.
5. The image data needs to be processed in a consistent, pre-defined manner, so that image appearance can be consistent across cases, hardware, software, and systems.
6. The image data needs to be displayed in a consistent fashion using the expected performance requirements for electronic medical displays (16)
7. Both raw and processed image data should be archived electronically for further assessment or analysis.

Provided that the minimum requirements outlined above are met, digital chest radiographs can be used for visual classification of pneumoconiosis, as images will provide a consistent appearance of the disease.

The digital image data can further be used in a computer-assisted classification algorithm to automatically or semi-automatically classify the extent of the disease. The analysis can be based on image features of segmented lesions such as contrast, size, and texture. Such an algorithm will need to operate on raw image data and will use the inherent image quality characteristics of the imaging system (MTF and noise) in order to “normalize” for those attributes.

Conclusions and Recommendations

Quality control is an essential component of a digital radiography operation, especially when the images are to be used for classification and quantification purposes. Key components of a quality control program include acceptance testing, system calibration, preventative maintenance, and periodic assessments. A robust QC program along with standardized acquisition and processing protocols would enable visual as well as automated classification of pneumoconiosis from digital chest radiographs.

To ensure robustness and integrity of digital image data and to enable a reliable classification scheme, the following are strongly recommended:

1. QC program: All NIOSH affiliated facilities should enact and maintain rigorous PM and QC programs as outlined above.
2. Protocols: All NIOSH affiliated facilities should follow predefined acquisition and processing protocols.
3. Web server: NIOSH should consider a central web server for affiliated facilities.
4. Communication: Using NIOSH’s server, all NIOSH affiliated facilities should register their imaging devices including uploading their inherent performance metrics. All raw, “for-processing” image data will also be uploaded. The data will be consistently processed and analyzed for visual or automated classification.
5. Accreditation: NIOSH should consider a process by which it could accredit affiliated facilities to ensure adherence to its minimum performance and operational requirements.

References

1. Rossmann K. Point spread-function, line spread-function, and modulation transfer function. Tools for the study of imaging systems. Radiology 1969; 93:257-272.
2. Samei E, Flynn MJ, Reimann DA. A method for measuring the presampled MTF of digital radiographic systems using an edge test device. Med Phys 1998; 25:102-113.
3. Dobbins JT, 3rd. Image quality metrics for digital systems. In: Beutel J, Kundel HL, Van Metter RL, eds. Handbook of medical imaging. 1 ed. Bellingham, WA: SPIE Press, 2000; 161-222.
4. Samei E, Ranger NT, Dobbins III JT, Chen Y. Inter-comparison of methods for image quality characterization: 1. Modulation transfer function. Medical Physics, 2006; 33:1454-1465.
5. Rossmann K. Spatial fluctuations of x-ray quanta and the recording of radiographic mottle. Radiology 1963; 90:863-869.
6. Barrett HH, Swindell W. Radiological imaging : the theory of image formation, detection, and processing. New York: Academic Press, 1981.
7. Dobbins JT, 3rd, Ergun DL, Rutz L, Hinshaw DA, Blume H, Clark DC. DQE(f) of four generations of computed radiography acquisition devices. Med Phys 1995; 22:1581-1593.
8. Flynn MJ, Samei E. Experimental comparison of noise and resolution for 2k and 4k storage phosphor radiography systems. Med Phys 1999; 26:1612-1623.
9. Dobbins III JT, Samei E, Ranger NT, Chen Y. Inter-comparison of methods for image quality characterization: 2. Noise power spectrum. Medical Physics, 2006; 33:1466-1475.
10. Rose A. The sensitivity performance of the human eye on the absolute scale. J Opt Soc Am 1948; 38:196-208.
11. Ranger NT, Dobbins III JT, Samei E, Ravin CE. Assessment of detective quantum efficiency: Inter-comparison of the recent international standard with prior methods. Radiology, 2007; 243:785-795.
12. Samei E, Saunders RS, Lo JY, Dobbins III JT, Jesneck JL, Floyd Jr CE, Ravin CE. Fundamental imaging characteristics of a slot-scan digital chest radiographic system. Medical Physics, 2004; 31:2687-2698.
13. Samei E, Lo JY, Yoshizumi TT, Dobbins III JT, Jesneck JL, Floyd Jr CE, McAdams HP, Ravin CE. Comparative scatter and dose performance of slot-scan and full-field digital chest radiographic systems. Radiology, 2005; 235:940-949.
14. Boyce S, Samei E. Imaging properties of digital magnification radiography. Medical Physics, 2006; 33: 984-996.
15. Samei E, Seibert JA, Willis CE, Flynn MJ, Mah E, Junck KL. Performance evaluation of computed radiography systems. Medical Physics; 2001; 28: 361-371.
16. Samei E, Badano A, Chakraborty D, Compton K, Cornelius C, Corrigan K, Flynn MJ, Hemminger B, Hangiandreou N, Johnson J, Moxley-Stevens DM, Pavlicek W, Roehrig H, Rutz L, Shepard J, Uzenoff RA, Wang J, Willis CE. Assessment of display performance for medical imaging systems: Executive summary of AAPM TG18 report. Medical Physics; 2005; 32:1205-1225.