Developed by Dr. James Tilton, a computer engineer with NASA’s Goddard Space Flight Center’s Computational and Information Sciences and Technology Office (Greenbelt, MD), Hierarchical Segmentation (HSEG) software allows for advanced image analysis. Dr. Tilton first began developing hierarchical segmentation technologies in 1983 after becoming familiar with earth sciences and remote sensing in graduate school. During his initial years at NASA, he began to think about image segmentation and analyzing the data beyond the typical “per-pixel” approach, because each pixel did not necessarily provide enough information about where it fit into the overall “scene.” Dr. Tilton theorized that a better understanding could be achieved by considering the context of the image and looking at the objects in the image rather than the individual pixels. This theory ultimately led to the initial version of the core HSEG software algorithm.

The software organizes an image’s pixels into regions based on their spectral similarity, so rather than focusing on individual pixels, HSEG focuses on image regions — and how they change from a coarse to fine perspective. These regions (segmentations) are at several levels of detail (hierarchies) in which the coarser segmentations can be produced from the finer-resolution segmentations by selective merging of regions. In addition, the segmentation hierarchies provide analysis clues through the behavior of the image region characteristics over several levels of segmentation detail. Thus, by enabling region-based analysis, the segmentation hierarchies organize image data in a manner that makes an image’s information content more accessible.

Within NASA, HSEG was used to identify and extract magnetospheric radioecho and natural plasma-wave signals captured by the Radio Plasma Imager aboard the Imager for Magnetopause-to-Aurora Global Exploration (IMAGE) spacecraft.

To advance the usability of HSEG, Dr. Tilton also developed Recursive Hierarchical Segmenting (RHSEG) Pre- Processing Software, which significantly improves the extraction of patterns from complex data sets. Optimized for speed and accuracy, the patent-pending algorithm that fuels the software provides the user with precise control for selecting the desired level of detail from the hierarchy of results.

Rather than working on the usual pixel-by-pixel basis, RHSEG automatically organizes pixels into regions hierarchically, based on their spectral similarity. Looking at these regions, as opposed to individual pixels, allows the user to isolate specific features that are impossible to distinguish by other methods. Thus, RHSEG provides a more reliable and accurate understanding of the image.
Dr. Tilton developed the RHSEG software system for use in Earth remote sensing applications, providing a new approach to image analysis. The system offers selectable levels of detail that increase accuracy for two-dimensional (and potentially three-dimensional) images. Rather than working on the usual pixel-by-pixel basis, RHSEG automatically organizes pixels into regions hierarchically, based on their spectral similarity. Looking at these regions as opposed to individual pixels allows the user to isolate specific features that are impossible to distinguish by other methods. Images can be two-dimensional or three-dimensional single-band, multispectral, or hyperspectral data at resolutions up to 16,000 x 16,000 pixels.

From Earth Science to Medical Imaging

In 2002, NASA’s Goddard Space Flight Center issued a nonexclusive license to Bartron Medical Imaging (BMI) LLC of New Haven, CT, which quickly realized that RHSEG was the solution they needed to differentiate difficult- to-see details from a complex matrix background. Bartron has since developed a product for use in medical imaging called MED-SEG™, and has reported that RHSEG has enabled them to successfully analyze and extract meaningful and significant features from grayscale data previously indistinguishable by the human eye. The clinicians can also isolate one particular area of interest in an image to compare it with many other reference images in databases at other healthcare facilities.

The RHSEG software presents its results in a straightforward format consisting of a hierarchical set of image segmentations in either two or three spatial dimensions. This hierarchical presentation of results allows the user to choose the segmentation(s) of interest and to perform additional analyses.
When powered by parallel processing computer clusters, the system provides the clinician with rapid, sensitive, specific, and precise analysis since each image pixel is treated separately. The result is an accurate graphical representation of the imagery data with fine resolution of detail and minimal distortion. Because the only information being analyzed originated in the actual image being processed, there is less distortion and better resolution, resulting in a more accurate and reliable analysis of the image. This furnishes the clinician with more fine-grain information, leading to a quicker and more definitive diagnosis or other evaluation.

The MED-SEG™ System is a powerful multiprocessor workstation that allows the wealth of information generated by processing through the MED-SEG system. Once images are processed by the MED-SEG System, the data is transmitted to a secure, capable target – the MED-SEG workstation. It is composed of a front-end terminal for segmentation feature extraction, pattern recognition, and classification of medical images. It incorporates the firmware and software essential to view the processed images, reliable and secure communication and storage, the requisite computer power for expected practitioner image manipulations, and display options for the MED-SEG image data. Images are transmitted to the MED-SEG System site via cable, DSL, satellite, or dial-up (for small images) in a format acceptable to the segmentation software. In most cases, it will be in a DICOM, TIFF, JPEG, or compressed format.

The device is intended to analyze medical imagery from computed tomography (CT or CAT) scans, positron emission tomography (PET) scans, magnetic resonance imaging (MRI), ultrasound, digitized X-rays, digitized mammography, dental X-rays, soft tissue analyses, and moving object analyses. The technology is also equipped to evaluate soft-tissue slides such as Pap smears for the diagnoses and management of diseases. The advanced image segmentations produced by the RHSEG software allow the MED-SEG system to bring out details in these tests not previously seen with the naked eye. This allows for quick and accurate diagnosis of diseases. Additionally, unlike some other image-analysis devices, MED-SEG does not manipulate the image, so what the physician sees is truly what is there, providing truer images than many other imaging techniques.

A comparison of mammogram images: (left) the original mammogram, and (right) the segmented image created with the MED-SEG system. (Bartron Medical Imaging)
By extending the software’s capabilities to three dimensions, BMI’s device may be able to produce a pixel-level view of all sides of a tumor or lesion. While current technology can produce 3D imagery, the RHSEG software will be able to segment an image in ways that more clearly define problem areas. For example, the 3D version of MED-SEG may be able to identify very early buildup of soft plaque within the arteries or identify density levels of microcalcification in mammograms, allowing physicians to diagnose malignant breast tumors well before they would normally be seen. In brain images, the physicians using MED-SEG will also be able to make earlier diagnoses of tumors or arteriovenous malformations.

Although there are various other potential applications for the MED-SEG system, Bartron has chosen the initial focus of its marketing efforts to be directed toward mammographic soft tissue imaging. It usually takes at least three to five years from the first spot of cancer until its detection, even with regular mammograms. In conjunction with digital mammographic modalities, the MED-SEG system is anticipated to offer a non-invasive, more precise method through analysis of shifts of pixel accumulation and concentration to specific breast areas. Image processing correlated with known breast disease pixel patterns such as calcifications, masses, or distortions may be identified long before they become visible on a mammogram.

BMI made NASA history by being the first company to partner with the Space Agency through a Cooperative Research and Development Agreement (CRADA). This agreement, for development of the 3D version of RHSEG, grants (in advance) a partially exclusive license for the resulting technology patents within BMI’s fields of use (the diagnosis and treatment of breast cancer, cervical cancer, brain cancer, heart disease, osteoporosis, and periodontal diseases).

In order to manage the image data, BMI also licensed two pattern-matching software programs from NASA’s Jet Propulsion Laboratory that were used in image analysis, and three data-mining and edge-detection programs from Kennedy Space Center. The Kennedy imaging technologies are currently in use at NASA to identify and track foreign object debris during space shuttle liftoff, and in the Cable and Line Inspection Mechanism used to test the shuttle’s emergency escape system. With U.S. Food and Drug Administration clearance, BMI will sell its MEDSEG imaging system with the 2D version of the RHSEG software to radiology centers, hospitals, and other clinical organizations. Regarding current use of MEDSEG, researchers at the University of Connecticut Health Center are working with Bartron on a mammographic study where MED-SEG is being used to analyze 20 “normal” mammogram images and 20 biopsied cancer mammogram images. Another study underway at Yale University is using MED-SEG to analyze ovarian cancer images.

For more information on the MED-SEG system, visit . Find out more about other NASA medical spinoffs at