A well-known legend has it that one of the greatest scientists and inventors of antiquity, Archimedes of Syracuse, stepped into a bath only to eject and propel himself naked throughout the city, yelling “Eureka!”, Greek for “I have found (it),” thus celebrating his discovery of how to measure the volume of irregular objects. Whether this indeed happened or not remains an open question, but a few important lessons can certainly be learned from this story. The first lesson is that good ideas could occur to us while we are taking a bath or a shower. The second lesson is that scientific or technological problems often seem to be difficult before a brilliantly simple solution is found. Does this second lesson hold true today? This is the question we will try to answer using an example of our own journey into discovery.
While we do not have to solve Archimedes’ problem again, we still have problems to solve and discoveries to make. This is a brief story of our contribution— MelaFind®—the world’s first automated noninvasive instrument to help detect the deadliest form of skin cancer (cutaneous melanoma) at its most curable stages. (See Figure 1)
Challenge and Opportunity
MelaFind was developed by a group of research scientists who were working on defense research projects. After a meeting with melanoma expert Dr. Alfred Kopf, the group was inspired to start working on detection of cutaneous melanoma. The goal was to develop a device that would save lives and make a positive impact on humanity. Although it took significantly longer to accomplish this goal than we envisioned, it is rather remarkable that even today there are no other solutions to this problem that were validated in rigorous blinded prospective clinical trials and approved as an automated medical device with a proven diagnostic profile.
The fundamental difficulty of this particular problem is that unlike many diseases that humans can objectively diagnose, or unlike many military targets that our group was working with, melanomas in their most curable stages often lack any known and specific physical or chemical markers. The empirical characteristics, such as clinical A-B-C-D for Asymmetry, Border Irregularity, Color Variegation, and Diameter > 6mm, are also present in many benign skin lesions. An example of asymmetry is shown in Figure 2. Unfortunately, the prognosis for advanced melanoma is not good.
A New Approach
We decided to develop a machine that can learn and make discoveries based on a large number of histologically confirmed examples, both malignant and benign. While the field of machine intelligence is not new, I think our project represents one example in the next chapter of research; specifically, we, as an intelligent species, seem to be reaching the limit of making discoveries by virtue of the sheer “Eureka!” moment or manual analysis of the data, because most of the “simple” problems seem to have already been solved. By “simple,” of course, we do not mean that these problems did not appear to be difficult before they were solved, rather that their solutions turned out to be understandable, as was the brilliant discovery by the naked man from Syracuse.
In the future though, we will have to rely more and more on a mutually complementary partnership between human and machine intelligence to analyze the available wealth of information, to arrive at solutions to today’s tough problems. We believe that humans will remain designers of “machine inventors,” but the search for the discoveries themselves will be delegated to increasingly intelligent computers.
The most amazing consequence of this new approach, as our own experience with MelaFind has demonstrated, is that the discoveries made by machines, the very working recipes to the problems we seek answers to, may be so complex that they will not be completely understandable to humans, the creators of these “electronic Archimedeses” of the future. Just as we do not completely comprehend all the details about how our brain works, we will have to accept the humbling fact that even our own creations may produce answers beyond our understanding.
I have spent 14 years working on MelaFind and some of my more gifted colleagues have spent even more, and yet, in all honesty, I can’t explain exactly how it works. Some of its design follows the principles employed in machine (and often biological) vision. Specifically, the device has a chain of algorithms that, starting from the multispectral (from blue to near infrared) images, arrive at an evaluation of a lesion. The chain is as follows: image calibration, segmentation, automated image quality control, feature extraction, and classification.
We begin with image calibration. The purpose here is twofold: to remove or reduce the systematic components of the “noise” and to ensure that lesion information does not depend on the instrument used to acquire images.
Segmentation then automatically locates the lesion and determines its boundaries.
Next is automated image quality control. At this stage, any foreseeable imaging problems (like hair, air bubbles trapped in interface liquid, movement of the instrument, etc.) must be detected and, if their presence is found to be excessive or significantly interfering with the lesion information, the machine gives a recommendation on how to correct the problem.
Feature extraction then converts the segmented portion of the image (corresponding to a lesion) into a set of characterizing values (called features) that are meaningful for classification purposes. For example, the A-B-C-D rule is an example of four features currently employed by dermatologists to identify melanomas. Most of the features that MelaFind uses are far more complex than simply A-B-C-D.
Classification combines several extracted features in multidimensional space in order to make a decision on whether the lesion presents with a high or low degree of morphological disorganization. MelaFind operates in 75-dimensional space to arrive at this decision.
This final multi-dimensional classifier recipe, as well as the choice of 75 particular features out of a pool of more than 1,000 candidates, is a product of machine training and its composition makes little sense to us, the developers of the machine that trained MelaFind.
Ironically, the phrase that Archimedes was shouting after his famous submersion into the body of water, the very phrase that became one of the symbols of serendipity, also gave birth to the entire scientific discipline called heuristics—the set of techniques of problem solving based on prior confirmed experience, not the systematic analysis of the problem. MelaFind is one example of a heuristic approach to the problem.
Developing the Imager
While the developments like MelaFind could not occur without massive amounts of mathematical computations, it didn’t take just these “virtual” entities to solve the problem. We also had to design the hardware: a multispectral imager that would be compact, reliable, thermally stable, and capable of delivering an information-bearing signal to the “computer brain.” In other words, we had to develop an “eye” of the system.
The purpose of developing a multispectral imager operating in both visible and near-infrared portions of the electromagnetic spectrum was to enable the machine to see more and deeper than a human eye can see, but, at the same time, limit ourselves to an imaging modality that would employ safe, nonionizing electromagnetic radiation. We used the same standards and design principles that are employed in developing space-based imagers: exhaustive characterization of the instruments, thermal stability, rigorous calibration, and a comprehensive testing of each unit produced. In addition to that, we had the challenge of producing not just a one-of-a-kind instrument, but a whole population of instruments that work similarly to each other.
The resulting imager operates in 10 spectral bands, with centroid wavelength ranging from 430 nm (blue) to 950 nm (near infrared). Each band yields a grayscale image subject to subsequent morphological analysis by the machine. Because the depth of penetration of light into human skin depends on the wavelength, the multi-spectral imager is able to obtain three-dimensional information from beneath the skin, including depths not reachable by the human eye, up to 2.5 mm deep (because humans cannot see infrared light, which has the largest depth of penetration). Seven of the 10 bands cover the visible portion of the electromagnetic spectrum, while the three remaining bands provide information from the near infrared portion. The imager is equipped with its own multi-band illuminator, nine-element lens system capable of delivering desired optical performance across the wide region of required wavelengths, and a photo sensor to register the images.
All information from the scan session can be stored to track data for particular moles to be monitored in future visits.
In addition to numerous in-house blind tests, MELA Sciences conducted the one of the largest prospective studies in melanoma detection with 1,383 patients presenting with 1,831 pigmented skin lesions. The study was multi-center and blinded. Of the 1,831 lesions in the subjects enrolled, 1,632 were considered eligible for analysis including 127 melanomas and 48 high grade lesions. For all subgroups analyzed, the sensitivity of MelaFind (its ability to detect disease when it is present) to melanoma and high-grade dysplastic nevi was greater than 95%, and its specificity (ability to rule out disease when it is absent) was statistically significantly higher than that of study clinicians. In supporting reader studies, MelaFind’s sensitivity to melanoma was significantly higher than of dermatologists— 97% versus 72%.
In early November 2011, MELA Sciences received premarket approval from the FDA for use in the United States. Following completion of a successful conformity assessment procedure, the company was also granted CE Mark approval for sale of MelaFind in the European Union.
At times we were laughed at and were accused of “overdoing the airspace industry”. We were told that we were wasting our time. Some of the best optical manufacturers told us that our designs “can’t be produced,” but it all paid off when a constellation of very real, manufactured MelaFind instruments were successful in the first-of-a-kind, FDA-regulated pivotal trial.
We certainly do not think that MelaFind is perfect and we, as its creators, are constantly seeking to improve its “brain” and its “eyes”. But as of now, this imperfect child of humanmachine intelligence has already helped catch several melanomas that dermatologists admitted they would otherwise have missed. This is one of the most rewarding feelings a scientist can experience.
This article was written by Nyq Kabelev, the Vice President of Research and Development, MELA Sciences, Inc., Irvington, NY. For more information, Click Here