Real-time medical imaging is enabling new levels of care by providing greater insight and decision support. As the use of video in healthcare increases, system designers are under pressure to ensure that systems are intuitive and easy to use while delivering performance advantages within tightening budgets. In addition, systems must be scalable to accommodate advances in machine learning and artificial intelligence (AI).
A key early decision in the design of x-ray flat panel detectors (FPDs) for medical imaging modalities is the sensor interface used to format imaging data and send it to processors and displays. Although the interface is a small part of the overall system, it has a large impact on the usability, costs, and scalability of the final product.
Machine Vision Standards and Medical Imaging
A factory floor and a sterile operating room may seem worlds apart, but interface standards from the industrial machine vision market are playing a key role in the advancement of FPD design. Medical imaging applications rely on data transmission from an imaging source to processing equipment. Traditionally, system designers developed proprietary point-to-point interface solutions, or leveraged interface standards from the telecom and consumer markets (such as LVDS and HDMI/DVI).
While these approaches met performance requirements, they added development and component costs and posed integration challenges in multi-vendor systems. As systems become more complex, with multiple image sources and display panels, higher bandwidth applications, and real-time requirements, these approaches become even more costly and difficult to scale.
The machine vision industry faced similar challenges, leading to the development of the GigE Vision standard. GigE Vision provides a framework for transmitting full resolution uncompressed video and data with low, consistent latency over Ethernet. Ratified in 2006, it is now the most widely deployed interface standard for industrial applications and is gaining a strong foothold in the medical and defense markets.
With GigE Vision, data is sent directly to existing ports on computers used for analysis, display, and recording. This eliminates the need for costly PCIe frame grabbers to capture images at endpoints, while enabling the use of a wider range of lower-cost computing platforms. The same connection is used to transmit control data between the computing platform and the imaging device, as well as configuration information for imaging systems used for different procedures. Per-frame metadata, such as the date and time of acquisition, sensor settings, and imaging equipment used, is transmitted with the images over the GigE link for easy integration with DICOM-compliant software and hardware.
GigE Vision addresses delivery requirements for vital patient data by including mechanisms to resend undelivered data to receivers. This mechanism can be turned off if resending data is not required for the application. Typically, in a properly architected network where the constant bandwidth of uncompressed data has been taken into consideration, packets will rarely if ever be dropped. This mechanism, together with other areas of the standard, allows performance-oriented implementations of the GigE Vision standard to guarantee sensor data transport with low and predictable latency even during a resend.
GigE Vision-compliant interfaces help lower the cost of new imaging system design and retrofit upgrades (see Figure 1). By building a sensor network on an Ethernet architecture, multiple sensors or channels of video can be aggregated into a single network link. Compared with other interface standards, Ethernet’s inherent support for meshed network configurations means it can easily accommodate different data rates and the addition of new processing nodes, displays, and sensors. Because each GigE Vision interface uses its own IP address, there’s no limit to how many endpoints can operate on the same network.
Data is transmitted directly to Ethernet ports that are standard on computing platforms, eliminating the need for PCIe frame grabbers and enabling the use of laptops, single-board computers, and tablets. Thanks to Ethernet’s universal deployment across consumer and telecom markets, networking equipment is readily available, affordable, and easy to maintain and install.
Off-the-shelf embedded solutions, including small footprint hardware and software solutions, make it straightforward to design GigE Vision-compliant static and dynamic FPDs. Software solutions are also available to convert an existing sensor device into a “virtual GigE Vision” transmitter. Interfaces designed for FPD applications can help improve system reliability by storing a temporary history of images, avoiding the need to retake an x-ray in the case of a power interruption or user error.
Video over GigE Vision is well suited for medical imaging applications because it addresses today’s bandwidth and cost requirements with the scalability required for future AI capabilities. Plus, with a standards-based approach developers can port designs to different panels, operating systems, and processors with minimal redesign effort.
One of the key performance advantages of the GigE Vision-based distributed network architecture is the ability to aggregate previously isolated image sources and patient data to display integrated information on a single dashboard (see Figure 2). In the operating room, for example, the single screen dashboard displays recorded and realtime patient data from different imaging devices and systems. The surgeon or operating team members can easily switch between imaging sources, such as white light and fluoroscopic cameras or pre-op and real-time images, without reconfiguring hardware.
Advances in x-ray imaging are also helping to reduce radiation doses for patients. Innovative fluoroscopy systems minimize a patient’s exposure by using multiple moving x-ray sources to irradiate tissue from numerous incremental angles in just seconds. Traditional interfaces would be uneconomical and too cumbersome for this application compared with the single, flexible cable connectivity of Ethernet (see Figure 3).
Medical Imaging and AI
AI is one of the most hyped technology trends of recent years, and maybe one of the most misunderstood. At the most basic level, AI uses machine learning techniques to perform tasks that previously required either significant algorithm programming or human intelligence. Deep learning takes that one step further by using artificial neural networks to analyze data and independently create inspection or processing algorithms.
Radiology is a natural fit for AI, thanks to a number of factors. The discipline has already embraced a wide range of digital modalities, meaning there is a large volume of collected data that can be used for machine learning training. There are also a number of time-consuming, repetitive, data-intense tasks where a radiologist could supervise an AI process and rely on the technology as a decision-support tool. This would leave more time for a radiologist to focus their expertise on further investigation of abnormal findings as well as patient and physician consultation.
From a system perspective, AI can help reduce the number of retakes required due to image artifacts and equipment errors. Faulty pixel patterns form over time with an FPD due to repeated radiation exposure. AI can identify pixel patterns and provide technicians with a preventative maintenance warning. An AI-based “inspector” can also be used to immediately check the quality of an x-ray image and validate if it is suitable for AI-assisted or manual analysis to prevent patient callbacks.
Image enhancement and reconstruction are also key areas where AI can help reduce patient exposure times and speed processes. Machine-learning-based image correction techniques for blurry x-ray images or missing data can reduce the number of rescans required. Texture analysis can be used to detect cancerous lesions, and AI-based shape modeling can use 2D images to recreate 3D segments of the body. For example, a tumor could be modelled by AI techniques to help guide surgical decisions.
As a screening tool, AI can use a benchmark-based algorithm for high-volume x-ray screening to highlight areas of concern for further examination by a radiologist. For unique situations where a radiologist may only see a handful of cases, AI could potentially review thousands of similar cases to help support a diagnoses. The screening abilities of AI will become more important as the amount of patient data increases. For example, AI can extract relevant information from health data collected by home care and personal monitoring systems and provide it to a healthcare professional.
For system designers, the evolution toward AI raises some significant concerns. Image quality and metadata become increasingly important to provide a standard dataset for algorithm development and subsequent performance evaluation. Adding AI capabilities, particularly in already installed infrastructure and alongside proven computer vision processes, poses cost and integration challenges.
GigE Vision provides a path forward for system designers to help address existing and future AI capabilities. Standardizing all FPD sensor streams to GigE Vision ensures a common dataset across all devices that can be used for algorithm training. With standards-based data, images are seamlessly shared with processors, other system elements, and local and cloud-based algorithms
The sensor interface also provides an entry point to integrate AI capabilities into systems. Edge processing can be integrated into the embedded interface to perform image correction, detection, and object identification before data is sent on to the processor. This helps reduce overall system bandwidth and processor requirements to both increase performance and lower costs. Enhanced image processing AI capabilities can be easily uploaded to the sensor as algorithms are improved.
Image processing and smart AI platforms that bridge existing infrastructure with new machine learning techniques are also available. The platform integrates multiple GigE Vision sensor streams, uses machine learning AI technique for image correction and enhancement, and then transmits the processed data to computing platforms for further analysis. As AI capabilities expand, multiple platform nodes can be added to the system to support advanced performance and distribute processing to meet bandwidth and latency requirements.
The Right Design Choice
Starting with the first clinical use of x-rays over 100 years ago, medical imaging has played an ever-increasing role in healthcare. Today, almost all aspects of care — from initial examination, to surgery, and nursing — rely on real-time video to identify issues, make accurate diagnoses, and provide treatments.
While the sensor interface is a small part of an overall medical imaging system, choosing the right interface delivers significant design advantages for manufacturers, cost-savings for healthcare providers, and performance benefits to help improve patient comfort and care. Designing or upgrading medical imaging systems based on GigE Vision interfaces allows manufacturers to shorten time-to-market, reduce risk, and lower system cost and complexity, while delivering interoperability and performance benefits as medical imaging begins to adopt AI-based machine learning.
This article was written by Jonathan Hou, CTO, Pleora Technologies, Kanata, ON, Canada. For more information, visit here .