Researchers have developed a technology that reveals the criteria AI systems use when making decisions. The innovative Spectral Relevance Analysis (SpRAy) method based on Layer-wise Relevance Propagation technology provides a first peek inside the “black box”.

Individual predictions exhibit very different heatmaps. (Credit: Nature Communications)

The technology renders the AI forecasts explainable and in doing so reveals unreliable problem solution strategies. SpRAy identifies and quantifies a broad spectrum of learned decision-making behaviors and thus identifies undesirable decisions even in enormous datasets.

The LRP technology decodes the functionality of neural networks and finds out which characteristic features are used, for example. to identify a horse as a horse and not as a donkey or a cow. It identifies the information flowing through the system at each node of the network. This makes it possible to investigate even very deep neural networks.

For more information, visit here .