Reverse Engineering the Human (and Machine) Observer: Motivations and Tools

Forum Details

Date: 04/22/2022 12-1 PM Eastern time

Speakers: Craig K. Abbey, Ph.D., University of California, Santa Barbara

In radiology, and many other fields, human observers are used to extract diagnostic information from images. We know relatively little about how human observers accomplish this. This is perhaps not surprising given the complexity of the human visual system, and the internal decision systems it supports. Increasingly, machine learning techniques are being considered for many tasks in medical image analysis. While machine learning algorithms are explicitly defined through their architecture and connection weights, their size and complexity make them difficult to understand as well.

In this talk, Dr. Abbey will describe a psychophysical methodology that has proven useful for understanding how human observers extract basic features related to task performance in noisy images. The same methodology can be applied to machine observers as well. The approach is based on the idea of efficiency with respect to the Ideal Observer, which tells us how much of the available diagnostic information is being accessed by human observers, and the classification-image technique, which tells us how this information is accessed. In combination, these two components provide a form of reverse engineering for observers that are not directly observable (humans) or prohibitively complex (machines).