Empirical Evaluation of Diagnostic Algorithm Performance Using a Generic Framework

Shared by SCOTT POLL, updated on Dec 18, 2013


Author(s) :
A. Feldman, T. Kurtoglu, S. Narasimhan, S. Poll, D. Garcia, J. De Kleer, L. Kuhn, A. Van Gemund

A variety of rule-based, model-based and datadriven
techniques have been proposed for detection
and isolation of faults in physical systems.
However, there have been few efforts to comparatively
analyze the performance of these approaches
on the same system under identical conditions.
One reason for this was the lack of a standard
framework to perform this comparison. In this paper
we introduce a framework, called DXF, that
provides a common language to represent the system
description, sensor data and the fault diagnosis
results; a run-time architecture to execute
the diagnosis algorithms under identical conditions
and collect the diagnosis results; and an evaluation
component that can compute performance
metrics from the diagnosis results to compare the
algorithms. We have used DXF to perform an empirical
evaluation of 13 diagnostic algorithms on a
hardware testbed (ADAPT) at NASA Ames Research
Center and on a set of synthetic circuits
typically used as benchmarks in the model-based
diagnosis community. Based on these empirical
data we analyze the performance of each algorithm
and suggest directions for future development.

show more info
Publication Name
Publication Location
Year Published


977.6 KB 39 downloads


Add New Comment