Kalman filter theory: a way to turn context rules into recurrent architectures?

The detection of spatial motion Gestalts occurs through a spatial recurrent filter that checks the consistency between the spatial structural properties of the input flow field pattern and a structural rule expressed by the process equation of the Kalman filter.

October 20, 2002 — Each measure of an observable property of the visual stimulus is affected by an uncertainty (not only due to the additive neural noise, but also to the fact that the visual properties are themselves random processes) that can be removed, or, better, reduced by making use of additional information (context information, a priori knowledge, etc.). Early cognitive vision can be related to the segment of perceptual vision that takes care of reducing the uncertainty on visual measures by capturing coherent properties over large (overlapping) retinal locations (Gestalts), a step that precedes the true understanding of the scene. Following an AI approach, the use of contextual information occurs through the application of specific verbal/knowledge-based rules to establish consistency relationships among the extracted visual features. By contrast, in visual cortex, contextual modulation of the sensorial input occurs through dense intra- and inter-area feedback interconnections that integrate context information by modulating cells’ responses, adapting their tuning and refining their selectivity. We challenged the goal of mimicking cortical computational paradigms to develop parallel distributed processing systems to implement adaptive visual filters, which are fully data-driven and avoid explicit use of AI rules. This would allow to define context-sensitive filters based on structural computation rather than on mere calculus. To this end, we defined a general framework to specify context sensitive filters (CSFs) through Kalman filter theory. Due to its recurrent formalization, Kalman filter appears particularly promising to design CSFs based on recurrent cortical-like interconnection architectures. To validate the approach, we defined a set of CSFs based on deterministic (i.e., geometric) spatial motion Gestalts. In particular, the geometric properties of the optic flow field have been described through a specific set of elementary gradient-type patterns, as cardinal components of a linear deformation space. By checking the presence of such Gestalts in optic flow fields, we make the interpretation of visual motion more confident. Given motion information represented by an optic flow field, we recognize if a group of velocity vectors belong to a specific pattern, on the basis of their relationships in a spatial neighborhood. Casting the problem as a Kalman filter, the detection occurs through a spatial recurrent filter that checks the consistency between the spatial structural properties of the input flow field pattern and a structural rule expressed by the process equation of the Kalman filter. A Gestalt emerges from a noisy flow as a solution of an iterative process of spatially interacting nodes that correlates the properties of the visual context with that of a structural model of the Gestalt. The resulting CSF behaves as a template model. Yet, its specificity lies in the fact that the template character is not built by highly specific feed-forward connections, but emerges by stereotyped recurrent interactions. We expect the approach can be extended to consider adaptive cross-modal templates (e.g., motion and stereo). By proper specification of the transition matrix in the process equation of the Kalman filter, it can, indeed, potentially model any type of multimodal spatio-temporal relationships (i.e., multimodal spatio-temporal context). To assess the performances of the CSFs, we applied them to real world optic flows, extracted from driving sequences, which show road scenes taken by a rear-view mirror of a moving car under different overtaking situations. The analysis was performed on overlapping local regions of the optic flow with twenty-four elementary gradient-type Gestalts (i.e. elementary flow components). In this way, we can compute a dense distribution of the local Gestalt probabilities for the overall optic flow. Thence, we obtain the most reliable local velocity patterns, e.g., the patterns of local Gestalts that characterize the sequence (see figure).

Silvio P. Sabatini
Fabio Solari
Giacomo M. Bisio
Dept. of Biophysical and Electronic Engineering
University of Genoa