Perceptual Intelligence

The Perceptual Intelligence Mission brings together computer scientists, cognitive scientists, and neuroscientists with a shared goal of producing the first machine executable models of human visual intelligence that work computationally, cognitively, and neurally.

Concretely, the interdisciplinary team working on this Mission aims to produce new machine executable models that take sensory data as input — any possible movie or sequence of images and achieve all three of the following: First, these computer models match or exceed the data efficiency, generalizability, and robustness of human vision, unlike current industry “AI” approaches that require large training datasets and struggle with difficult viewing conditions (especially ones that induce perceptual uncertainty) and often fail to transfer from the factory to the field; Second, these models are theoretically well-grounded in that they are provably constructed by design to implement sound approximate probabilistic inference in generative models that accord with core principles of optics and mechanics, with controllable tradeoffs between perceptual latency and error rate, for fixed-size circuit implementations, unlike leading neural network architectures for vision such as transformers and CNNs; And third, these models have internal components that simultaneously align with internal brain responses measured across the multiple brain areas in primates (including humans) that are thought to support scene understanding and thus are the leading scientific models of the brain mechanisms of visual perception.

If this Mission is successful, it will:

• Produce theoretically grounded, open source software and tools to catalyze new AI approaches that scale far better in data and compute than current industry approaches. This will allow transformative AI applications far beyond human visual capabilities that this Mission aims to explain.
• Unlock the ability to use the brain-aligned models of human visual processing and perception for biomedical applications. For example, the development of non-invasive methods for targeted, precision deep brain modulation.
• Enable other scientific teams to use the same approach to explain human auditory and tactile perception.
• Form the foundation of other machine executable models of other domains of intelligence that draw on the contents of visual perception, such as planning and navigation.