A Universal Theory of Brain Function
The brain minimizes variational free energy by building generative models that predict sensory data using learned priors and approximate inference.
Key takeaways
5 points- 1Brains minimize free energy by testing internal predictions against sensory input.
- 2Generative models use priors and a generator network to reconstruct expected sensations.
- 3Recognition models perform fast approximate inference by inverting the generative process.
- 4Strong priors can override raw sensory data, producing illusions such as the hollow mask.
- 5Both perception and learning reduce uncertainty by refining the same world model.
Summary
The video presents the Free Energy Principle as a unifying account of brain function in which neural circuits build and test internal models of the world.
Core Mechanism
- Brains minimize variational free energy by trading off accuracy of sensory predictions against complexity of the model (0:00-8:02).
- Latent variables in hierarchical generative networks represent causes at increasing levels of abstraction.
Generative Models and Priors
- A generative model combines priors (learned probabilities) with a generator network that reconstructs expected sensory data (8:02-16:05).
- A separate recognition model performs fast approximate inference by inverting the generative process.
Explaining Illusions
- Strong priors cause the brain to perceive a hollow mask as convex, choosing the lower-free-energy explanation over conflicting sensory data (16:06-19:22).
Deep read
Executive Summary
The video outlines the Free Energy Principle (FEP) as a unifying theory of brain function, arguing that the brain acts as a predictive machine that continuously builds and tests internal models of the world. By employing a generative model (how latent causes produce sensory data) and a recognition model (how to infer causes from data), the brain minimizes free energy—a balance between explaining sensory input accurately and keeping the model simple (parsimonious). Priors, learned from environment and experience, bias interpretation toward plausible causes, explaining everyday phenomena like optical illusions where strong priors override sensory input. The talk uses intuitive, non-mathematical explanations (e.g., tiger occlusion, blender sliders, convex/concave mask) to illustrate how perception emerges from iterative inference and learning across fast (perception) and slow (learning) timescales. A deeper mathematical treatment is promised in future videos, connecting FEP to modern machine learning and AI systems capable of perception, prediction, and model-building.
00:00:00 Introduction
Speaker: Artem
- The video introduces the Free Energy Principle (FEP) as a unifying framework for understanding brain function: the brain acts as a prediction machine, actively generating hypotheses about the world and testing them against sensory input.
- The core idea is that percepts are not raw input streams but outcomes of the brain’s predictive models; perception is a process of minimizing surprise (or prediction error) given priors about how the world typically is.
- A striking hook is the face mask illusion: a convex/concave mask that appears to protrude or retract due to the brain’s strong priors about faces and depth, illustrating how top-down predictions can override sensory input.
- The video promises a high-level, conceptual tour (not a mathematical treatment) of how free energy, world models, priors, and inference fit together to explain everyday phenomena like optical illusions.
> > "this video was brought to you by Squarespace take a look at this mask it looks like a convex face protruding outwards now let's rotate it at this point you know what you expect to see a concave mask protruding inwards however somehow you get a strong sense that something is off and The Mask sure looks like it got warped to be convex again even though part of you knows that it's not the case but why is your brain so stubbornly convinced that the mask is protruding outward" (00:00:00)
- The sponsor message (Squarespace) is embedded early to set up the example that the brain’s expectations shape perception, just as a site design can shape user experience.
- The video frames the Free Energy Principle as a bridge between neuroscience and machine learning, hinting at both descriptive (how brains work) and prescriptive (how to build AI that perceives) uses.
- The outline provides a roadmap for the talk: role of world models, the tradeoff between accuracy and complexity, generative and recognition models, priors, approximate inference, the balance of free energy, and an optical illusion explanation.
01:34 Role of world models
- The brain builds world models to explain sensory inputs by inferring hidden causes behind what we sense (e.g., determining whether a visual pattern corresponds to a tiger or something else).
- World models must account for occlusion, partial information, and ambiguity; the same sensory data can be explained by multiple causes, so priors (learned beliefs about what is likely) guide interpretation.
- The brain’s inference is not about direct data matching but about selecting explanations (latent causes) that minimize a quantity called free energy, a balance between fit to data and model plausibility given priors.
- The notion of “latent causes” (hidden representations) is key: the brain uses neurons that do not directly interface with the world to encode higher-level features and abstract causes (e.g., tiger vs. occlusion, edges, textures).
> > "the brain evolved not just as reaction machines but as sophisticated model builders that try to explain sensory inputs by inferring their hidden causes" (00:01:34)
- The idea of a hierarchy of latent causes: abstract causes (e.g., Tiger) feed into intermediate features (e.g., stripes, fur texture) which feed into basic perceptual components (edges, color).
- Because latent variables have no ground truth, the brain tests which latent configurations best explain observed data by their consequences on actual sensory patterns, not by direct truth-testing of the latent state.
03:56 Free Energy as tradeoff between accuracy and complexity
- Free energy is a quantity the brain minimizes; it encodes a tradeoff between how accurately the model explains data and how complex (overfit) the model is.
- A simple analogy: imagine two explanations for a visual pattern; one explains the data but contradicts priors (high free energy), while another fits both data and priors (low free energy).
- The brain tends to prefer explanations that balance data fit with prior expectations to achieve robust, efficient representations.
- The concept is framed as a principle that accounts for both perception and learning: the brain continuously updates its internal model to better predict future observations while avoiding unnecessary complexity.
> > "the main purpose of the brain... is to explain sensory inputs by inferring their hidden causes... the brain is constantly trying to find the perfect balance between those two forces" (00:03:56)
- This section foreshadows how the Free Energy Principle underpins both immediate perception and longer-term learning, via ongoing refinement of the latent structure of causes and the generative mapping to sensory data.
05:20 Sponsor: Squarespace
- Sponsor message describing Squarespace as an all-in-one platform to build websites with design intelligence, generative AI tools, and templates to jump-start site creation.
- The ad ties back to the theme of prediction and models: Squarespace helps you quickly generate a professional site (a model of your online presence) and customize it with drag-and-drop interfaces.
- This segment is largely an interlude and example of how prediction tools (AI-assisted design) can streamline creating a representational model of your work, echoing the brain’s use of priors and generative processes.
06:35 Generative Model
- A generative model describes how latent causes generate observable data. Latent causes are the hidden factors the brain infers to explain sensory input.
- In the brain, there are two kinds of networks: latent (hidden) neurons that encode meaning (abstract causes like Tig or occlusion) and sensory neurons that encode observable data (edges, colors, textures).
- Latent causes are not directly observable; the brain can choose among many latent configurations because there is no ground truth for latent activity.
- The generative model consists of the connections (weights) between latent and sensory neurons plus neural circuits that reconstruct or uncompress latent representations into sensory predictions.
- The blender analogy: a simple scene can be controlled by a few sliders (latent variables) that generate a high-dimensional image; the brain’s latent slider combinations capture the regularities of the real world (e.g., light usually above objects, objects rest on surfaces), so some combinations are far more probable than others.
- Priors emerge from experience: the brain learns which latent configurations are common in the environment and assigns probabilities to them. Priors encode expectations about what kinds of scenes are likely.
> > "some slider combinations occur much more frequently than others in the real world... you will develop an intuitive sense of which parameter combinations are more likely" (00:06:35)
- The prior distribution and the generative model together form the brain’s world model: priors say what causes are likely, the generator maps causes to predicted sensory data.
- The difficulty of inversion: given a sensory observation, inferring the right latent causes would naively require examining a combinatorial explosion of possible causes (millions of combinations), which is computationally prohibitive.
- This motivates the need for approximate inference mechanisms that can efficiently infer plausible causes without exhaustive search.
10:03 Priors
- Priors are the brain’s beliefs about how likely various latent causes are before considering new data.
- They reflect statistical regularities learned from the environment (e.g., lights tend to be above objects; objects rest on surfaces; faces are typically convex).
- Priors bias interpretation of sensory input toward the most plausible explanations given the context.
- The context variable matters: priors differ across environments (city park vs safari). A glimpse of orange and stripes is interpreted differently depending on context, due to different priors.
- Priors are crucial for resolving ambiguity in perception and for guiding efficient learning of the generative model.
12:18 Approximate Inference via Recognition Model
- The inversion problem (inferring latent causes from data) is computationally hard; the brain uses a two-network solution: a recognition model and a generative model.
- Recognition model: maps sensory observations to a distribution over latent causes; effectively a fast, rough guess of what latent state would explain the data.
- Generative model: takes a latent state and predicts what sensory data would look like; used to test whether the guessed causes can recreate the observed input.
- The two networks need to be aligned: the latent causes proposed by the recognition model must be compatible with what the generative model would predict; otherwise there is a mismatch that raises free energy.
- The brain can loop between recognition and generation (inference and generation) to refine the explanation until free energy is minimized.
- Inference is approximate, but the loop with learning gradually aligns the two models so that the recognition network can provide good initial guesses that the generative model can refine.
> > "the brain has a separate Network a recognition model which works in the opposite direction by mapping a sensory observation to the distribution of causes... this result is only an approximation" (00:12:18)
- The alignment of recognition and generative networks is learned through experience; over time, perception becomes faster and learning produces better priors and a better-generating map.
- The overall picture is: perception arises from an iterative interaction where the recognition model proposes latent explanations, the generative model tests them, and both adapt to minimize free energy.
14:14 Free Energy balance revisited
- The perception-learning system has two time scales: rapid perceptual inference (fast loops between recognition and generation) and slower learning (adjusting synaptic weights to improve priors and the generator’s accuracy).
- Both scales serve the same ultimate goal: reduce uncertainty about the world by building and refining optimal models of the environment.
- The two-model loop provides a mechanistic explanation for how the brain continuously updates its beliefs while remaining robust to noisy or ambiguous input.
- The balance between the two networks ensures that perception remains coherent with prior knowledge while staying adaptable to new observations.
16:34 Explanation for optical illusion
- The mask illusion is used as a concrete demonstration: the retinal pattern might suggest a concave or a convex shape, but a strong prior that faces are convex biases interpretation toward a convex face.
- This creates a conflict: the sensory data might imply one interpretation, but the prior about faces being convex pushes toward another.
- From the Free Energy perspective, the brain chooses the explanation that minimizes free energy even if the sensory data is consistent with the alternative; it accepts a small mismatch with evidence if it preserves a strong prior.
- The explanation is rooted in evolved, robust circuitry for visual perception; even when conscious reasoning knows the truth, the prediction-driven interpretation can remain robust due to the lower free energy of the consistent interpretation.
> > "the brain chooses the explanation which minimizes total free energy... the prior belief about faces being convex is incredibly strong" (00:16:34)
- This illustrates why perceptual illusions endure: higher-level priors can dominate sensory input to create a stable, predictable interpretation at the cost of occasionally misrepresenting reality.
17:55 Review
- Core thesis: brains are prediction machines that minimize a quantity called free energy by maintaining a generative model (how the world could produce sensory data) and a recognition model (how to infer latent causes from data).
- Free energy quantifies the tradeoff between how well the data are explained and how complex the model is (how many latent assumptions are needed).
- Perception emerges from fast, iterative inference between prediction generation and data-driven correction, plus slower but steady learning to improve the models and priors.
- The text teases a deeper mathematical treatment to come in future videos, linking FEP to modern machine learning and potential AI systems that perceive and learn like brains.
## Resources Mentioned
- Bogacz, Rafal. “A Tutorial on the Free-Energy Framework for Modelling Perception and Learning.” Journal of Mathematical Psychology 76 (February 2017): 198–211. https://doi.org/10.1016/j.jmp.2015.11.003.
- Friston, Karl. “Learning and Inference in the Brain.” Neural Networks 16, no. 9 (November 2003): 1325–52. https://doi.org/10.1016/j.neunet.2003.06.005.
- Friston, Karl, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, and Giovanni Pezzulo. “Active Inference and Learning.” Neuroscience & Biobehavioral Reviews 68 (September 2016): 862–79. https://doi.org/10.1016/j.neubiorev.2016.06.022.
- Parr, Thomas, Giovanni Pezzulo, and K. J. Friston. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. Cambridge, Massachusetts: The MIT Press, 2022.
00:01:34 Role of world models (Expanded)
- The world model acts as a structured hypothesis about what could be out there. It enables the brain to fill in missing information (occluded objects, partial views) by predicting hidden causes that would generate the observed pattern.
- This predictive capacity is what makes perception efficient: instead of storing every possible sensory pattern, the brain stores a compact, probabilistic model of how causes map to data.
> > "your brain is constantly trying to find the perfect balance between those two forces" (00:01:34)
00:03:56 Free Energy as tradeoff between accuracy and complexity (Expanded)
- Free energy couples two ideas: accuracy (how well the generative model can reproduce actual sensory data) and complexity (how intricate or unlikely the chosen latent configuration is).
- Lower free energy means a better explanation with respect to both data fit and parsimony; higher free energy means a worse match to either the data or the priors (or both).
- The practical upshot: perception tends to settle on explanations that are both plausible and simple relative to the brain’s prior expectations.
> > "Free energy is a measure that balances fit to the data with the complexity of the explanation" (00:03:56)
00:05:20 Sponsor: Squarespace (Expanded)
- The sponsor talk emphasizes design intelligence and AI-assisted generation for creating websites and domains, tying back to the topic of models and predictions in both biology and technology.
- It highlights how modern tools can help you quickly realize a model of your online presence, analogous to how brains quickly form and refine models of the world.
00:06:35 Generative Model (Expanded)
- The brain’s generative model is a network that can generate sensory predictions from latent causes.
- Latent neurons encode abstract causes; the generator network translates latent representations into predicted sensory outputs; a reverse mapping (inference) approximates which latent state produced observed data.
- The realist example uses a three-stage generative structure: latent causes (e.g., object identity), intermediate features (e.g., stripes, fur texture), and basic perceptual elements (edges, color).
- The “blender” analogy: a scene controlled by a few sliders yields a high-dimensional image; real-world data show that a small number of parameters captures most variation—this is what latent representations exploit.
- The brain’s priors shape what latent configurations are plausible, guiding the inference to commonly observed patterns (e.g., lighting from above, stability on surfaces).
> > "the solution found by Evolution was quite elegant alongside neurons that directly correspond to sensory inputs the brain evolved neurons that don't directly connect to the outside world these hidden or latent neurons learn to represent meaningful features" (00:06:35)
00:10:03 Priors (Expanded)
- Priors encode the expected likelihood of various latent causes before observing data.
- They are learned from environmental statistics and personal experience; priors influence how ambiguous sensory data are interpreted.
- The example contrasts park vs safari priors: in a park, a striped orange object in peripheral vision might be a toy or clothing; in a safari context, a tiger becomes a more likely interpretation given the same visual input.
- Priors help disambiguate perception by biasing towards plausible explanations given the context.
00:12:18 Approximate Inference via Recognition Model (Expanded)
- Exact inversion of the generative model is computationally infeasible; the brain uses a recognition model to propose latent causes quickly from data.
- The recognition model yields a rough distribution over latent states; then the generative model refines this by simulating the data those causes would produce.
- The two models must be trained to speak the same language of causes; misalignment increases free energy and is corrected through learning.
- The loop can be repeated to improve the estimate: recognition proposes, generation tests, adjust, repeat until a low-free-energy explanation is found.
> > "the recognition model... maps a sensory observation to the distribution of causes... this result is only an approximation" (00:12:18)
00:14:14 Free Energy balance revisited (Expanded)
- Perception and learning differ in time scale: perception is fast (milliseconds to seconds), learning is slow (days to weeks), but both contribute to minimizing free energy.
- The two processes are complementary: perception adapts quickly to new data, learning updates priors and the generator to better reflect the world over longer periods.
- The unified view explains how short-term perception can be robust to noisy inputs while long-term learning improves the priors to reduce ambiguity in future similar situations.
00:16:34 Explanation for optical illusion (Expanded)
- The mask illusion is a concrete demonstration of predictive processing: a pattern on the retina could indicate a concave or convex face, but the brain’s prior that faces are convex biases interpretation toward convexity.
- A plausible explanation that preserves priors minimizes free energy even if the sensory data might support the alternate interpretation (concave).
- This reveals how deeply entrenched priors can override sensory evidence, leading to persistent perceptual illusions.
> > "the brain chooses the explanation which minimizes total free energy the prior belief about faces being convex is incredibly strong... the brain would rather assume there is something unusual about lighting than to accept the existence of an inwardly protruding face" (00:16:34)
00:17:55 Review (Expanded)
- Recap of the central thesis: brains are prediction machines integrating a generative model (production of predictions) and a recognition model (inference from data) to minimize free energy.
- The balance between data fit and prior assumptions governs both moment-to-moment perception and longer-term learning.
- Acknowledgement that there is a deeper mathematical underpinning to Free Energy Principles; future videos may connect FEP to modern machine learning and AI systems that perceive and learn like brains.
Resources Mentioned (Expanded)
- Bogacz, Rafal. "A Tutorial on the Free-Energy Framework for Modelling Perception and Learning." Journal of Mathematical Psychology 76 (February 2017): 198–211. https://doi.org/10.1016/j.jmp.2015.11.003.
- Friston, Karl. "Learning and Inference in the Brain." Neural Networks 16, no. 9 (November 2003): 1325–52. https://doi.org/10.1016/j.neunet.2003.06.005.
- Friston, Karl, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, and Giovanni Pezzulo. "Active Inference and Learning." Neuroscience & Biobehavioral Reviews 68 (September 2016): 862–79. https://doi.org/10.1016/j.neubiorev.2016.06.022.
- Parr, Thomas, Giovanni Pezzulo, and K. J. Friston. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. Cambridge, Massachusetts: The MIT Press, 2022.
00:01:34 Summary of the Core Idea (Reinforcement)
- The brain’s perception is an act of constructing plausible interpretations that minimize surprise given learned priors.
- Predictive processing, free energy, and Bayesian-style inference provide a unifying explanation for a wide range of cognitive phenomena, including illusions, learning, and adaptive behavior.
> > "the Free Energy Principle suggests that our brains are essentially prediction machines constantly trying to explain the chaos of incoming sensory information" (00:01:34)
00:10:03 Final Notes on Mechanisms (Reinforcement)
- The generative and recognition networks operate in tandem, continuously adapting to maintain low free energy.
- Perception is fast because recognition gives a confident initial hypothesis; perception improves with feedback from the generative model, bringing it into agreement with actual data.
- Learning consolidates priors and the generative mapping to reduce future free energy and improve efficiency in inference.
> > "the compressed representation of sensory pattern that balances observed evidence with prior beliefs" (00:01:34)
00:17:55 Takeaways for Researchers and Students
- If you want to study the brain as a prediction engine, focus on how latent causes map to sensory data (generative model) and how data constrain or revise latent explanations (recognition model).
- Consider how priors shape perception in real-world contexts and how illusions reveal the sometimes dominant role of priors over sensory data.
- Explore the iterative loop between inference and generation as a foundational mechanism for perception and learning.
> > "Future videos we'll explore this deeper mathematical foundation and see how it connects to modern machine learning" (00:17:55)
Resources Mentioned (Short-form)
- Bogacz, Rafal. A Tutorial on the Free-Energy Framework for Modelling Perception and Learning. J Math Psych. 2017. doi:10.1016/j.jmp.2015.11.003
- Friston, Karl. Learning and Inference in the Brain. Neural Networks. 2003. doi:10.1016/j.neunet.2003.06.005
- Friston, Karl et al. Active Inference and Learning. Neuro Sci Biobehav Rev. 2016. doi:10.1016/j.neubiorev.2016.06.022
- Parr, Thomas; Pezzulo, Friston. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press, 2022
00:00:00 Summary of the Video’s Core Idea
- The Free Energy Principle posits that brains construct and refine internal models to predict sensory input, thereby minimizing a free energy measure that trades off accuracy and complexity.
- Perception arises from the interaction of a generative model (how data could be produced) and a recognition model (how data map to latent causes), refined through experience and context-driven priors.
- Optical illusions like the convex/concave mask provide vivid demonstrations of how strong priors can override sensory input to minimize free energy.
- The mathematics (not covered in depth here) promises a bridge to machine learning and AI systems that learn to perceive and predict in human-like ways.
> > "we're going to explore Free Energy Principle – a powerful framework in neuroscience, which aims to explain brain function as predicting sensory observations through building models of the world" (00:00:00)
00:01:34 Final Note on Terminology
- Free Energy Principle (FEP): a unifying framework that explains brain function as predictive coding to minimize a quantity called free energy, balancing data fit and model complexity.
- Generative Model: the brain’s internal mapping from latent causes to predicted sensory data.
- Recognition Model: the brain’s fast, approximate inference mapping from sensory data to probable latent causes.
- Priors: beliefs about what causes are likely, learned from the environment.
- Variational Inference: a method of approximating complex probability distributions; in the brain, it underpins approximate inference via the recognition model.
00:15:00 Practical Takeaways
- When thinking about perception, consider how priors influence interpretation; what seems obvious may be a predictable consequence of the brain’s learned model.
- In designing AI or educational tools, consider building explicit generative and recognition components that learn to predict and explain sensory data, and use priors to guide efficient learning.
> > "perception and learning operate on two different time scales they both serve the same overarching goal reducing uncertainty in the world by building optimal models of the environment" (00:15:00)
Resources Mentioned (Expanded)
- Bogacz, Rafal. A Tutorial on the Free-Energy Framework for Modelling Perception and Learning. Journal of Mathematical Psychology 76 (February 2017): 198–211. https://doi.org/10.1016/j.jmp.2015.11.003.
- Friston, Karl. Learning and Inference in the Brain. Neural Networks 16, no. 9 (November 2003): 1325–52. https://doi.org/10.1016/j.neunet.2003.06.005.
- Friston, Karl, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, and Giovanni Pezzulo. Active Inference and Learning. Neuroscience & Biobehavioral Reviews 68 (September 2016): 862–79. https://doi.org/10.1016/j.neubiorev.2016.06.022.
- Parr, Thomas, Giovanni Pezzulo, and K. J. Friston. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. Cambridge, Massachusetts: The MIT Press, 2022.
00:16:34 Key Takeaway on Illusions (Verbatim)
> > "the brain chooses the explanation which minimizes total free energy... the prior belief about faces being convex is incredibly strong" (00:16:34)
00:17:55 Closing Thought
- The video invites viewers to anticipate a deeper dive into the mathematics of FEP and its connections to modern machine learning, signaling a path toward AI systems that can perceive, predict, and autonomously build their own world models.
> > "Future videos we'll explore this deeper mathematical foundation and see how it connects to modern machine learning" (00:17:55)
00:01:34 Glossary (quick reference)
- Free Energy Principle (FEP): A theory that the brain minimizes a quantity (free energy) to explain perception and learning, balancing data fit with model complexity.
- Generative Model: A model that outputs data given latent causes, representing how the world could produce observations.
- Recognition Model: A model that infers latent causes from observed data, used to approximate the posterior distribution over latent states.
- Priors: Pre-existing beliefs about what is likely, guiding interpretation of sensory data.
- Latent Variables: Hidden variables representing abstract causes not directly observed.
- Variational Inference: An optimization approach to approximate complex probability distributions; in brains, an approximate inference mechanism.
- Active Inference: A broader term describing how agents act to minimize free energy by affecting the world to confirm predictions.
Figures/Diagrams (to Recreate in Notes)
- A three-level generative chain: latent causes -> intermediate features -> basic perceptual elements (edges/colors).
- A blender-like scene with sliders: three latent sliders controlling a high-dimensional image; this illustrates how a small set of parameters can govern complex data via a generative map.
- The mask illusion: a flat 2D retinal image that can be interpreted as convex or concave, with priors biasing toward convex faces.
Notable analogies/stories (Explanations of Why Used)
- The tiger example demonstrates how priors can override ambiguous data if a certain cause (tiger) is plausible given context, yet the world model still has to explain the data with low free energy.
- The blender scene analogy clarifies how latent parameters can encode complex data efficiently, illustrating the brain’s use of latent representations to compress information.
Visual/Tooling Notes
- The mask illusion shows a geometry-based illusion where lighting and shading can mislead a well-formed priors-driven interpretation.
- The Squarespace sponsor block uses a practical example of a model-based approach to build and deploy a product (a website) quickly, paralleling the brain’s rapid deployment of predictive models in the real world.
3–6 Tags (Topics)
- Free Energy Principle
- Brain Function
- Predictive Processing
- Generative Model
- Recognition Model
- Perception and Illusion
10–15 Key Takeaways
- The brain is fundamentally a prediction engine, not a passive receiver of sensory data.
- Perception arises from the alignment of a generative model (how data could be produced) with a recognition model (how to infer causes from data).
- Priors are learned beliefs about what causes are likely, shaping interpretation of sensory data.
- Inference is approximate and often iterative, involving loops between proposing latent causes and testing them against sensory consequences.
- Free energy is a unifying objective that balances accuracy (data fit) and complexity (model simplicity).
- Two networks (recognition and generative) must be aligned to efficiently reduce free energy.
- Perception operates on fast time scales; learning updates priors and mappings on slower scales.
- Illusions reveal the power and limits of priors in shaping perception.
- The brain’s approach supports robust interpretation in noisy, ambiguous environments.
- The mathematics behind FEP, while complex, offers a bridge to advanced machine learning approaches.
- The future of AI may involve similar dual-model architectures that both predict and infer to minimize uncertainty.
Key Quotes (verbatim excerpts)
- > "this video was brought to you by Squarespace take a look at this mask it looks like a convex face protruding outwards now let's rotate it" (00:00:00)
- > "the brain isn't passively receiving information about the world it is actively generating predictions about what should be out there" (00:00:00)
- > "the brain is constantly trying to find the perfect balance between those two forces" (00:01:34)
- > "the brain has two possible explanations either that there is a concave face in front of you or that there is normal convex face with somewhat unusual lighting" (00:16:34)
- > "Future videos we'll explore this deeper mathematical foundation and see how it connects to modern machine learning" (00:17:55)
Resources Mentioned (Explicitly)
- Bogacz, Rafal. “A Tutorial on the Free-Energy Framework for Modelling Perception and Learning.” Journal of Mathematical Psychology 76 (February 2017): 198–211. https://doi.org/10.1016/j.jmp.2015.11.003.
- Friston, Karl. “Learning and Inference in the Brain.” Neural Networks 16, no. 9 (November 2003): 1325–52. https://doi.org/10.1016/j.neunet.2003.06.005.
- Friston, Karl, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, and Giovanni Pezzulo. “Active Inference and Learning.” Neuroscience & Biobehavioral Reviews 68 (September 2016): 862–79. https://doi.org/10.1016/j.neubiorev.2016.06.022.
- Parr, Thomas, Giovanni Pezzulo, and K. J. Friston. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. Cambridge, Massachusetts: The MIT Press, 2022.
Action Items
- Study the definitions of generative and recognition models and how they interact to minimize free energy.
- Read the referenced papers to understand the formal math behind FEP (see Resources Mentioned).
- Experiment with simple generative models (e.g., a small latent-variable VAE) to observe how priors influence inference.
- Create a mental model of perception: map a sensory input to a plausible latent cause and test via a generator.
- Explore perceptual illusions to illustrate how priors can dominate sensory data; consider designing your own simple illusion experiment.
- If building AI, implement a dual-model architecture where a recognition model provides fast-inference hints and a generative model validates and refines them.
- Track time scales: separate your own fast perceptual judgments from slower belief updates (learning) to study their roles in adaptation.
Mental Models
Glossary
- Free Energy Principle (FEP)
- A theoretical framework proposing that brains minimize a quantity called free energy, which trades off data fit against model complexity to explain perception and learning.
- Generative Model
- A model that describes how latent causes produce observable data; in the brain, it maps hidden causes to predicted sensory input.
- Recognition Model
- A model that infers latent causes from observed data; provides fast, approximate estimates of what latent states could have generated the input.
- Priors
- Pre-existing beliefs about what causes are likely, learned from environment and experience, used to bias interpretation of data.
- Latent Variables
- Hidden factors that influence observable data but are not directly observed; encode abstract causes in a hierarchical model.
- Variational Inference
- An approach to approximate complex probability distributions; in the brain, it refers to approximate posterior inference via recognition and generation loops.
- Active Inference
- An extension of FEP that includes action selection to minimize expected free energy by changing the world to better match predictions.
Notable Quotes
“"this video was brought to you by Squarespace take a look at this mask it looks like a convex face protruding outwards now let's rotate it at this point you know what you expect to see a concave mask protruding inwards however somehow you get a strong sense that something is off"”
“"the brain isn't passively receiving information about the world it is actively generating predictions about what should be out there"”
“"the brain is constantly trying to find the perfect balance between those two forces"”
“"the brain has two possible explanations either that there is a concave face in front of you or that there is normal convex face with somewhat unusual lighting"”
“"Future videos we'll explore this deeper mathematical foundation and see how it connects to modern machine learning"”
Topics
More from Polymath
See all




