Alice
Project
Home
Simulation Replay
Worldview
Version Log
Mind Research
中文
🧠 Neuroscience
Foundations
Neuroanatomy
Cellular & Molecular Neuroscience
Systems Neuroscience
Cognitive Neuroscience
Computational Neuroscience
Brain-Computer Interface
Neurotech Frontiers
Neuro Disorders
Computational Neuroscience
Spiking Neural Network (SNN) — Neuromorphic Computing
Predictive Coding
Bayesian Brain
Hopfield Networks
Grid Cells — Neural Spatial Map
Reinforcement Learning in the Brain
Free Energy Principle
Attractor Networks
Neural Population Dynamics
Drift-Diffusion Model
Efficient Coding Hypothesis
Divisive Normalization
Synaptic Plasticity Models
Deep Learning vs Brain
Reservoir Computing
Compartmental Models
Whole-Brain Modeling
Information Theory in Neuroscience
Neural ODEs & Continuous-Time Models
Energy-Based Models & the Brain
Table of Contents
1. Basic Quantities
2. MI in Neural Coding
3. Estimation Methods + Bias
4. Spike Train Information
5. PyTorch — Mutual Information Estimation
6. Fisher Information
7. Information Bottleneck (Tishby)
8. Redundancy + Synergy
9. Relation to AI
10. Common Pitfalls
10.1 MI Estimate Unbiased
10.2 High MI = Brain Uses All
10.3 Entropy = Information (Semantic)
10.4 IB Explains DL Settled
10.5 More Bits = Better Coding
11. Related Concepts
References