Concept study / Information & adaptationConceptual illustration. Shapes, colors, and motion are not measured data.
If knowing one variable changes the uncertainty about another, the variables share statistical information. A temperature reading may inform a prediction of a nearby sensor; a stored state may inform a guess about an earlier input. Mutual information turns that dependence into a number once the joint probability distribution is defined. This is useful for Hill’s memory question because it avoids relying only on visual resemblance. However, the number inherits the choice of variables and measurement procedure. A high value can result from a shared input, duplicated data, or selection effects. It is therefore a measure to interpret, not a verdict about mechanism.
A clear use for physical memory
Label a set of training histories and measure the later states produced by each. Mutual information between the history label and measured state can quantify how distinguishable those histories remain. A label that encodes more detail than the system could plausibly retain may complicate the analysis without adding insight. We propose starting with a small set of physically meaningful alternatives and checking whether a blinded decoder reaches the same practical conclusion. The measured relationship should be reported alongside the actual decoding error.
Finite data can create false structure
An estimator can report apparent dependence when the available sample is small relative to the number of possible states. We propose a mandatory shuffled-label baseline, repeated samples, and a held-out evaluation for any learned decoder. Time-series data need extra care because adjacent observations may not be independent. The report should show how results change with lag, binning, and sample size. A convincing effect should be larger than estimator artifacts and should survive reasonable choices that were not selected to maximize the result.
Predictive information without mind reading
Bialek, Nemenman, and Tishby studied predictive information as dependence between past and future observations. This gives the project a quantitative way to ask which historical features matter for later dynamics. It does not establish that the studied system consciously anticipates events. We propose comparing the measured dependence with a model that has ordinary dynamics and no learning rule. If that baseline explains the data, a predictive relationship exists, but the added claim of adaptive learning has not been demonstrated.
The mathematical lens
Discrete mutual information in bits; estimation requires a justified joint distribution.
Turn it into a question.
- Question
- Is the measured dependence larger than sampling and selection artifacts?
- Method
- Use shuffled controls, held-out decoding, and prespecified lag and binning analyses.
- What would count against it?
- An effect matching shuffled data or vanishing out of sample is not reliable evidence of retained information.
Dependence is measurable; causation and function require more work.
Follow the original work.
These sources support the mechanisms and limits discussed here. They do not endorse Hill’s framework. Notes distinguish research proposals, reviews, and observations.
Continue the investigation
Edition 02 · 18 October 2026 · AI-assisted synthesis. See editorial method.