The supplement explains how hierarchical Bayesian models account for transfer learning: a learner can use related past tasks to infer higher-level constraints that make future tasks easier.

The key idea is the blessing of abstraction. Abstract knowledge may be harder to infer at first, but once learned it can strongly constrain many lower-level inferences and enable one-shot generalization.

Transfer learning is learning-to-learn in a concrete form. Instead of treating each task independently, the learner infers what related tasks have in common and transfers that inductive bias to a new task.

The marble-bag example shows how one observation can become informative only after the learner has discovered a higher-level regularity: each bag tends to contain one color, even though different bags may have different colors.

This higher-level regularity is an overhypothesis: a hypothesis about what kinds of hypotheses are likely in a domain.

A hierarchical Bayesian model has multiple levels of inference. Lower levels explain specific observations, while higher levels explain the structure shared across many related observations or tasks.

In the marble example, lower-level parameters describe the color distribution in one bag. Higher-level parameters describe the population of bags and how strongly each bag is expected to resemble that population.

A small concentration parameter makes one-shot learning possible because one local observation can dominate the inferred distribution for a new task.

Nonparametric hierarchical models allow the amount of structure to grow with the data. The learner does not need to know in advance how many categories, clusters, dimensions, or relations will be needed.

The Chinese Restaurant Process mixture model captures unsupervised category discovery: each new object can be assigned to an existing category or can justify creating a new category.

CrossCat extends this idea by discovering which variables belong together. It can learn that some features form one dependency group while other features form another.

Figures S1 and S2 illustrate transfer learning. They show how a learner can infer a high-level constraint from several related examples and then generalize strongly from one new example.

Figures S3 to S5 illustrate flexible representation learning. They emphasize that the learner can discover structure rather than merely fit parameters inside a fixed structure.

For a lifelong agent, the important lesson is that memory should not be a flat archive. Experience should be compressed into abstractions that become priors for future learning.

The paper supports a developmental architecture in which specific episodes feed task models, task models feed domain abstractions, and domain abstractions feed faster future learning.