Introduction to ML

Reinforcement learning (RL): RL is a third paradigm sitting outside the supervised/unsupervised split. Rather than learning from a fixed dataset, an RL agent generates its own data by taking actions in an environment, observing the resulting state, and receiving a scalar reward. The key challenge is that rewards are delayed and sparse — the agent must figure out which past actions deserve credit for a good outcome. Bayesian optimization, which you will use in this course, is a close relative: the GP acts as a world model, the acquisition function acts as a policy, and each experiment is one decision step.

Self-supervised learning: Self-supervised learning is a way of obtaining a training signal from unlabeled data by constructing a pretext task from the data itself. A model might be trained to predict a masked region of a spectrum, reconstruct a corrupted image, or identify whether two views of the same sample are similar or different. No human annotation is needed — the labels are generated automatically from the data structure. The representations learned this way often transfer remarkably well to downstream supervised tasks, which is why large foundation models for science (trained on millions of spectra, structures, or simulation outputs) are built this way. For materials science, self-supervised pre-training on large unlabeled characterization datasets before fine-tuning on a small labeled set is an increasingly practical workflow.

Active learning: Active learning sits at the boundary of supervised and unsupervised: a model exists and can make predictions, but instead of passively consuming a fixed labeled dataset, it queries an oracle — typically a human annotator or an instrument — for the label of the most informative unlabeled point. The goal is to reach a target accuracy with as few labeled examples as possible. This is the intellectual home of gpCAM: the GP posterior tells you where uncertainty is high, and the acquisition function decides which measurement would reduce that uncertainty most efficiently. Active learning differs from RL in that the underlying function is assumed fixed and there is no notion of sequential state.

Transfer learning and fine-tuning: Transfer learning exploits the fact that representations learned on one task often encode structure useful for a related task. A model pre-trained on a large general dataset can be fine-tuned on a small domain-specific dataset at a fraction of the cost of training from scratch. In materials science this appears in two forms: fine-tuning image models pre-trained on ImageNet for electron microscopy analysis, and adapting large language models or chemistry foundation models (e.g. trained on the Materials Project or PubChem) to specific prediction tasks. The key question is always how well the source and target distributions overlap — negative transfer, where pre-training actively hurts, is a real failure mode.

Semi-supervised learning: Semi-supervised learning addresses the common experimental situation where you have a large pool of unlabeled measurements and a small number of expensive labeled ones. The model is trained jointly on both: the labeled data provides direct supervision, while the unlabeled data provides structural constraints — the assumption being that nearby points in input space should have similar labels. In practice, modern self-supervised pre-training followed by supervised fine-tuning has largely superseded classical semi-supervised methods, but the underlying intuition remains important for understanding when and why either approach works.