Warm-up: Normal Distribution PDF and CDF

PDF $f(x)$: blue curve; blue shaded area = $\Phi(x)$

CDF $\Phi(x)$: green curve; marker shows $\Phi(x)$

Tip: Click on either plot to set cursor $x$. Range: $x\in[\mu-4\sigma,\,\mu+4\sigma]$.

$$ f(x) = \frac{1}{\sqrt{2\pi}\,\sigma}\,\exp\!\Big( -\frac{(x-\mu)^2}{2\sigma^2} \Big),\qquad \Phi(x) = \int_{-\infty}^{x} f(t)\,\mathrm{d}t $$

Current: $\mu$=0.0, $\sigma$=1.0; $x$=0.0, $\Phi(x)$=0.5

Part 1: Gaussian Processes · Interactive Demo

Left-click to add points, right-click to remove nearest; supports prior sampling and regression posterior

Mode

Kernel

Signal amplitude σ_f

Length-scale ℓ

Observation noise σ_n

Number of prior samples

Plot resolution (points)

Tip: Left-click to add a point, right-click to remove nearest; if preview does not render, open this page in a system browser.

Prior samples

Posterior mean

±2σ credible band

Observations

Default range x ∈ [-5, 5]; y-axis auto-adjusts to curves and bands.

Gaussian Processes: Formulas and Notes

For any finite input set, prior and regression posterior:

$$ f(\mathbf{X}) \sim \mathcal{N}\big(m(\mathbf{X}),\, K(\mathbf{X},\mathbf{X})\big) $$

$$ \begin{aligned} \mu(\mathbf{X}_*) &= K(\mathbf{X}_*,\mathbf{X})\,\big[K(\mathbf{X},\mathbf{X})+\sigma_n^2 I\big]^{-1}\,\mathbf{y}\\ \Sigma(\mathbf{X}_*) &= K(\mathbf{X}_*,\mathbf{X}_*) - K(\mathbf{X}_*,\mathbf{X})\,\big[K(\mathbf{X},\mathbf{X})+\sigma_n^2 I\big]^{-1}\,K(\mathbf{X},\mathbf{X}_*) \end{aligned} $$

Parameters $\sigma_n$=0.15

Common kernels (matrix elements $K_{ij} = k(x_i, x_j)$:

RBF/Gaussian: $ k(x,x') = \sigma_f^2\,\exp\!\big( -\tfrac{(x-x')^2}{2\,\ell^2} \big) $ $\sigma_f$=1.0 $\ell$=1.2
Matérn 3/2: $ k = \sigma_f^2\,\big(1 + \tfrac{\sqrt{3}\,r}{\ell}\big)\,\exp\!\big(-\tfrac{\sqrt{3}\,r}{\ell}\big),\; r=|x-x'| $ $\sigma_f$=1.0 $\ell$=1.2
Periodic: $ k = \sigma_f^2\,\exp\!\big( -\tfrac{2\,\sin^2(\pi|x-x'|/p)}{\ell^2} \big) $ $\sigma_f$=1.0 $\ell$=1.2 $p$=3.0
Linear: $ k = \sigma_b^2 + \sigma_f^2\,x\,x' $ $\sigma_f$=1.0 $\sigma_b$=0.1

Part 2: Probit Likelihood (Classification)

Probit maps latent real $f$ to a binary class probability via the standard normal CDF $\Phi(\cdot)$:

$$ p(y=1\mid f)=\Phi(f),\qquad p(y=0\mid f)=1-\Phi(f) $$

Equivalent form (recode labels as $t=2y-1\in\{-1,+1\}$):

$$ p(t\mid f)=\Phi(tf) $$

Key concepts:

Link function: $\Phi(z)$ is the standard normal CDF, monotonic S-shaped, mapping reals to $[0,1]$.
Latent function: $f(x)$ indicates signed distance/confidence to the decision boundary; $f\!>\!0$ favors $y=1$; larger $|f|$ means higher confidence.
Decision boundary: $p(y=1\mid f)=0.5$ corresponds to $f=0$.
Relation to kernels: Kernels impose prior smooth/periodic/linear structure on $f$, shaping how classification probability varies with $x$.

$$ z = f + \varepsilon,\; \varepsilon\sim\mathcal{N}(0,1),\quad y=\mathbf{1}[z>0] \Rightarrow p(y=1\mid f)=\Phi(f) $$

Interactive Probit demo: place 0/1 labels, adjust kernel and hyperparameters, and see $p(y=1\mid x)$ update in real time.

Kernel

Signal amplitude $\sigma_f$

Length-scale $\ell$

Plot resolution (points)

Add label

y=1 y=0

Latent f display Show mean of f Show ±2σ band of f

Interaction: Left-click to add a point (using selected label), right-click to remove nearest.

Probability curve $p(y=1\mid x)$

y=1 samples

y=0 samples

Threshold view: $z=f+\varepsilon,\ \varepsilon\sim\mathcal{N}(0,1),\ y=\mathbf{1}[z>0]\Rightarrow p(y=1\mid f)=\Phi(f)$. Bernoulli view: $y\mid f\sim\mathrm{Bernoulli}(\Phi(f))$.