← main page
1. Random Initialization
01a · Random networks
01b · Two images
width
depth
σ
ReLU
GELU
tanh
erf
linear
biases
randomly initialized:
trials
show NNGP theory
input coordinate
\(f(\mathbf{x})\) at init
count of \(f(\mathbf{x}_*)\)
\[ f(\mathbf{x}) = W_2\,\mathrm{ReLU}(W_1 \mathbf{x} + \mathbf{b}_1) \]
width
trials
show NNGP theory
\(\mathbf{x}_1\)
\(\mathbf{x}_2\)
sample two images
\(f(\mathbf{x}_1)\) at init
\(f(\mathbf{x}_2)\) at init