Batch processing stops the lesson from lying

A single example is a good place to learn a neuron. It is a poor place to decide what the neuron does.

With one input, the arithmetic fits in your head: multiply, add the bias, apply ReLU, inspect the result. That small loop is useful because every number has somewhere obvious to go. It also hides the model’s shared behavior. A weight can look harmless. A hidden unit can look permanently useful. A bias can look like an afterthought.

Three inputs are enough to expose those assumptions.

one network, several columns

The operation is not different when examples are grouped together. The data shape is.

z = W x + b

For a batch, x becomes a matrix:

Z = W X + b

The uppercase X carries the important idea. The network reuses the same weights and biases for every column. This is one network answering several inputs, not three separately tuned networks.

That reuse is easy to miss when you study only one result. Batching makes it visible in the calculation itself.

activation is a pattern, not a verdict

Imagine one hidden row produces:

[-3, 2, 5]

ReLU changes it to:

[0, 2, 5]

The neuron is not simply “on” or “off.” It is off for the first input and active for the other two. The same rule produces different behavior because the inputs interact with the same parameters differently.

This is a better way to think about a hidden unit: it is a weighted test whose result may survive the gate for some examples and disappear for others.

the bias belongs to the rule

Bias is easy to understate in a one-example walkthrough. It looks like a number added at the end.

Across a batch, its role is harder to miss. One bias value shifts the pre-activation for that neuron in every column. If two inputs are near zero and a third is far below it, a small bias change may activate the first two while leaving the third quiet.

Try that in the interactive grid. Before changing the value, predict which columns should cross zero. The useful part is not getting the prediction right every time. It is seeing that the bias moves a decision boundary shared by the examples.

shared weights create shared consequences

W1, b1, W2, and b2 do not change from one column to the next. That is what separates a model from a lookup table. A parameter has to work across inputs that are related without being identical.

The same constraint is what makes batching relevant to training. A parameter update is informed by several examples, and those examples may pull in different directions. The model cannot improve one column by quietly assigning that column its own weights.

You do not need the full backpropagation story to see this. Change one weight and watch the effect appear across the batch.

use the grid as a small experiment

Run the interactive example with a prediction in mind:

• Which hidden values will be positive before ReLU?
• Which columns should move when one weight changes?
• Which outputs should change when a hidden value crosses zero?
• What did the result reveal about your assumption?

The batch is not there to make the arithmetic look more advanced. It prevents one convenient example from pretending to describe the whole network.